Exploiting neighborhood knowledge for single document summarization and keyphrase extraction

• Published in: ACM transactions on information systems 2010-05, Vol.28 (2)
• Main Authors: Wan, Xiaojun, Xiao, Jianguo
• Format: Article
• Language: English
• Subjects: Conferences; Extraction; Information systems; Representations; Robustness; Sentences; Tasks; Texts
• ISSN: 1046-8188
• DOI: 10.1145/564376.564397

Document summarization and keyphrase extraction are two related tasks in the IR and NLP fields, and both of them aim at extracting condensed representations from a single text document. Existing methods for single document summarization and keyphrase extraction usually make use of only the information contained in the specified document. This article proposes using a small number of nearest neighbor documents to improve document summarization and keyphrase extraction for the specified document, under the assumption that the neighbor documents could provide additional knowledge and more clues. The specified document is expanded to a small document set by adding a few neighbor documents close to the document, and the graph-based ranking algorithm is then applied on the expanded document set to make use of both the local information in the specified document and the global information in the neighbor documents. Experimental results on the Document Understanding Conference (DUC) benchmark datasets demonstrate the effectiveness and robustness of our proposed approaches. The cross-document sentence relationships in the expanded document set are validated to be beneficial to single document summarization, and the word cooccurrence relationships in the neighbor documents are validated to be very helpful to single document keyphrase extraction.