Corpus-based web document summarization using statistical and linguistic approach

Single document summarization generates summary by extracting the representative sentences from the document. In this paper, we presented a novel technique for summarization of domain-specific text from a single web document that uses statistical and linguistic analysis on the text in a reference co...

Full description

Saved in:
Bibliographic Details
Main Authors: Shams, R, Hashem, M M A, Hossain, A, Akter, S R, Gope, M
Format: Conference Proceeding
Language:English
Subjects:
Artificial neural networks
Feature extraction
Humans
Knowledge Extraction
Manuals
POS Tagging
Pragmatics
Subject Weight
Tagging
Text Summarization
Web Document Summarization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Single document summarization generates summary by extracting the representative sentences from the document. In this paper, we presented a novel technique for summarization of domain-specific text from a single web document that uses statistical and linguistic analysis on the text in a reference corpus and the web document. The proposed summarizer uses the combinational function of Sentence Weight (SW) and Subject Weight (SuW) to determine the rank of a sentence, where SW is the function of number of terms (t n ) and number of words (w n ) in a sentence, and term frequency (t f ) in the corpus and SuW is the function of t n and w n in a subject, and t f in the corpus. 30 percent of the ranked sentences are considered to be the summary of the web document. We generated three web document summaries using our technique and compared each of them with the summaries developed manually from 16 different human subjects. Results showed that 68 percent of the summaries produced by our approach satisfy the manual summaries.
DOI:10.1109/ICCCE.2010.5556854