Older versions of the ROUGEeval summarization evaluation system were easier to fool

We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however n...

Full description

Saved in:
Bibliographic Details
Published in:Information processing & management 2007-11, Vol.43 (6), p.1500-1505
Main Author: Sjöbergh, Jonas
Format: Article
Language:English
Subjects:
Automatic evaluation
Automatic summarization
Automation
Information processing
Information technology
Informationsteknik
Markov analysis
Markov models
Studies
TECHNOLOGY
TEKNIKVETENSKAP
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers. The method can be adapted to trick different settings of the ROUGEeval package.
ISSN:0306-4573
1873-5371
1873-5371
DOI:10.1016/j.ipm.2007.01.014