Evaluating Temporal Graphs Built from Texts via Transitive Reduction

Temporal information has been the focus of recent attention in information extraction, leading to some standardization effort, in particular for the task of relating events in a text. This task raises the problem of comparing two annotations of a given text, because relations between events in a sto...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of artificial intelligence research 2011-01, Vol.40 (40), p.375-413
Main Authors: Tannier, X., Muller, P.
Format: Article
Language:English
Subjects:
Annotations
Artificial Intelligence
Computer Science
Evaluation
Information retrieval
Machine learning
Reduction
Standardization
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Temporal information has been the focus of recent attention in information extraction, leading to some standardization effort, in particular for the task of relating events in a text. This task raises the problem of comparing two annotations of a given text, because relations between events in a story are intrinsically interdependent and cannot be evaluated separately. A proper evaluation measure is also crucial in the context of a machine learning approach to the problem. Finding a common comparison referent at the text level is not obvious, and we argue here in favor of a shift from event-based measures to measures on a unique textual object, a minimal underlying temporal graph, or more formally the transitive reduction of the graph of relations between event boundaries. We support it by an investigation of its properties on synthetic data and on a well-know temporal corpus.
ISSN:1076-9757
1076-9757
1943-5037
DOI:10.1613/jair.3118