A study on methods for revising dependency treebanks: in search of gold

Reliably annotated corpora with reliable annotation are a valuable resource for Natural Language Processing, which justifies the search for methods capable of assisting linguistic revision. In this context, we present a study on methods for revising dependency treebanks, investigating the contributi...

Full description

Saved in:
Bibliographic Details
Published in:Language resources and evaluation 2024-03, Vol.58 (1), p.111-131
Main Authors: Freitas, Cláudia, de Souza, Elvis
Format: Article
Language:English
Subjects:
Annotations
Computational Linguistics
Computer Science
Computerized corpora
Corpus analysis
Corpus linguistics
Language and Literature
Linguistics
Methods
N-Gram language models
Natural language processing
Original Paper
Portuguese language
Revisions
Searching
Social Sciences
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Reliably annotated corpora with reliable annotation are a valuable resource for Natural Language Processing, which justifies the search for methods capable of assisting linguistic revision. In this context, we present a study on methods for revising dependency treebanks, investigating the contribution of three different strategies to the corpus review: (i) linguistic rules; (ii) an adaptation of the n-grams method proposed by Boyd et al. (2008) applied to Portuguese; and (iii) Inter-Annotator Disagreement, a linguistically motivated approach that draws inspiration from the human annotation process. The results are promising, and taken together the three methods can lead to the revision of up to 58% of the errors in a specific corpus at the cost of revising only 20% of the corpus. We also present a tool that integrates treebank editing, evaluation and search capabilities with the review methods, as well as a gold-standard Portuguese corpus from the oil and gas domain.
ISSN:1574-020X
1574-0218
DOI:10.1007/s10579-023-09653-4