Towards systematic grammar profiling.Test suite technology 10 years after

An experiment with recent test suite and grammar (engineering) resources is outlined: a criticial assessment of the EU-fundedtsnlp(Test Suites for Natural Language Processing) package as a diagnostic and benchmarking facility for a distributed (multi-site) large-scalehpsggrammar engineering effort....

Full description

Saved in:
Bibliographic Details
Published in:Computer speech & language 1998-10, Vol.12 (4), p.411-435
Main Authors: Oepen, S, Flickinger, DP
Format: Article
Language:English
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An experiment with recent test suite and grammar (engineering) resources is outlined: a criticial assessment of the EU-fundedtsnlp(Test Suites for Natural Language Processing) package as a diagnostic and benchmarking facility for a distributed (multi-site) large-scalehpsggrammar engineering effort. This paper argues for a generalized, systematic, and fully automated testing and diagnosis facility as an integral part of the linguistic engineering cycle and gives a practical assessment of existing resources; both a flexible methodology and tools for competence and performance profiling are presented. By comparison to earlier evaluation work as reflected in the Hewlett-Packard test suite data, released exactly 10 years beforetsnlp, it is judged where test-suite-based evaluation has improved (and where not) over time.
ISSN:0885-2308
1095-8363
DOI:10.1006/csla.1998.0105