Benchmarking infrastructure for mutation text mining

Experimental research on the automatic extraction of information about mutations from texts is greatly hindered by the lack of consensus evaluation infrastructure for the testing and benchmarking of mutation text mining systems. We propose a community-oriented annotation and benchmarking infrastruct...

Full description

Saved in:
Bibliographic Details
Published in:Journal of biomedical semantics 2014-02, Vol.5 (1), p.11-11
Main Authors: Klein, Artjom, Riazanov, Alexandre, Hindle, Matthew M, Baker, Christopher Jo
Format: Article
Language:English
Subjects:
Annotations
Benchmarks
Business metrics
Business performance management
Comparative analysis
Data mining
Experimental research
Extensibility
Infrastructure
Interoperability
Knowledge representation
Mutation
Performance measurement
Programming languages
Proteins
Software
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Experimental research on the automatic extraction of information about mutations from texts is greatly hindered by the lack of consensus evaluation infrastructure for the testing and benchmarking of mutation text mining systems. We propose a community-oriented annotation and benchmarking infrastructure to support development, testing, benchmarking, and comparison of mutation text mining systems. The design is based on semantic standards, where RDF is used to represent annotations, an OWL ontology provides an extensible schema for the data and SPARQL is used to compute various performance metrics, so that in many cases no programming is needed to analyze results from a text mining system. While large benchmark corpora for biological entity and relation extraction are focused mostly on genes, proteins, diseases, and species, our benchmarking infrastructure fills the gap for mutation information. The core infrastructure comprises (1) an ontology for modelling annotations, (2) SPARQL queries for computing performance metrics, and (3) a sizeable collection of manually curated documents, that can support mutation grounding and mutation impact extraction experiments. We have developed the principal infrastructure for the benchmarking of mutation text mining tasks. The use of RDF and OWL as the representation for corpora ensures extensibility. The infrastructure is suitable for out-of-the-box use in several important scenarios and is ready, in its current state, for initial community adoption.
ISSN:2041-1480
2041-1480
DOI:10.1186/2041-1480-5-11