An Overview of BioCreative II.5

We present the results of the BioCreative II.5 evaluation in association with the FEBS Letters experiment, where authors created Structured Digital Abstracts to capture information about protein-protein interactions. The BioCreative II.5 challenge evaluated automatic annotations from 15 text mining...

Full description

Saved in:
Bibliographic Details
Published in:IEEE/ACM transactions on computational biology and bioinformatics 2010-07, Vol.7 (3), p.385-399
Main Authors: Leitner, F, Mardis, S A, Krallinger, M, Cesareni, G, Hirschman, L A, Valencia, A
Format: Article
Language:English
Subjects:
Abstracting and Indexing as Topic
Abstracts
biological curation
Computational Biology - methods
Data Collection - methods
Data mining
Data Mining - methods
Database Management Systems
Databases, Factual
Gold
Humans
Information Management - methods
molecular biology
Natural Language Processing
Ontologies
Organisms
Protein Interaction Mapping - classification
Proteins
Testing
text analysis
Text mining
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present the results of the BioCreative II.5 evaluation in association with the FEBS Letters experiment, where authors created Structured Digital Abstracts to capture information about protein-protein interactions. The BioCreative II.5 challenge evaluated automatic annotations from 15 text mining teams based on a gold standard created by reconciling annotations from curators, authors, and automated systems. The tasks were to rank articles for curation based on curatable protein-protein interactions; to identify the interacting proteins (using UniProt identifiers) in the positive articles (61); and to identify interacting protein pairs. There were 595 full-text articles in the evaluation test set, including those both with and without curatable protein interactions. The principal evaluation metrics were the interpolated area under the precision/recall curve (AUC iP/R), and (balanced) F-measure. For article classification, the best AUC iP/R was 0.70; for interacting proteins, the best system achieved good macroaveraged recall (0.73) and interpolated area under the precision/recall curve (0.58), after filtering incorrect species and mapping homonymous orthologs; for interacting protein pairs, the top (filtered, mapped) recall was 0.42 and AUC iP/R was 0.29. Ensemble systems improved performance for the interacting protein task.
ISSN:1545-5963
1557-9964
DOI:10.1109/TCBB.2010.61