Assessing the practice of biomedical ontology evaluation: Gaps and opportunities

• Published in: Journal of biomedical informatics 2018-04, Vol.80, p.1-13
• Main Authors: Amith, Muhammad, He, Zhe, Bian, Jiang, Lossio-Ventura, Juan Antonio, Tao, Cui
• Format: Article
• Language: English
• Subjects: Biological Ontologies; Biomedical ontologies; Electronic Health Records; Humans; Knowledge representation; Medical Informatics - standards; Ontology evaluation; Quality assurance; Quality Assurance, Health Care; Semantics
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2018.02.010

[Display omitted] •Ontology evaluation is an integral part of ontology development and maintenance.•We assessed the ontology evaluation practice of a sample of 200 BioPortal ontologies.•We reviewed recent ontology quality assurance and auditing techniques.•We identified the gaps between ontology evaluation and quality assurance. With the proliferation of heterogeneous health care data in the last three decades, biomedical ontologies and controlled biomedical terminologies play a more and more important role in knowledge representation and management, data integration, natural language processing, as well as decision support for health information systems and biomedical research. Biomedical ontologies and controlled terminologies are intended to assure interoperability. Nevertheless, the quality of biomedical ontologies has hindered their applicability and subsequent adoption in real-world applications. Ontology evaluation is an integral part of ontology development and maintenance. In the biomedicine domain, ontology evaluation is often conducted by third parties as a quality assurance (or auditing) effort that focuses on identifying modeling errors and inconsistencies. In this work, we first organized four categorical schemes of ontology evaluation methods in the existing literature to create an integrated taxonomy. Further, to understand the ontology evaluation practice in the biomedicine domain, we reviewed a sample of 200 ontologies from the National Center for Biomedical Ontology (NCBO) BioPortal—the largest repository for biomedical ontologies—and observed that only 15 of these ontologies have documented evaluation in their corresponding inception papers. We then surveyed the recent quality assurance approaches for biomedical ontologies and their use. We also mapped these quality assurance approaches to the ontology evaluation criteria. It is our anticipation that ontology evaluation and quality assurance approaches will be more widely adopted in the development life cycle of biomedical ontologies.