Predicting skin sensitizers with confidence — Using conformal prediction to determine applicability domain of GARD

• Published in: Toxicology in vitro 2018-04, Vol.48, p.179-187
• Main Authors: Forreryd, Andy, Norinder, Ulf, Lindberg, Tim, Lindstedt, Malin
• Format: Article
• Language: English
• Subjects: Allergens; Applicability domain; Assaying; Basic Medicine; Bioassays; Biomarkers; Chemical compounds; Chemical reactions; Classification; Confidence intervals; Conformal prediction; Dendritic cells; Farmakologi och toxikologi; GARD; In vitro assay; Mathematical models; Medical and Health Sciences; Medicin och hälsovetenskap; Medicinska och farmaceutiska grundvetenskaper; Model accuracy; Organic chemistry; Performance prediction; Pharmacology and Toxicology; Skin; Skin sensitization; Skin tests; Test sets
• ISSN: 0887-2333; 1879-3177; 1879-3177
• DOI: 10.1016/j.tiv.2018.01.021

GARD — Genomic Allergen Rapid Detection is a cell based alternative to animal testing for identification of skin sensitizers. The assay is based on a biomarker signature comprising 200 genes measured in an in vitro model of dendritic cells following chemical stimulations, and consistently reports predictive performances ~90% for classification of external test sets. Within the field of in vitro skin sensitization testing, definition of applicability domain is often neglected by test developers, and assays are often considered applicable across the entire chemical space. This study complements previous assessments of model performance with an estimate of confidence in individual classifications, as well as a statistically valid determination of the applicability domain for the GARD assay. Conformal prediction was implemented into current GARD protocols, and a large external test dataset (n = 70) was classified at a confidence level of 85%, to generate a valid model with a balanced accuracy of 88%, with none of the tested chemical reactivity domains identified as outside the applicability domain of the assay. In conclusion, results presented in this study complement previously reported predictive performances of GARD with a statistically valid assessment of uncertainty in each individual prediction, thus allowing for classification of skin sensitizers with confidence. •GARD is an in vitro alternative for testing of chemical skin sensitizers.•Conformal prediction was implemented into GARD for determination of applicability domain.•Statistically valid assessment of uncertainty allows for classification of sensitizers with confidence.•A reliable and highly accurate in vitro testing strategy applicable across a large chemical space.