AN ACTIVE LEARNING ALGORITHM FOR EFFICIENT DEVELOPMENT OF EMULATORS OF COMPLEX MODELS, WITH AN APPLICATION IN PROSTATE CANCER SCREENING

• Published in: Value in health 2017-05, Vol.20 (5), p.A322
• Main Authors: Ellis, AG, Trikalinos, TA
• Format: Article
• Language: English
• Subjects: Algorithms; Cancer screening; Deviation; Emulation; Learning; Mathematical models; Medical screening; Metastatic seeding; Prostate; Prostate cancer; Prostate-specific antigen; Sampling; Simulators; Statistical analysis
• ISSN: 1098-3015; 1524-4733
• DOI: 10.1016/j.jval.2017.05.005

OBJECTIVES: Emulators are fast-to-evaluate statistical approximations of (typically computationally expensive) mathematical models (simulators). Using emulators in lieu of simulators can speed up computationally expensive analyses. Emulators are developed using the output of simulators at specified input parameter values (design points). Developing well-performing emulators can require many design points, which becomes computationally expensive. We describe an iterative active learning (AL) algorithm to efficiently develop emulators. We explicate by developing emulators for a prostate cancer screening simulator (PSAPC). METHODS: The AL algorithm starts with a seeding set of design points and sequentially chooses additional design points in regions where (1) the simulator output is fast-changing and (2) the emulator predictions' variance is maximized. We developed one- and two-dimensional Gaussian Process-based emulators of the PSAPC using the AL algorithm versus using current standards (Latin Hypercube Sampling [LHS]). The simulator output was mean life-years saved with prostate-specific antigen based screening versus no screening. We compared the accuracy of emulators' predictions by calculating the maximum difference between the emulator prediction and the PSAPC (lower is better) and the emulator's 95% prediction volume (lower is better). RESULTS: The median maximum deviation in life-days saved between emulator predictions and the PSAPC were comparable between the AL emulators (one dimension: 0.008 [range: 0.006-0.024]; two dimensions: 0.217 [range: 0.171-0.234]) and LHS emulators (0.012 [range: 0.002-0.038], 0.201 [range: 0.079-0.441], respectively). Compared with LHS, performance results with AL had smaller variance. Furthermore, the AL algorithm improved the emulators' accuracy 25% faster per additional design point. Results were comparable with the other metric. CONCLUSIONS: In the example, compared with emulators trained with LHS, emulators trained with AL (1) attain comparable accuracy; (2) have smaller variance in their performance; (3) improve their performance faster per additional design point. Efficiency gains may be greater in (well-behaved) larger-dimensional problems.