Classification uncertainty for transient gravitational-wave noise artefacts with optimised conformal prediction

With the increasing use of Machine Learning (ML) algorithms in scientific research comes the need for reliable uncertainty quantification. When taking a measurement it is not enough to provide the result, we also have to declare how confident we are in the measurement. This is also true when the res...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-12
Main Authors: Malz, Ann-Kristin, Ashton, Gregory, Colombo, Nicolo
Format: Article
Language:English
Subjects:
Algorithms
Astronomy
Classification
Gravitational waves
Machine learning
Noise measurement
Noise prediction
Optimization
Statistical methods
Uncertainty
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the increasing use of Machine Learning (ML) algorithms in scientific research comes the need for reliable uncertainty quantification. When taking a measurement it is not enough to provide the result, we also have to declare how confident we are in the measurement. This is also true when the results are obtained from a ML algorithm, and arguably more so since the internal workings of ML algorithms are often less transparent compared to traditional statistical methods. Additionally, many ML algorithms do not provide uncertainty estimates and auxiliary algorithms must be applied. Conformal Prediction (CP) is a framework to provide such uncertainty quantifications for ML point predictors. In this paper, we explore the use and properties of CP applied in the context of glitch classification in gravitational wave astronomy. Specifically, we demonstrate the application of CP to the Gravity Spy glitch classification algorithm. CP makes use of a score function, a nonconformity measure, to convert an algorithm's heuristic notion of uncertainty to a rigorous uncertainty. We use the application on Gravity Spy to explore the performance of different nonconformity measures and optimise them for our application. Our results show that the optimal nonconformity measure depends on the specific application, as well as the metric used to quantify the performance.
ISSN:2331-8422