Limitations of Out-of-Distribution Detection in 3D Medical Image Segmentation

Deep learning models perform unreliably when the data come from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection methods help to identify such data samples, preventing erroneous predictions. In this paper, we furthe...

Full description

Saved in:
Bibliographic Details
Published in:Journal of imaging 2023-09, Vol.9 (9), p.191
Main Authors: Vasiliuk, Anton, Frolova, Daria, Belyaev, Mikhail, Shirokikh, Boris
Format: Article
Language:English
Subjects:
Abdomen
Ablation
anomaly detection
Benchmarks
computed tomography
COVID-19
Datasets
Design
Image segmentation
magnetic resonance imaging
Medical imaging
Methods
out-of-distribution detection
segmentation
Semantics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning models perform unreliably when the data come from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection methods help to identify such data samples, preventing erroneous predictions. In this paper, we further investigate OOD detection effectiveness when applied to 3D medical image segmentation. We designed several OOD challenges representing clinically occurring cases and found that none of the methods achieved acceptable performance. Methods not dedicated to segmentation severely failed to perform in the designed setups; the best mean false-positive rate at a 95% true-positive rate (FPR) was 0.59. Segmentation-dedicated methods still achieved suboptimal performance, with the best mean FPR being 0.31 (lower is better). To indicate this suboptimality, we developed a simple method called Intensity Histogram Features (IHF), which performed comparably or better in the same challenges, with a mean FPR of 0.25. Our findings highlight the limitations of the existing OOD detection methods with 3D medical images and present a promising avenue for improving them. To facilitate research in this area, we release the designed challenges as a publicly available benchmark and formulate practical criteria to test the generalization of OOD detection beyond the suggested benchmark. We also propose IHF as a solid baseline to contest emerging methods.
ISSN:2313-433X
2313-433X
DOI:10.3390/jimaging9090191