Deep multi-task learning and random forest for series classification by pulse sequence type and orientation

• Published in: Neuroradiology 2023-01, Vol.65 (1), p.77-87
• Main Authors: Kasmanoff, Noah, Lee, Matthew D., Razavian, Narges, Lui, Yvonne W.
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial neural networks; Brain; Brain - diagnostic imaging; Classification; Classifiers; Complexity; Datasets; Diagnostic Neuroradiology; Humans; Identification methods; Imaging; Machine learning; Magnetic Resonance Imaging; Medical imaging; Medicine; Medicine & Public Health; Neural networks; Neural Networks, Computer; Neuroimaging; Neurology; Neuroradiology; Neurosciences; Neurosurgery; Orientation; Radiology; Random Forest; Sequences; State-of-the-art reviews
• ISSN: 0028-3940; 1432-1920
• DOI: 10.1007/s00234-022-03023-7

  Purpose Increasingly complex MRI studies and variable series naming conventions reveal limitations of rule-based image routing, especially in health systems with multiple scanners and sites. Accurate methods to identify series based on image content would aid post-processing and PACS viewing. Recent deep/machine learning efforts classify 5–8 basic brain MR sequences. We present an ensemble model combining a convolutional neural network and a random forest classifier to differentiate 25 brain sequences and image orientation. Methods Series were grouped by descriptions into 25 sequences and 4 orientations. Dataset A, obtained from our institution, was divided into training (16,828 studies; 48,512 series; 112,028 images), validation (4746 studies; 16,612 series; 26,222 images) and test sets (6348 studies; 58,705 series; 3,314,018 images). Dataset B, obtained from a separate hospital, was used for out-of-domain external validation (1252 studies; 2150 series; 234,944 images). We developed an ensemble model combining a 2D convolutional neural network with a custom multi-task learning architecture and random forest classifier trained on DICOM metadata to classify sequence and orientation by series. Results The neural network, random forest, and ensemble achieved 95%, 97%, and 98% overall sequence accuracy on dataset A, and 98%, 99%, and 99% accuracy on dataset B, respectively. All models achieved > 99% orientation accuracy on both datasets. Conclusion The ensemble model for series identification accommodates the complexity of brain MRI studies in state-of-the-art clinical practice. Expanding on previous work demonstrating proof-of-concept, our approach is more comprehensive with greater sequence diversity and orientation classification.