MIXCAPS: A capsule network-based mixture of experts for lung nodule malignancy prediction

• Published in: Pattern recognition 2021-08, Vol.116, p.107942, Article 107942
• Main Authors: Afshar, Parnian, Naderkhani, Farnoosh, Oikonomou, Anastasia, Rafiee, Moezedin Javad, Mohammadi, Arash, Plataniotis, Konstantinos N.
• Format: Article
• Language: English
• Subjects: Capsule network; Mixture of experts; Tumor type classification
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2021.107942

•Capsule networks (CapsNets) are developed aiming to overcome key drawbacks of the CNNs by identifying the spatial relations via their “Routing by Agreement” process.•Mixture of Capsule networks (MIXCAPS), proposed in this work, for the task of lung nodule malignancy prediction, has the potential to improve the classification accuracy by integrating/coupling several CapsNet experts.•Each CapsNet within the MIXCAPS architecture focuses on a specific subset of the nodules, therefore, improving the overall classification performance of the model.•Output of the gating model is investigated for potential correlations with nodule hand-crafted features to improve the potential interpretability of the proposed MIXCAPS.•Generalizability of the proposed MIXCAPS is illustrated via extension and evaluation based on a separate dataset associated with a different prediction task other than the one initially used to design the framework. Lung cancer is among the most common and deadliest cancers with a low 5-year survival rate. Timely diagnosis of lung cancer is, therefore, of paramount importance as it can save countless lives. In this regard, Computed Tomography (CT) scan is widely used for early detection of lung cancer, where human judgment is currently considered as the gold standard approach. Recently, there has been a surge of interest on development of automatic solutions via radiomics, as human-centered diagnosis is subject to inter-observer variability and is highly burdensome. Hand-crafted radiomics, serving as a radiologist assistant, requires fine annotations and pre-defined features. Deep learning radiomics solutions, however, have the promise of extracting the most useful features on their own in an end-to-end fashion without having access to the annotated boundaries. Among different deep learning models, Capsule Networks are proposed to overcome shortcomings of the Convolutional Neural Networks (CNNs) such as their inability to recognize detailed spatial relations. Capsule networks have so far shown satisfying performance in medical imaging problems. Capitalizing on their success, in this study, we propose a novel capsule network-based mixture of experts, referred to as the MIXCAPS. The proposed MIXCAPS architecture takes advantage of not only the capsule network’s capabilities to handle small datasets, but also automatically splitting dataset through a convolutional gating network. MIXCAPS enables capsule network experts to specialize on different subset