Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction

• Main Authors: Wu, Botong, Zhou, Zhen, Wang, Jianwei, Wang, Yizhou
• Format: Conference Proceeding
• Language: English
• Subjects: 3D convolutional neural network; attribute learning; Cancer; Computed tomography; Computer-aided diagnosis; Image segmentation; Lung; lung nodule classification; lung nodule segmentation; multi-task learning; Semantics; Task analysis; Three-dimensional displays
• ISSN: 1945-8452
• DOI: 10.1109/ISBI.2018.8363765

Refer to the literature of lung nodule classification, many studies adopt Convolutional Neural Networks (CNN) to directly predict the malignancy of lung nodules with original thoracic Computed Tomography (CT) and nodule location. However, these studies cannot tell how the CNN works in terms of predicting the malignancy of the given nodule, e.g., it's hard to conclude that whether the region within the nodule or the contextual information matters according to the output of the CNN. In this paper, we propose an interpretable and multi-task learning CNN - Joint learning for Pulmonary Nodule Segmentation Attributes and Malignancy Prediction (PN-SAMP). It is able to not only accurately predict the malignancy of lung nodules, but also provide semantic high-level attributes as well as the areas of detected nodules. Moreover, the combination of nodule segmentation, attributes and malignancy prediction is helpful to improve the performance of each single task. In addition, inspired by the fact that radiologists often change window widths and window centers to help to make decision on uncertain nodules, PN-SAMP mixes multiple WW/WC together to gain information for the raw CT input images. To verify the effectiveness of the proposed method, the evaluation is implemented on the public LIDC-IDRI dataset, which is one of the largest dataset for lung nodule malignancy prediction. Experiments indicate that the proposed PN-SAMP achieves significant improvement with respect to lung nodule classification, and promising performance on lung nodule segmentation and attribute learning, compared with the-state-of-the-art methods.