A pre-trained convolutional neural network based method for thyroid nodule diagnosis

• Published in: Ultrasonics 2017-01, Vol.73, p.221-230
• Main Authors: Ma, Jinlian, Wu, Fa, Zhu, Jiang, Xu, Dong, Kong, Dexing
• Format: Article
• Language: English
• Subjects: Biopsy, Fine-Needle; Classification; Convolutional neural network; Diagnosis; Feature extraction; Female; Humans; Image Processing, Computer-Assisted - methods; Male; Middle Aged; Neural Networks (Computer); Reproducibility of Results; Thyroid nodule; Thyroid Nodule - diagnostic imaging; Thyroid Nodule - pathology; Ultrasonography - methods; Ultrasound image
• ISSN: 0041-624X; 1874-9968
• DOI: 10.1016/j.ultras.2016.09.011

•A hybrid approach is proposed to diagnose thyroid nodules in ultrasound.•It is a fusion of two pre-trained CNNs with different architectures.•All the CNNs are pre-trained with 1.3million natural images from ImageNet database.•A multi-view strategy is applied to improve the performance of CNNs.•Large clinical thyroid nodule images are studied in our experiments.•Novel single-valued integrated indices called TMI are determined.•It is easy and transparent of TMI to diagnose thyroid nodules using this technique. In ultrasound images, most thyroid nodules are in heterogeneous appearances with various internal components and also have vague boundaries, so it is difficult for physicians to discriminate malignant thyroid nodules from benign ones. In this study, we propose a hybrid method for thyroid nodule diagnosis, which is a fusion of two pre-trained convolutional neural networks (CNNs) with different convolutional layers and fully-connected layers. Firstly, the two networks pre-trained with ImageNet database are separately trained. Secondly, we fuse feature maps learned by trained convolutional filters, pooling and normalization operations of the two CNNs. Finally, with the fused feature maps, a softmax classifier is used to diagnose thyroid nodules. The proposed method is validated on 15,000 ultrasound images collected from two local hospitals. Experiment results show that the proposed CNN based methods can accurately and effectively diagnose thyroid nodules. In addition, the fusion of the two CNN based models lead to significant performance improvement, with an accuracy of 83.02%±0.72%. These demonstrate the potential clinical applications of this method.