COVID-19 CT image recognition algorithm based on transformer and CNN

• Published in: Displays 2022-04, Vol.72, p.102150-102150, Article 102150
• Main Authors: Fan, Xiaole, Feng, Xiufang, Dong, Yunyun, Hou, Huichao
• Format: Article
• Language: English
• Subjects: Bi-directional feature fusion; CNN; COVID-19; Transformer
• ISSN: 0141-9382; 1872-7387
• DOI: 10.1016/j.displa.2022.102150

•A branch network model based on CNN and Transformer structure is proposed.•Design a bi-directional feature fusion structure, which can fuse features extracted from two branches.•Innovatively apply the model proposed in this paper to COVIDx-CT—— a large-scale COVID-19 data set. Novel corona virus pneumonia (COVID-19) broke out in 2019, which had a great impact on the development of world economy and people's lives. As a new mainstream image processing method, deep learning network has been constructed to extract medical features from chest CT images, and has been used as a new detection method in clinical practice. However, due to the medical characteristics of COVID-19 CT images, the lesions are widely distributed and have many local features. Therefore, it is difficult to diagnose directly by using the existing deep learning model. According to the medical features of CT images in COVID-19, a parallel bi-branch model (Trans-CNN Net) based on Transformer module and Convolutional Neural Network module is proposed by making full use of the local feature extraction capability of Convolutional Neural Network and the global feature extraction advantage of Transformer. According to the principle of cross-fusion, a bi-directional feature fusion structure is designed, in which features extracted from two branches are fused bi-directionally, and the parallel structures of branches are fused by a feature fusion module, forming a model that can extract features of different scales. To verify the effect of network classification, the classification accuracy on COVIDx-CT dataset is 96.7%, which is obviously higher than that of typical CNN network (ResNet-152) (95.2%) and Transformer network (Deit-B) (75.8%). These results demonstrate accuracy is improved. This model also provides a new method for the diagnosis of COVID-19, and through the combination of deep learning and medical imaging, it promotes the development of real-time diagnosis of lung diseases caused by COVID-19 infection, which is helpful for reliable and rapid diagnosis, thus saving precious lives.