Annotation of enhanced radiographs for medical image retrieval with deep convolutional neural networks

• Published in: PloS one 2018-11, Vol.13 (11), p.e0206229-e0206229
• Main Authors: Pelka, Obioma, Nensa, Felix, Friedrich, Christoph M
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Annotations; Artificial intelligence; Artificial neural networks; Biology and Life Sciences; Classification; Classification schemes; Computer and Information Sciences; Computer science; Datasets; Deep learning; Diagnosis, Computer-Assisted - statistics & numerical data; Digital imaging; Engineering and Technology; Humans; Image annotation; Image classification; Image enhancement; Image management; Image Processing, Computer-Assisted - methods; Image Processing, Computer-Assisted - statistics & numerical data; Image retrieval; International conferences; Machine Learning; Medical imaging; Medicine and Health Sciences; Neural networks; Neural Networks (Computer); Pattern recognition; Radiographs; Radiography; Radiography - methods; Relevance feedback; Research and Analysis Methods; Retrieval; Search engines
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0206229

The number of images taken per patient scan has rapidly increased due to advances in software, hardware and digital imaging in the medical domain. There is the need for medical image annotation systems that are accurate as manual annotation is impractical, time-consuming and prone to errors. This paper presents modeling approaches performed to automatically classify and annotate radiographs using several classification schemes, which can be further applied for automatic content-based image retrieval (CBIR) and computer-aided diagnosis (CAD). Different image preprocessing and enhancement techniques were applied to augment grayscale radiographs by virtually adding two extra layers. The Image Retrieval in Medical Applications (IRMA) Code, a mono-hierarchical multi-axial code, served as a basis for this work. To extensively evaluate the image enhancement techniques, five classification schemes including the complete IRMA code were adopted. The deep convolutional neural network systems Inception-v3 and Inception-ResNet-v2, and Random Forest models with 1000 trees were trained using extracted Bag-of-Keypoints visual representations. The classification model performances were evaluated using the ImageCLEF 2009 Medical Annotation Task test set. The applied visual enhancement techniques proved to achieve better annotation accuracy in all classification schemes.