OMCBIR: Offline mobile content-based image retrieval with lightweight CNN optimization

• Published in: Displays 2023-01, Vol.76, p.102355, Article 102355
• Main Authors: Zhang, Xiaoqing, Bai, Cong, Kpalma, Kidiyo
• Format: Article
• Language: English
• Subjects: Engineering Sciences; Image retrieval; Lightweight neural network; Mobile devices; Model compression
• ISSN: 0141-9382; 1872-7387
• DOI: 10.1016/j.displa.2022.102355

Convolutional Neural Networks (CNNs) have achieved great success in computer vision applications. However, due to the high requirements for computation power and memory usage, most state-of-the-art CNNs are difficult to deploy on resource-constrained mobile devices. Although many typical lightweight neural networks have been proposed in the industry, such as MobileNetV2, which reduce the amount of parameters and calculations, they still have a lot of redundancy. Furthermore, few papers consider the use of deep learning models to implement image retrieval on terminals, so we propose a new offline retrieval framework based on lightweight neural network models, called Offline Mobile Content-Based Image Retrieval (OMCBIR). In this framework, we focus on the feature extraction of the model, by introducing pointwise group convolution and channel shuffle into the bottleneck block, reconstructing the network structure, and introducing the convolutional attention module, we propose an extremely lightweight small network-Attention-based Lightweight Network (ALNet). Compared to MobileNetV2, ALNet obtains a higher mAP on each dataset in OMCBIR when the model parameters are reduced by more than 62% and the model size is reduced by more than 63%. Extensive experiments conducted on five public datasets provide a trade-off between retrieval performance and model size of different algorithms, which proves the efficiency of the proposed OMCBIR. •We propose a novel offline framework for content-based image retrieval on the mobile side, namely OMCBIR. As far as we know, this is the first time that a deep learning model is used for a completely offline mobile content-based image retrieval task.•We propose an extremely lightweight network architecture ALNet and ALBlock unit. By introducing pointwise group convolution, channel shuffle and convolution attention module into ALBlock, we not only improve retrieval accuracy, but also greatly reduce the computational cost and memory occupation.•The experimental results on five public datasets demonstrate the effectiveness of our model. Compared to MobileNetV2, ALNet obtains a higher mAP on each dataset in OMCBIR with over 62% reduction in model parameters and over 63% reduction in model size.