Auto CNN classifier based on knowledge transferred from self-supervised model

Training with unlabeled datasets using self-supervised models has the edge over training with labeled datasets, reducing human effort, and no need for annotated data during training. The knowledge obtained from the pre-trained self-supervised model can be transferred to an image classifier as a targ...

Full description

Saved in:
Bibliographic Details
Published in:Applied intelligence (Dordrecht, Netherlands) Netherlands), 2023-10, Vol.53 (19), p.22086-22104
Main Authors: Kishore, Jaydeep, Mukherjee, Snehasis
Format: Article
Language:English
Subjects:
Artificial Intelligence
Classifiers
Computer Science
Datasets
Image classification
Knowledge management
Machines
Manufacturing
Mechanical Engineering
Optimization
Performance evaluation
Processes
Training
Tuning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Training with unlabeled datasets using self-supervised models has the edge over training with labeled datasets, reducing human effort, and no need for annotated data during training. The knowledge obtained from the pre-trained self-supervised model can be transferred to an image classifier as a target task. Transferring knowledge from source to target task requires tuning a large number of hyperparameters, such as number of CNN layers, number of filters in each CNN layer, kernel size, stride, number of fully connected (FC) layers, units in each FC layer, dropouts, learning rate, and many more. An efficient process is required to automatically tune these hyperparameters to reduce the manual efforts and enhance the performance of the self-supervised model. This paper uses a pre-trained self-supervised model to transfer knowledge for image classification. We propose an efficient Bayesian optimization-based method for automatically tuning (autotuning) the hyperparameters of the self-supervised model during the knowledge transfer. The proposed autotuned image classifier consists of a few CNN layers followed by an FC layer. Finally, we use a softmax layer to obtain the probability of classes for the images. To evaluate the performance of the proposed method, state-of-the-art self-supervised models such as SimClr, SWAV, and SWAV 2x are used to extract the learned features. According to our experiments on three benchmarked datasets (CIFAR-10, CIFAR-100, and Tiny Imagenet datasets), the proposed method not only enhances the performances of the self-supervised models, but also provides state-of-the-art results.
ISSN:0924-669X
1573-7497
DOI:10.1007/s10489-023-04598-1