A meta-active learning approach exploiting instance importance

• Published in: Expert systems with applications 2024-08, Vol.247, p.123320, Article 123320
• Main Authors: Flesca, Sergio, Mandaglio, Domenico, Scala, Francesco, Tagarelli, Andrea
• Format: Article
• Language: English
• Subjects: Active learning; Classification; Gradient variation; Instance importance; Meta-learning; Neural networks
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2024.123320

Active learning is focused on minimizing the effort required to obtain labeled data by iteratively choosing fresh data samples for training a machine learning model. One of the primary challenges in active learning involves the selection of the most informative instances for labeling by an annotation oracle at each iteration. A viable approach is to develop an active learning strategy that aligns with the performance of a meta-learning model. This strategy evaluates the quality of previously selected instances and subsequently trains a machine learning model to predict the quality of instances to be labeled in the current iteration. This paper introduces a novel approach to learning for active learning, wherein instances are chosen for labeling based on their potential to induce the most substantial change in the current classifier. We explore various strategies for assessing the significance of an instance, taking into account variations in the learning gradient of the classification model. Our approach can be applied to any classifier that can be trained using gradient descent optimization. Here, we present a formulation that leverages a deep neural network model, which has not been extensively explored in existing learning-to-active-learn methodologies. Through experimental validation, our approach demonstrates promising results, especially in scenarios where there are limited initially labeled instances, particularly when the number of labeled instances per class is extremely limited. •A new meta-active learning strategy for deep neural network models.•The instance selection problem is modeled as a regression problem.•Four strategies of instance importance scoring by considering gradient variations.•Experiments have shown good results when few initially labeled objects are available.•A dimensionality reduction step for the instances improves the performances.