Vision transformer-based meta loss landscape exploration with actor-critic method

Detecting and mitigating overfitting in deep neural networks remains a critical challenge in modern machine learning. This paper investigates innovative approaches to address these challenges, particularly focusing on vision transformer-based models. By leveraging meta-learning techniques and reinfo...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of supercomputing 2025, Vol.81 (1), Article 350
Main Authors: Zhang, Enzhi, Zhong, Rui, Du, Xingbang, Wahib, Mohamed, Munetomo, Masaharu
Format: Article
Language:English
Subjects:
Algorithms
Artificial neural networks
Compilers
Computer Science
Deep learning
Image analysis
Image classification
Image segmentation
Interpreters
Machine learning
Optimization
Processor Architectures
Programming Languages
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Detecting and mitigating overfitting in deep neural networks remains a critical challenge in modern machine learning. This paper investigates innovative approaches to address these challenges, particularly focusing on vision transformer-based models. By leveraging meta-learning techniques and reinforcement learning frameworks, we introduce transformer-based loss landscape exploration (TLLE), which utilizes the validation loss landscape to guide gradient descent optimization. Unlike conventional methods, TLLE employs the actor-critic algorithm to learn the mapping from model weights to future values, facilitating efficient sample collection and precise value predictions. Experimental results demonstrate the superior performance of TLLE-enhanced transformer models in image classification and segmentation tasks, showcasing the efficacy of our approach in optimizing deep learning models for image analysis.
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-024-06867-3