Across-task neural architecture search via meta learning

Adequate labeled data and expensive compute resources are the prerequisites for the success of neural architecture search (NAS). It is challenging to apply NAS in meta-learning scenarios with limited compute resources and data. In this paper, an across-task neural architecture search (AT-NAS) method...

Full description

Saved in:
Bibliographic Details
Published in:International journal of machine learning and cybernetics 2023-03, Vol.14 (3), p.1003-1019
Main Authors: Rong, Jingtao, Yu, Xinyi, Zhang, Mingyang, Ou, Linlin
Format: Article
Language:English
Subjects:
Accuracy
Algorithms
Artificial Intelligence
Classification
Complex Systems
Computational Intelligence
Control
Datasets
Engineering
Evolutionary algorithms
Learning
Mechatronics
Optimization
Original Article
Pattern Recognition
Robotics
Search algorithms
Systems Biology
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Adequate labeled data and expensive compute resources are the prerequisites for the success of neural architecture search (NAS). It is challenging to apply NAS in meta-learning scenarios with limited compute resources and data. In this paper, an across-task neural architecture search (AT-NAS) method is proposed to address this problem via combining gradient-based meta-learning with EA-based NAS to learn over the distribution of tasks. The supernet is learned over an entire set of tasks by meta learning. Architecture encodes of subnets sampled from the supernet are iteratively adapted by evolutionary algorithms while simultaneously searching for a task-sensitive meta-network. The searched meta-network can adapt to a novel task via a few learning steps and it only costs a little search time. Empirical results show that AT-NAS achieves excellent performance in few-shot classification. The performance of AT-NAS on classification benchmarks is comparable to that of models searched from scratch, by adapting the architecture in less than an hour from a 5-GPU-day pretrained meta-network.
ISSN:1868-8071
1868-808X
DOI:10.1007/s13042-022-01678-5