A Cell-Based Fast Memetic Algorithm for Automated Convolutional Neural Architecture Design

• Published in: IEEE transaction on neural networks and learning systems 2023-11, Vol.34 (11), p.9040-9053
• Main Authors: Dong, Junwei, Hou, Boyu, Feng, Liang, Tang, Huajin, Tan, Kay Chen, Ong, Yew-Soon
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Automation; Cell-based space; Computer architecture; Convolution; convolutional neural network (CNN); Convolutional neural networks; Datasets; Design; evolutionary algorithm (EA); Evolutionary algorithms; Machine learning; memetic algorithm (MA); Memetics; neural architecture search (NAS); Neural networks; Optimization; Search problems; Searching
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2022.3155230

Neural architecture search (NAS) has attracted much attention in recent years. It automates the neural network construction for different tasks, which is traditionally addressed manually. In the literature, evolutionary optimization (EO) has been proposed for NAS due to its strong global search capability. However, despite the success enjoyed by EO, it is worth noting that existing EO algorithms for NAS are often very computationally expensive, which makes these algorithms unpractical in reality. Keeping this in mind, in this article, we propose an efficient memetic algorithm (MA) for automated convolutional neural network (CNN) architecture search. In contrast to existing EO algorithms for CNN architecture design, a new cell-based architecture search space, and new global and local search operators are proposed for CNN architecture search. To further improve the efficiency of our proposed algorithm, we develop a one-epoch-based performance estimation strategy without any pretrained models to evaluate each found architecture on the training datasets. To investigate the performance of the proposed method, comprehensive empirical studies are conducted against 34 state-of-the-art peer algorithms, including manual algorithms, reinforcement learning (RL) algorithms, gradient-based algorithms, and evolutionary algorithms (EAs), on widely used CIFAR10 and CIFAR100 datasets. The obtained results confirmed the efficacy of the proposed approach for automated CNN architecture design.