IntelliSwAS: Optimizing deep neural network architectures using a particle swarm-based approach

• Published in: Expert systems with applications 2022-01, Vol.187, p.115945, Article 115945
• Main Authors: Nistor, Sergiu Cosmin, Czibula, Gabriela
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Classification; Computer architecture; Convolutional neural network; Deep learning; Domains; Graph neural network; Graph theory; Image classification; Machine learning; Neural architecture search; Neural networks; Optimization; Particle swarm optimization; Recurrent neural network
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.115945

Deep learning is in a continuous evolution and many domains benefit from this substantial progress in the development of intelligent solutions. While this progress has been swift, there are more and more applications and the specific requirements of each application domain entails much work done by researchers to adapt existing models or create new ones. This traditional approach has produced many successful designs, but recently, automated methods of finding neural network architectures emerged. We introduce IntelliSwAS approach for optimizing deep neural network architectures for a classification or regression task. Image classification was selected as the task, but IntelliSwAS is generic enough such that it could be successfully applied on other classification or regression problems. A particle swarm-based optimization algorithm is proposed for the automatic search for convolutional neural network architectures. The search technique is enhanced by a machine learning model (DAGRNN) which we designed for predicting the quality of the network architectures and thus increasing the performance of the algorithm. The proposed model is able to process data structured as directed acyclic graphs in general and we applied it specifically on network architectures. The network architecture that we discovered using IntelliSwAS surpassed 89.8% of the competing image classification models that we considered for comparison. •We propose a novel swarm-based approach for neural architecture search.•The search technique is enhanced with a model that extends recurrent neural networks.•The proposed model is able to process data structured as directed acyclic graphs.•We used image classification for performance-evaluation experiments of the method.•Our approach surpassed 89.8% of image classification models from the literature.