Shallow and deep neural network training by water wave optimization

• Published in: Swarm and evolutionary computation 2019-11, Vol.50, p.100561, Article 100561
• Main Authors: Zhou, Xiao-Han, Zhang, Min-Xia, Xu, Zhi-Ge, Cai, Ci-Yun, Huang, Yu-Jiao, Zheng, Yu-Jun
• Format: Article
• Language: English
• Subjects: Artificial neural networks (ANNs); Deep neural networks (DNNs); Evolutionary neural networks; Structure and parameter optimization; Water wave optimization (WWO)
• ISSN: 2210-6502
• DOI: 10.1016/j.swevo.2019.100561

It is well known that the performance of artificial neural networks (ANNs) is significantly affected by their structure design and parameter selection, for which traditional training methods have drawbacks such as long training times, over-fitting, and premature convergence. Evolutionary algorithms (EAs) have provided an effective tool for ANN parameter optimization. However, simultaneously optimizing ANN structures and parameters remains a difficult problem. In this study, we adapt water wave optimization (WWO), a relatively new EA, for optimizing both the parameters and structures of ANNs, including classical shallow ANNs and deep neural networks (DNNs). We use a variable-dimensional solution encoding to represent both the structure and parameters of an ANN, and adapt WWO propagation, refraction, and breaking operators to efficiently evolve variable-dimensional solutions to solve the complex network optimization problems. Computational experiments on a variety of benchmark datasets show that the WWO algorithm achieves a very competitive performance compared to other popular gradient-based algorithms and EAs.