Optimizing Parameters of Multi-Layer Convolutional Neural Network by Modeling and Optimization Method

• Published in: IEEE access 2019, Vol.7, p.68316-68330
• Main Authors: Chou, Fu-I, Tsai, Yun-Kai, Chen, Yao-Mei, Tsai, Jinn-Tsong, Kuo, Chun-Cheng
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Convolution; Convolutional neural networks; Data models; Design; Design of experiments; Design optimization; Design parameters; Fashion-MNIST dataset; Feature extraction; Mathematical models; MNIST database; Modeling and optimization method; Modelling; Multilayers; multiple regression; Neural networks; NIST; Optimization; Optimization methods; Parameter identification; Parameter modification; Performance enhancement; PhysioNet dataset; Search algorithms; uniform experimental design
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2918563

The modeling and optimization method (MAOM) proposed in this study finds the best combination of parameters for a multi-layer convolutional neural network (MCNN). This study emphasizes that in addition to the importance of the MCNN structure, the parameter design within the layers is also very important. After determining the structure of the MCNN, the parameter optimization in the layer can improve the performance of the MCNN. The MCNN parameters for convolutional layers include filter size, number of filters, padding, and filter stride. Parameters for max-pooling layers also include pooling size and pooling stride. After the MCNN architecture is designed, the major challenge is finding the combination of parameters that enhances the MCNN performance. The proposed MAOM optimizes the MCNN parameters by integrating uniform experimental design (UED), multiple regression (MR), and optimization method. After the MCNN architecture is designed, UED is used to design the MCNN parameters. The parameter layout obtained by the UED is then used in experiments to collect data that can be used for modeling. Next, MR is performed using the parameters with the average correct rate to build an MCNN parameter model. Finally, a full-factorial search algorithm is used to find the best combination of the MCNN parameters for obtaining the maximum average correct rate. Images from the modified National Institute of Standards and Technology (modified NIST or MNIST) resources, Fashion-MNIST, and PhysioNet databases are used to test the performance of the architecture and parameters of the MCNN. The experimental results demonstrate the excellent performance of the MAOM in obtaining the best combination of MCNN parameters and maximum average correct rate. The main advantage of the proposed MAOM is its systematic method of finding the best combination of the MCNN parameters for image identification and obtaining high correct rate.