A hybrid constriction coefficient-based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron

• Published in: International journal of intelligent computing and cybernetics 2020-07, Vol.13 (2), p.129-165
• Main Authors: Rather, Sajad Ahmad, Bala, P. Shanthi
• Format: Article
• Language: English
• Subjects: Accuracy; Back propagation; Business metrics; Constrictions; Convergence; Datasets; Efficiency; Exploitation; Heuristic; Heuristic methods; Learning; Literature reviews; Multilayer perceptrons; Neural networks; Optimization algorithms; Optimization techniques; Particle swarm optimization; Performance evaluation; Performance measurement; Power; Scientific papers; Search algorithms; Stagnation; Statistical tests; Training; Trigonometric functions
• ISSN: 1756-378X; 1756-3798
• DOI: 10.1108/IJICC-09-2019-0105

PurposeIn this paper, a newly proposed hybridization algorithm namely constriction coefficient-based particle swarm optimization and gravitational search algorithm (CPSOGSA) has been employed for training MLP to overcome sensitivity to initialization, premature convergence, and stagnation in local optima problems of MLP.Design/methodology/approachIn this study, the exploration of the search space is carried out by gravitational search algorithm (GSA) and optimization of candidate solutions, i.e. exploitation is performed by particle swarm optimization (PSO). For training the multi-layer perceptron (MLP), CPSOGSA uses sigmoid fitness function for finding the proper combination of connection weights and neural biases to minimize the error. Secondly, a matrix encoding strategy is utilized for providing one to one correspondence between weights and biases of MLP and agents of CPSOGSA.FindingsThe experimental findings convey that CPSOGSA is a better MLP trainer as compared to other stochastic algorithms because it provides superior results in terms of resolving stagnation in local optima and convergence speed problems. Besides, it gives the best results for breast cancer, heart, sine function and sigmoid function datasets as compared to other participating algorithms. Moreover, CPSOGSA also provides very competitive results for other datasets.Originality/valueThe CPSOGSA performed effectively in overcoming stagnation in local optima problem and increasing the overall convergence speed of MLP. Basically, CPSOGSA is a hybrid optimization algorithm which has powerful characteristics of global exploration capability and high local exploitation power. In the research literature, a little work is available where CPSO and GSA have been utilized for training MLP. The only related research paper was given by Mirjalili et al., in 2012. They have used standard PSO and GSA for training simple FNNs. However, the work employed only three datasets and used the MSE performance metric for evaluating the efficiency of the algorithms. In this paper, eight different standard datasets and five performance metrics have been utilized for investigating the efficiency of CPSOGSA in training MLPs. In addition, a non-parametric pair-wise statistical test namely the Wilcoxon rank-sum test has been carried out at a 5% significance level to statistically validate the simulation results. Besides, eight state-of-the-art meta-heuristic algorithms were employed for comparative analysis of the e