Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development

• Published in: Neurocomputing (Amsterdam) 2013-12, Vol.121, p.498-511
• Main Authors: Soares, Symone, Antunes, Carlos Henggeler, Araújo, Rui
• Format: Article
• Language: English
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Computer simulation; Connectionism. Neural networks; Data processing. List processing. Character string processing; Detection, estimation, filtering, equalization, prediction; Ensemble learning; Exact sciences and technology; Genetic algorithm; Genetic algorithms; Information, signal and communications theory; Learning; Mathematical models; Memory organisation. Data processing; Neural network; Neural networks; Signal and communications theory; Signal, noise; Simulated annealing; Software; Strategy; Telecommunications and information theory; Theoretical computing; Training
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2013.05.024

In the last decades ensemble learning has established itself as a valuable strategy within the computational intelligence modeling and machine learning community. Ensemble learning is a paradigm where multiple models combine in some way their decisions, or their learning algorithms, or different data to improve the prediction performance. Ensemble learning aims at improving the generalization ability and the reliability of the system. Key factors of ensemble systems are diversity, training and combining ensemble members to improve the ensemble system performance. Since there is no unified procedure to address all these issues, this work proposes and compares Genetic Algorithm and Simulated Annealing based approaches for the automatic development of Neural Network Ensembles for regression problems. The main contribution of this work is the development of optimization techniques that selects the best subset of models to be aggregated taking into account all the key factors of ensemble systems (e.g., diversity, training ensemble members and combination strategy). Experiments on two well-known data sets are reported to evaluate the effectiveness of the proposed methodologies. Results show that these outperform other approaches including Simple Bagging, Negative Correlation Learning (NCL), AdaBoost and GASEN in terms of generalization ability.