An online learning algorithm for adaptable topologies of neural networks

• Published in: Expert systems with applications 2013-12, Vol.40 (18), p.7294-7304
• Main Authors: Pérez-Sánchez, Beatriz, Fontenla-Romero, Oscar, Guijarro-Berdiñas, Bertha, Martínez-Rego, David
• Format: Article
• Language: English
• Subjects: Adaptive topology; Algorithmics. Computability. Computer arithmetics; Algorithms; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Distance education; Distance learning; Dynamics; Exact sciences and technology; Forgetting ability; General aspects; Incremental learning; Learning; Learning and adaptive systems; Mathematical models; Neural networks; Occupational training. Personnel. Work management; On-line systems; Online; Sequential learning; Software; Theoretical computing; Topology
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2013.06.066

•We proposed an online learning algorithm for feedforward neural networks.•The learning method allows the dynamic incorporation of hidden neurons.•It can be used in non-stationary problems allowing adaptation to changes. Many real scenarios in machine learning are of dynamic nature. Learning in these types of environments represents an important challenge for learning systems. In this context, the model used for learning should work in real time and have the ability to act and react by itself, adjusting its controlling parameters, even its structures, depending on the requirements of the process. In a previous work, the authors presented an online learning algorithm for two-layer feedforward neural networks that includes a factor that weights the errors committed in each of the samples. This method is effective in dynamic environments as well as in stationary contexts. As regards this method’s incremental feature, we raise the possibility that the network topology is adapted according to the learning needs. In this paper, we demonstrate and justify the suitability of the online learning algorithm to work with adaptive structures without significantly degrading its performance. The theoretical basis for the method is given and its performance is illustrated by means of its application to different system identification problems. The results confirm that the proposed method is able to incorporate units to its hidden layer, during the learning process, without high performance degradation.