TAO-robust backpropagation learning algorithm

• Published in: Neural networks 2005-03, Vol.18 (2), p.191-204
• Main Authors: Pernía-Espinoza, Alpha V., Ordieres-Meré, Joaquín B., Martínez-de-Pisón, Francisco J., González-Marcos, Ana
• Format: Article
• Language: English
• Subjects: Algorithms; Animals; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Exact sciences and technology; Humans; Learning - physiology; Neural networks; Neural Networks (Computer); Nonlinear Dynamics; Outliers; Robotics; Robust learning algorithm; Robust modelling
• ISSN: 0893-6080; 1879-2782
• DOI: 10.1016/j.neunet.2004.11.007

In several fields, as industrial modelling, multilayer feedforward neural networks are often used as universal function approximations. These supervised neural networks are commonly trained by a traditional backpropagation learning format, which minimises the mean squared error ( mse) of the training data. However, in the presence of corrupted data (outliers) this training scheme may produce wrong models. We combine the benefits of the non-linear regression model τ-estimates [introduced by Tabatabai, M. A., Argyros, I. K. Robust Estimation and testing for general nonlinear regression models. Applied Mathematics and Computation. 58 (1993) 85–101 ] with the backpropagation algorithm to produce the TAO-robust learning algorithm, in order to deal with the problems of modelling with outliers. The cost function of this approach has a bounded influence function given by the weighted average of two ψ functions, one corresponding to a very robust estimate and the other to a highly efficient estimate. The advantages of the proposed algorithm are studied with an example.