Improved sign-based learning algorithm derived by the composite nonlinear Jacobi process

• Published in: Journal of computational and applied mathematics 2006-07, Vol.191 (2), p.166-178
• Main Authors: Anastasiadis, Aristoklis D., Magoulas, George D., Vrahatis, Michael N.
• Format: Article
• Language: English
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Convergence analysis; Exact sciences and technology; Feedforward neural networks; Global convergence; Learning and adaptive systems; Mathematics; Nonlinear iterative methods; Nonlinear Jacobi; Pattern classification; Pattern recognition. Digital image processing. Computational geometry; Probability and statistics; Probability theory and stochastic processes; Sciences and techniques of general use; Special processes (renewal theory, markov renewal processes, semi-markov processes, statistical mechanics type models, applications); Supervised learning; Theoretical computing
• ISSN: 0377-0427; 1879-1778
• DOI: 10.1016/j.cam.2005.06.034

In this paper a globally convergent first-order training algorithm is proposed that uses sign-based information of the batch error measure in the framework of the nonlinear Jacobi process. This approach allows us to equip the recently proposed Jacobi–Rprop method with the global convergence property, i.e. convergence to a local minimizer from any initial starting point. We also propose a strategy that ensures the search direction of the globally convergent Jacobi–Rprop is a descent one. The behaviour of the algorithm is empirically investigated in eight benchmark problems. Simulation results verify that there are indeed improvements on the convergence success of the algorithm.