Design of Recurrent Neural Networks for Solving Constrained Least Absolute Deviation Problems

• Published in: IEEE transaction on neural networks and learning systems 2010-07, Vol.21 (7), p.1073-1086
• Main Authors: Hu, Xiaolin, Sun, Changyin, Zhang, Bo
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial intelligence; Computer architecture; Computer networks; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Constraint optimization; Constraints; Convergence; Deviation; Equations; Exact sciences and technology; Humans; Inequalities; Inequality; Laboratories; least absolute deviation (LAD); minimax optimization; Networks; Neural networks; Neural Networks (Computer); Nonlinear Dynamics; Operations research; Parameter estimation; Pattern Recognition, Automated - methods; recurrent neural network (RNN); Recurrent neural networks; Signal processing algorithms; stability analysis; Studies; Sun; {{L}_{1}} -norm optimization
• ISSN: 1045-9227; 2162-237X; 1941-0093; 2162-2388
• DOI: 10.1109/TNN.2010.2048123

Recurrent neural networks for solving constrained least absolute deviation (LAD) problems or L 1 -norm optimization problems have attracted much interest in recent years. But so far most neural networks can only deal with some special linear constraints efficiently. In this paper, two neural networks are proposed for solving LAD problems with various linear constraints including equality, two-sided inequality and bound constraints. When tailored to solve some special cases of LAD problems in which not all types of constraints are present, the two networks can yield simpler architectures than most existing ones in the literature. In particular, for solving problems with both equality and one-sided inequality constraints, another network is invented. All of the networks proposed in this paper are rigorously shown to be capable of solving the corresponding problems. The different networks designed for solving the same types of problems possess the same structural complexity, which is due to the fact these architectures share the same computing blocks and only differ in connections between some blocks. By this means, some flexibility for circuits realization is provided. Numerical simulations are carried out to illustrate the theoretical results and compare the convergence rates of the networks.