The modified sufficient conditions for echo state property and parameter optimization of leaky integrator echo state network

• Published in: Applied soft computing 2019-04, Vol.77, p.750-760
• Main Authors: Lun, Shu-xian, Hu, Hai-feng, Yao, Xian-shuang
• Format: Article
• Language: English
• Subjects: Barrier method; Echo state property; Leaky integrator echo state network; Parameter optimization; Time series prediction
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2019.02.005

Leaky integrator echo state network (Leaky-ESN), an improved echo state network, is suitable for learning very slow dynamic systems and replaying the learnt system at different speeds. To ensure the echo state property of Leaky-ESN, spectral radius of reservoir connection weight matrix needs to be less than leaking rate, which means that the output feedback of Leaky-ESN is actually ignored. Leaky-ESN without output feedback is equivalent to being a feedforward type neural network. Therefore, in this paper, we consider Leaky-ESN model with output feedback. Firstly, we give the modified sufficient conditions for echo state property, which are actually some inequality constraints about the control parameters (mainly leaking rate, spectral radius of reservoir connection weight matrix and the scaling of output feedback) and the maximum singular values of feedback and output connection weight matrices. Secondly, to reduce the influence of the initial values of the control parameters on Leaky-ESN, we use the barrier method to optimize the control parameters. The barrier method can convert the constrained optimization problem into the unconstrained optimization problem. Newton’s method and its improve method, for example, eigenvalue modification methods, are further used to solve the unconstrained optimization problem and then obtain the control parameters of Leaky-ESN. Finally, Mackey–Glass chaotic time series prediction is selected to validate the method proposed in this paper. Simulation results show that the method proposed in this paper can make Leaky-ESN model more stable and higher approximation precision. [Display omitted] •Leaky-ESN model with output feedback is considered in this paper.•Modified sufficient conditions for echo state property of Leaky-ESN are given.•The barrier method is used to optimize the control parameters of Leaky-ESN.•Newton’s method is used to solve the unconstrained optimization problem.•The optimized Leaky-ESN is used to predict Mackey–Glass chaotic time series.