Learning Topology and Dynamics of Large Recurrent Neural Networks

Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on signal processing 2014-11, Vol.62 (22), p.5881-5891
Main Authors: She, Yiyuan, He, Yuejia, Wu, Dapeng
Format: Article
Language:English
Subjects:
Algorithms
Computational efficiency
Dynamical systems
Estimation
Learning
Lyapunov stability
Mathematical model
Network topologies
Network topology
Networks
Nonlinear dynamics
recurrent networks
shrinkage estimation
Signal processing algorithms
Stability
Stability analysis
Terrorism
Topology
topology learning
variable selection
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node observations. This task becomes extremely challenging in modern network applications, because the available observations are usually very noisy and limited, and the associated dynamical system is strongly nonlinear. By formulating the problem as multivariate sparse sigmoidal regression, we develop simple-to-implement network learning algorithms, with rigorous convergence guarantee in theory, for a variety of sparsity-promoting penalty forms. A quantile variant of progressive recurrent network screening is proposed for efficient computation and allows for direct cardinality control of network topology in estimation. Moreover, we investigate recurrent network stability conditions in Lyapunov's sense, and integrate such stability constraints into sparse network learning. Experiments show excellent performance of the proposed algorithms in network topology identification and forecasting.
ISSN:1053-587X
1941-0476
DOI:10.1109/TSP.2014.2358956