Impact of noise structure and network topology on tracking speed of neural networks

• Published in: Neural networks 2011-12, Vol.24 (10), p.1110-1119
• Main Authors: Huang, Longwen, Cui, Yuwei, Zhang, Danke, Wu, Si
• Format: Article
• Language: English
• Subjects: Action Potentials - physiology; Animals; Applied sciences; Artificial intelligence; Cerebral Cortex - physiology; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Connectionism. Neural networks; Exact sciences and technology; Fast neural computation; Humans; Linear Models; Membrane Potentials - physiology; Models, Neurological; Nerve Net - physiology; Network topology; Neural Networks (Computer); Neurons - physiology; Noise structure; Nonlinear dynamics and nonlinear dynamical systems; Physics; Reaction Time - physiology; Software; Statistical physics, thermodynamics, and nonlinear dynamical systems; Synaptic Transmission - physiology; Synchronization; coupled oscillators; Tracking speed
• ISSN: 0893-6080; 1879-2782
• DOI: 10.1016/j.neunet.2011.05.018

Understanding why neural systems can process information extremely fast is a fundamental question in theoretical neuroscience. The present study investigates the effect of noise on accelerating neural computation. To evaluate the speed of network response, we consider a computational task in which the network tracks time-varying stimuli. Two noise structures are compared, namely, the stimulus-dependent and stimulus-independent noises. Based on a simple linear integrate-and-fire model, we theoretically analyze the network dynamics, and find that the stimulus-dependent noise, whose variance is proportional to the mean of external inputs, has better effect on speeding up network computation. This is due to two good properties in the transient network dynamics: (1) the instant firing rate of the network is proportional to the mean of external inputs, and (2) the stationary state of the network is robust to stimulus changes. We investigate two network models with varying recurrent interactions, and find that recurrent interactions tend to slow down the tracking speed of the network. When the biologically plausible Hodgkin–Huxley model is considered, we also observe that the stimulus-dependent noise accelerates neural computation, although the improvement is smaller than that in the case of linear integrate-and-fire model. ► We investigate the effect of noise on accelerating neural computation. ► Two noise structures are compared, namely, the stimulus-dependent and stimulus-independent noises. ► The stimulus-dependent noise has better effect on speeding up network computation. ► Recurrent interactions tend to slow down the tracking speed of the network.