Mean Field and Capacity in Realistic Networks of Spiking Neurons Storing Sparsely Coded Random Memories

• Published in: Neural computation 2004-12, Vol.16 (12), p.2597-2637
• Main Authors: Curti, Emanuele, Mongillo, Gianluigi, Camera, Giancarlo La, Amit, Daniel J.
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Biological and medical sciences; Computer science; control theory; systems; Computer Simulation; Computer Systems; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects. Models. Methods; Learning and adaptive systems; Mathematics; Memory; Memory - physiology; Miscellaneous; Models, Neurological; Neural networks; Neural Networks (Computer); Neurons; Neurons - physiology; Probability and statistics; Probability theory and stochastic processes; Sciences and techniques of general use; Special processes (renewal theory, markov renewal processes, semi-markov processes, statistical mechanics type models, applications); Theory; Vertebrates: nervous system and sense organs
• ISSN: 0899-7667; 1530-888X
• DOI: 10.1162/0899766042321805

Mean-field (MF) theory is extended to realistic networks of spiking neurons storing in synaptic couplings of randomly chosen stimuli of a given low coding level. The underlying synaptic matrix is the result of a generic, slow, long-term synaptic plasticity of two-state synapses, upon repeated presentation of the fixed set of the stimuli to be stored. The neural populations subtending the MF description are classified by the number of stimuli to which their neurons are responsive ( ). This involves 2 + 1 populations for a network storing memories. The computational complexity of the MF description is then significantly reduced by observing that at low coding levels ( ), only a few populations remain relevant: the population of mean multiplicity – and those of multiplicity of order √pf around the mean. The theory is used to produce (predict) bifurcation diagrams (the onset of selective delay activity and the rates in its various stationary states) and to compute the storage capacity of the network (the maximal number of single items used in training for each of which the network can sustain a persistent, selective activity state). This is done in various regions of the space of constitutive parameters for the neurons and for the learning process. The capacity is computed in MF versus potentiation amplitude, ratio of potentiation to depression probability and coding level . The MF results compare well with recordings of delay activity rate distributions in simulations of the underlying microscopic network of 10,000 neurons.