A Mathematical Theory of Energy Efficient Neural Computation and Communication

• Published in: IEEE transactions on information theory 2010-02, Vol.56 (2), p.852-874
• Main Authors: Berger, T., Levy, W.B.
• Format: Article
• Language: English
• Subjects: Applied sciences; Artificial intelligence; Beta distribution; bits per joule (bpj); bits per second (bps); Broadcasting; Computation; Computer science; control theory; systems; Connectionism. Neural networks; Degradation; Density; Distributing; Energy efficiency; Engines; Exact sciences and technology; Excitation; gamma distribution; History; Information; Information analysis; information rate; Information theory; Information, signal and communications theory; integrate-and-fire neuron; interspike interval; Mathematical model; Mathematical models; Modulation, demodulation; Nerve fibers; Neural networks; Neurons; Neurosciences; Poisson random measure; Pulse modulation; Signal and communications theory; Spikes; Synapses; Systems, networks and services of telecommunications; Telecommunications; Telecommunications and information theory; Theory; timing code; Transmission and modulation (techniques and equipments)
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2009.2037089

A neuroscience-based mathematical model of how a neuron stochastically processes data and communicates information is introduced and analyzed. Call the neuron in question 'neuron j", or just "j". The information j transmits approximately describes the time-varying intensity of the excitation j is continuously experiencing from neural spike trains delivered to its synapses by thousands of other neurons. Neuron j "encodes" this excitation history into a sequence of time instants at which it generates neural spikes of its own. By propagating these spikes along its axon, j acts as a multiaccess, partially degraded broadcast channel with thousands of input and output terminals that employs a time-continuous version of pulse position modulation. The mathematical model features three parameters, m, ¿, and b, which largely characterize j as an engine of computation and communication. Each set of values of these parameters corresponds to a long term maximization of the bits j conveys to its targets per joule it expends doing so, which is achieved by distributing the random duration between successive spikes j generates according to a gamma pdf with parameters ¿ and b and distributing b/A according to a beta probability density with parameters ¿ and m-¿, where A is the random intensity of the effectively Poisson process of spikes that arrive to the union of all of j 's synapses at a randomly chosen time instant.