Theoretical Exploration of Flexible Transmitter Model

• Published in: IEEE transaction on neural networks and learning systems 2024-03, Vol.PP (3), p.1-15
• Main Authors: Wu, Jin-Hui, Zhang, Shao-Qun, Jiang, Yuan, Zhou, Zhi-Hua
• Format: Article
• Language: English
• Subjects: Approximation complexity; Biological neural networks; Biological system modeling; Complexity theory; Data models; Firing pattern; flexible transmitter (FT) model; local minimum; Neural networks; Neurons; Recurrent neural networks; Search algorithms; Transmitters
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2022.3195909

Neural network models generally involve two important components, i.e., network architecture and neuron model. Although there are abundant studies about network architectures, only a few neuron models have been developed, such as the MP neuron model developed in 1943 and the spiking neuron model developed in the 1950s. Recently, a new bio-plausible neuron model, flexible transmitter (FT) model (Zhang and Zhou, 2021), has been proposed. It exhibits promising behaviors, particularly on temporal-spatial signals, even when simply embedded into the common feedforward network architecture. This article attempts to understand the properties of the FT network (FTNet) theoretically. Under mild assumptions, we show that: 1) FTNet is a universal approximator; 2) the approximation complexity of FTNet can be exponentially smaller than those of commonly used real-valued neural networks with feedforward/recurrent architectures and is of the same order in the worst case; and 3) any local minimum of FTNet is the global minimum, implying that it is possible to identify global minima by local search algorithms.