Analysis of the effects of quantization in multilayer neural networks using a statistical model

• Published in: IEEE transactions on neural networks 1992-03, Vol.3 (2), p.334-338
• Main Authors: Yun Xie, Jabri, M.A.
• Format: Article
• Language: English
• Subjects: Applied sciences; Artificial neural networks; Australia Council; Degradation; Exact sciences and technology; Feedforward neural networks; Information, signal and communications theory; Intelligent networks; Multi-layer neural network; Neural network hardware; Neural networks; Neurons; Quantization; Sampling, quantization; Signal and communications theory; Telecommunications and information theory
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/72.125876

A statistical quantization model is used to analyze the effects of quantization when digital techniques are used to implement a real-valued feedforward multilayer neural network. In this process, a parameter called the effective nonlinearity coefficient, which is important in the studying of quantization effects, is introduced. General statistical formulations of the performance degradation of the neural network caused by quantization are developed as functions of the quantization parameters. The formulations predict that the network's performance degradation gets worse when the number of bits is decreased; that a change of the number of hidden units in a layer has no effect on the degradation; that for a constant effective nonlinearity coefficient and number of bits, an increase in the number of layers leads to worse performance degradation; and the number of bits in successive layers can be reduced if the neurons of the lower layer are nonlinear.< >