Shallow neural network with kernel approximation for prediction problems in highly demanding data networks

• Published in: Expert systems with applications 2019-06, Vol.124, p.196-208
• Main Authors: Lopez-Martin, Manuel, Carro, Belen, Sanchez-Esguevillas, Antonio, Lloret, Jaime
• Format: Article
• Language: English
• Subjects: Algorithms; Approximation; Classification; Communications traffic; Cybersecurity; Datasets; Fourier transforms; Intrusion detection; Intrusion detection systems; Kernel approximation; Kernels; Machine learning; Mathematical models; Network traffic classification; Neural networks; Phase transitions; Shallow neural network; Traffic conflicts; Training
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2019.01.063

•Most prediction problems of data networks are non-linear.•Shallow neural networks are fast.•Kernel approximation is a non-linear data transformation.•Shallow neural networks with kernel approximation are non-linear and fast models.•Proposed model is faster with prediction results comparable to deep models. Intrusion detection and network traffic classification are two of the main research applications of machine learning to highly demanding data networks e.g. IoT/sensors networks. These applications present new prediction challenges and strict requirements to the models applied for prediction. The models must be fast, accurate, flexible and capable of managing large datasets. They must be fast at the training, but mainly at the prediction phase, since inevitable environment changes require constant periodic training, and real-time prediction is mandatory. The models need to be accurate due to the consequences of prediction errors. They need also to be flexible and able to detect complex behaviors, usually encountered in non-linear models and, finally, training and prediction datasets are usually large due to traffic volumes. These requirements present conflicting solutions, between fast and simple shallow linear models and the slower and richer non-linear and deep learning models. Therefore, the perfect solution would be a mixture of both worlds. In this paper, we present such a solution made of a shallow neural network with linear activations plus a feature transformation based on kernel approximation algorithms which provide the necessary richness and non-linear behavior to the whole model. We have studied several kernel approximation algorithms: Nystrom, Random Fourier Features and Fastfood transformation and have applied them to three datasets related to intrusion detection and network traffic classification. This work presents the first application of a shallow linear model plus a kernel approximation to prediction problems with highly demanding network requirements. We show that the prediction performance obtained by these algorithms is positioned in the same range as the best non-linear classifiers, with a significant reduction in computational times, making them appropriate for new highly demanding networks.