Anomaly detection and classification using a metric for determining the significance of failures: Case study: mobile network management data from LTE network

• Published in: Neural computing & applications 2017-06, Vol.28 (6), p.1265-1275
• Main Authors: Jerome, Robin Babujee, Hätönen, Kimmo
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Engineering Applications of Neural Networks; Image Processing and Computer Vision; Probability and Statistics in Computer Science
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-016-2570-7

Big data analytics and machine learning applications are often used to detect and classify anomalous behaviour in telecom network measurement data. The accuracy of findings during the analysis phase greatly depends on the quality of the training dataset. If the training dataset contains data from network elements (NEs) with high number of failures and high failure rates, such behaviour will be assumed as normal. As a result, the analysis phase will fail to detect NEs with such behaviour. Effective post-processing techniques are needed to analyse the anomalies, to determine the different kinds of anomalies, as well as their relevance in real-world scenarios. Manual post-processing of anomalies detected in an Anomaly Detection experiment is a cumbersome task, and ways to automate this process are not much researched upon. There exists no universally accepted method for effective classification of anomalous behaviour. High failure ratios have traditionally been considered as signs of faults in NEs. Operators use well-known key performance indicators (KPIs) such as drop call ratio and handover failure ratio to identify misbehaving NEs. The main problem with these KPIs based on failure ratios is their unstable nature. This paper proposes a method of measuring the significance of failures. The usage of this method is proposed in two stages of anomaly detection: training set filtering (pre-processing stage) and classification of anomalies (post-processing stage) using an automated process.