Log Analysis and Prediction for Anomaly Detection in Network Switches

• Main Authors: Nam, Sukhyun, Jeong, Euidong, Hong, Jibum, Yoo, Jae-Hyoung, Hong, James Won-Ki
• Format: Conference Proceeding
• Language: English
• Subjects: Anomaly detection; Classification algorithms; Log Analysis; Machine Learning; Prediction algorithms; Predictive models; Systems operation
• ISSN: 2165-963X
• DOI: 10.23919/CNSM59352.2023.10327879

In this study, we propose a three-step anomaly detection system for network switches. The proposed system consists of the following steps: 1) Log parsing, where log messages from switches are analyzed to identify patterns and events, 2) Analysis of the identified event flow to distinguish normal and abnormal event sequences, and 3) Prediction of the next log message, with detection of anomalies if the predicted log message differs from the normal log messages. For event classification, a log parser is proposed by modifying existing algorithms, and experimental results confirm that similar log patterns are correctly classified into the same event. To learn normal event sequences, both FSM and LSTM models are trained. Lastly, we proposed a BERT-LSTM model to predict the next log message and detect unexpected log messages. The proposed system is validated using data collected from a constructed testbed and achieves a high-performance level with an F1 score of 83.72%. Notably, our system achieved a recall of 94.74%. Our system has an advantage in that if misclassified cases occur, network administrators can retrain each model to improve precision during system operation.