An incremental intrusion detection system using a new semi‐supervised stream classification method

• Published in: International journal of communication systems 2017-03, Vol.30 (4), p.np-n/a
• Main Authors: Noorbehbahani, Fakhroddin, Fanian, Ali, Mousavi, Rasoul, Hasannejad, Homa
• Format: Article
• Language: English
• Subjects: Classification; Computer information security; imbalanced data; incremental learning; Intrusion; intrusion detection; Intrusion detection systems; Machine learning; Networks; semi‐supervised learning; State of the art; stream classification; Streams
• ISSN: 1074-5351; 1099-1131
• DOI: 10.1002/dac.3002

Summary In recent years, the utilization of machine learning and data mining techniques for intrusion detection has received great attention by both security research communities and intrusion detection system (IDS) developers. In intrusion detection, the most important constraints are the imbalanced class distribution, the scarcity of the labeled data, and the massive amounts of network flows. Moreover, because of the dynamic nature of the network flows, applying static learned models degrades the detection performance significantly over time. In this article, we propose a new semi‐supervised stream classification method for intrusion detection, which is capable of incremental updating using limited labeled data. The proposed method, called the incremental semi‐supervised flow network‐based IDS (ISF‐NIDS), relies on an incremental mixed‐data clustering, a new supervised cluster adjustment method, and an instance‐based learning. The ISF‐NIDS operates in real time and learns new intrusions quickly using limited storage and processing power. The experimental results on the KDD99, Moore, and Sperotto benchmark datasets indicate the superiority of the proposed method compared with the existing state‐of‐the‐art incremental IDSs. We propose a new semi‐supervised stream classification method for intrusion detection, which is capable of incremental updating using limited labeled data. The proposed method handles imbalanced data and concept drift while operating in real time and learning new intrusions quickly. The experimental results on benchmark datasets indicate the superiority of the proposed method compared with the existing state‐of‐the‐art incremental IDSs.