Developing an Intrusion Detection Framework for High-Speed Big Data Networks: A Comprehensive Approach

In network intrusion detection research, two characteristics are generally considered vital to building efficient intrusion detection systems (IDSs): an optimal feature selection technique and robust classification schemes. However, the emergence of sophisticated network attacks and the advent of bi...

Full description

Saved in:
Bibliographic Details
Published in:KSII transactions on Internet and information systems 2018, 12(8), , pp.4021-4037
Main Authors: Siddique, Kamran, Akhtar, Zahid, Khan, Muhammad Ashfaq, Jung, Yong-Hwan, Kim, Yangwoo
Format: Article
Language:English
Subjects:
Algorithms
Analysis
Big data
Data security
Machine learning
Methods
Safety and security measures
컴퓨터학
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In network intrusion detection research, two characteristics are generally considered vital to building efficient intrusion detection systems (IDSs): an optimal feature selection technique and robust classification schemes. However, the emergence of sophisticated network attacks and the advent of big data concepts in intrusion detection domains require two more significant aspects to be addressed: employing an appropriate big data computing framework and utilizing a contemporary dataset to deal with ongoing advancements. As such, we present a comprehensive approach to building an efficient IDS with the aim of strengthening academic anomaly detection research in real-world operational environments. The proposed system has the following four characteristics: (i) it performs optimal feature selection using information gain and branch-and-bound algorithms; (ii) it employs machine learning techniques for classification, namely, Logistic Regression, Naive Bayes, and Random Forest; (iii) it introduces bulk synchronous parallel processing to handle the computational requirements of large-scale networks; and (iv) it utilizes a real-time contemporary dataset generated by the Information Security Centre of Excellence at the University of Brunswick (ISCX-UNB) to validate its efficacy. Experimental analysis shows the effectiveness of the proposed framework, which is able to achieve high accuracy, low computational cost, and reduced false alarms. Keywords: Network intrusion detection systems, anomaly detection, bulk synchronous parallel, BSP, big data, machine learning, Darpa, KDD Cup 99, ISCX-UNB dataset
ISSN:1976-7277
1976-7277
DOI:10.3837/tiis.2018.08.026