Impact of class distribution on the detection of slow HTTP DoS attacks using Big Data

• Published in: Journal of big data 2019-07, Vol.6 (1), p.1-18, Article 67
• Main Authors: Calvert, Chad L., Khoshgoftaar, Taghi M.
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Big Data; Class imbalance; Communications Engineering; Communications traffic; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Data sampling; Database Management; Datasets; Denial of service attacks; Information Storage and Retrieval; Intrusion; Machine learning; Mathematical Applications in Computer Science; Networks; Protocol (computers); Slow HTTP DoS
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-019-0230-3

The integrity of modern network communications is constantly being challenged by more sophisticated intrusion techniques. Attackers are consistently shifting to stealthier and more complex forms of attacks in an attempt to bypass known mitigation strategies. In recent years, attackers have begun to focus their attack efforts on the application layer, allowing them to produce attacks that can exploit known issues within specific application protocols. Slow HTTP Denial of Service attacks are one such attack variant, which targets the HTTP protocol and can imitate legitimate user traffic in order to deny resources from a service. Successful mitigation of this attack type requires network analysts to evaluate large quantities of network traffic to identify and block intrusive traffic. The issue, is that the number of legitimate traffic instances can far outnumber the amount of attack instances, making detection problematic. Machine learning techniques can be used to aid in detection, but the large level of imbalance between normal (majority) and attack (minority) instances can lead to inaccurate detection results. In this work, we evaluate the use of data sampling to produce varying class distributions in order to counteract the effects of severely imbalanced Slow HTTP DoS big datasets. We also detail our process for collecting real-world representative Slow HTTP DoS attack traffic from a live network environment to create our datasets. Five class distributions are generated to evaluate the Slow HTTP DoS detection performance of eight machine learning techniques. Our results show that the optimal learner and class distribution combination is that of Random Forest with a 65:35 distribution ratio, obtaining an AUC value of 0.99904. Further, we determine through the use of significance testing, that the use of sampling techniques can significantly increase learner performance when detecting Slow HTTP DoS attack traffic.