Realtime Feature Engineering for Anomaly Detection in IoT based MQTT Networks

• Published in: IEEE access 2024-01, Vol.12, p.1-1
• Main Authors: Imran, Zuhairi, Megat F., Ali, Syed Mubashir, Shahid, Zeeshan, Alam, Muhammad Mansoor, Su'ud, Mazliham Mohd
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Anomalies; Anomaly detection; Behavioral sciences; Computing costs; Data conversion; Data models; Datasets; Decision trees; Denial of service attacks; Denial-of-service attack; DoS; Engineering; Feature extraction; Filtration; Internet of Things; IoT; Model accuracy; MQTT; Random forests; Security; System effectiveness; Testing time; Training
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3363889

The MQTTset dataset has become a focal point in the realm of anomaly detection within IoT-based systems. This study involves into refining anomaly detection techniques by employing various filtering methods, such as data conversion, attribute filtering, handling missing values, and scaling. The primary objective is to enhance the identification of anomalies, with a particular focus on detecting Denial of Service (DoS) attacks. The research not only examines existing techniques but also addresses a significant gap in MQTT traffic anomaly detection. To fill this void, the study proposes the integration of the 'source' attribute extracted from PCAP files, leveraging hand-crafted feature engineering. This addition aims to provide a more comprehensive understanding of the anomalies present in MQTT traffic. Moreover, the research undertakes the crucial task of categorizing and prioritizing anomalies based on their frequency of occurrence. Special attention is given to discerning patterns associated with DoS attacks, offering a nuanced approach to anomaly identification. To evaluate the efficacy of the proposed feature engineering, the study conducts a comparative analysis of prediction accuracy, F1 score, and computational metrics (training and testing time). The benchmark algorithm under scrutiny is the decision tree, along with its eight variant models, namely Iterative Dichotomiser (ID3), C4.5, Random Forest, CatBoost, LightGBM, XGBoost, Classification and Regression Tree (CART), and Gradient Boosting. This comprehensive evaluation aims to provide insights into the performance enhancements achieved through the incorporation of hand-crafted feature engineering in the MQTTset dataset for anomaly detection in IoT-based systems. Also, hyperparameter fine-tuning techniques such as grid search and random search are employed to further refine model performance, enhance accuracy, and reduce computational costs. This anomaly detection technique used in the proposed method is able to provide early threat warnings and enhance overall system security and reliability in IoT-based systems. Experiment results show that the benchmark Decision Tree model achieved 92.57% accuracy and a 92.38% F1 score with training and testing times of 2.95 seconds and 0.86 seconds, respectively. The Feature Engineering (Modified) dataset yielded a remarkable improvement, with 98.56% accuracy, a 98.50% F1 score, and comparable training and testing times of 0.70 seconds and 0.02 seconds, respec