MMID-Bench: A Comprehensive Benchmark for Multi-domain Multi-category Intrusion Detection

• Published in: IEEE transactions on intelligent vehicles 2024, p.1-17
• Main Authors: Han, Fujun, Ye, Peng, She, Chunyan, Duan, Shukai, Wang, Lidan, Liu, Derong
• Format: Article
• Language: English
• Subjects: Adaptation models; Datasets; Evaluation Metrics; Framework; Intelligent vehicles; Intrusion detection; Measurement; Meteorology; Multi-domain Multi-category Intrusion detection; Task analysis; Vehicle dynamics
• ISSN: 2379-8858; 2379-8904
• DOI: 10.1109/TIV.2024.3367895

Vision-based intrusion detection tasks have extensive applications in crucial fields such as automatic driving, intelligent security, and monitoring. Previous vision-based intrusion detection methods have mainly focused on whether pedestrians intrude on a restricted Area-of-Interest (AoI) from a static or dynamic view. However, in real-world scenarios, intrusion detection tasks often involve a variety of intrusion categories and encounter some challenging environments, beyond just pedestrians in a normal environment. Therefore, exploring related tasks of dynamicview multi-domain multi-category intrusion detection (namely MM-ID) is important and valuable but remains unexplored. In this work, we propose a comprehensive benchmark to address the aforementioned intrusion detection tasks, including multiple datasets, reasonable metrics, End-to-End framework design, and extensive evaluations. First, we develop five publicly available datasets, namely, Normal-Cityintrusion-Multicategory (NormalCMC), Foggy-Cityintrusion-Multicategory (Foggy-CMC), RainyCityintrusion-Multicategory (Rainy-CMC), Night-CityintrusionMulticategory (Night-CMC), and BDD-intrusion, conduct statistical analysis on these datasets and design three evaluation metrics. Then, an efficient End-to-End framework (MMID-YOLO) is proposed for the MM-ID task, with two effective improvements: (1) We propose a new unsupervised domain adaptation algorithm (namely, I-DANN) to conduct important feature alignment between the Source domain and Target domain. (2) We propose a diffusion model-based data augmentation method to improve the generalization of the model to unknown environments. Further, we design and propose three training schemes to conduct comprehensive experiments and comparisons, which can also serve as strong baselines for our MM-ID task: the Combined model, the End-to-End model, and the Domain Adaption model, respectively. The experimental results show that our MMIDYOLO can perform much better than other schemes and reach the SOTA performance