An empirical study of problems and evaluation of IoT malware classification label sources

• Published in: Journal of King Saud University. Computer and information sciences 2024-01, Vol.36 (1), p.101898, Article 101898
• Main Authors: Lei, Tianwei, Xue, Jingfeng, Wang, Yong, Baker, Thar, Niu, Zequn
• Format: Article
• Language: English
• Subjects: Anti-Virus engine label; IoT malware analysis; Label tool; Malware classification; Malware label
• ISSN: 1319-1578; 2213-1248
• DOI: 10.1016/j.jksuci.2023.101898

With the proliferation of malware on IoT devices, research on IoT malicious code has also become more mature. Most studies use learning models to detect or classify malware. Therefore, ensuring high-quality labels for malware samples is crucial to maintaining research accuracy. Researchers typically submit malware samples to Anti-Virus (AV) engines to obtain labels, but different engines have varying rules for detecting maliciousness and variants. This study aims to improve future IoT malware research accuracy by investigating label quality. We address three label-related issues, including Anti-Virus detection technology, naming rules, and label expiration. Additionally, we examine multiple sources of malware labels, including 63 studies on IoT, Windows, and Android malware, as well as commonly used tools such as AVClass and Anti-Virus engines. To evaluate and recommend label sources, we construct classification models using an IoT malware dataset obtained from VirusShare, which is labeled with common tools and Anti-Virus engines and classified based on ELF features. For family classification, we investigate four methods and tools, and for variant hierarchy classification, we compare label overlaps with sample clustering from 12 Anti-Virus engines. Based on our findings, we recommend AVClass for obtaining labels for IoT family classification. For small-scale malware families at the variant level, we recommend using the labels from Ad-Aware, BitDefender, and Emsisoft engines. For large-scale malware families, we advise employing labels from Jiangmin, NANO-Antivirus, and Avira engines, serving as a valuable guide for future IoT malware research. [Display omitted] •Present an empirical study on the quality and sources of labels for IoT malware.•Investigate the issues with IoT malware labels and potential causes.•Analyse malware labels and survey dataset label sources for future research use.•Build an IoT malware dataset and assess label sources at family and variant levels.