Automatic analysis of malware behavior using machine learning

• Published in: Journal of computer security 2011-01, Vol.19 (4), p.639-668
• Main Authors: Rieck, Konrad, Trinius, Philipp, Willems, Carsten, Holz, Thorsten
• Format: Article
• Language: English
• Subjects: Classification; Clustering; Computer information security; Computer programs; Dynamics; Internet; Machine learning; Run time (computers)
• ISSN: 0926-227X; 1875-8924
• DOI: 10.3233/JCS-2010-0410

Malicious software so called malware poses a major threat to the security of computer systems. The amount and diversity of its variants render classic security defenses ineffective, such that millions of hosts in the Internet are infected with malware in the form of computer viruses, Internet worms and Trojan horses. While obfuscation and polymorphism employed by malware largely impede detection at file level, the dynamic analysis of malware binaries during run-time provides an instrument for characterizing and defending against the threat of malicious software.In this article, we propose a framework for the automatic analysis of malware behavior using machine learning. The framework allows for automatically identifying novel classes of malware with similar behavior (clustering) and assigning unknown malware to these discovered classes (classification). Based on both, clustering and classification, we propose an incremental approach for behavior-based analysis, capable of processing the behavior of thousands of malware binaries on a daily basis. The incremental analysis significantly reduces the run-time overhead of current analysis methods, while providing accurate discovery and discrimination of novel malware variants.