Log-Based Fault Localization with Unsupervised Log Segmentation

• Published in: Applied sciences 2024-09, Vol.14 (18), p.8421
• Main Authors: Dobrowolski, Wojciech, Iwach-Kowalski, Kamil, Nikodem, Maciej, Unold, Olgierd
• Format: Article
• Language: English
• Subjects: automated log analysis; Deep learning; Failure; Information sources; Localization; log sequence; log-based fault localization; Methods; Rankings; Software; software reliability; Telecommunications equipment industry; unsupervised log sequence segmentation
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app14188421

Localizing faults in a software is a tedious process. The manual approach is becoming impractical because of the large size and complexity of contemporary computer systems as well as their logs, which are often the primary source of information about the fault. Log-based Fault Localization (LBFL) is a popular method applied for this purpose. However, in real-world scenarios, this method is vulnerable to a large number of previously unseen log lines. In this paper, we propose a novel method that can guide programmers to the location of a fault by creating a hierarchy of log lines with the highest rank, selected by the traditional LBFL method. We use the intuition that the symptoms of faults are in the context of normal behavior, whereas suspicious log lines grouped together are from new or additional functionalities turned on during faulty execution. To obtain this context, we used unsupervised log sequence segmentation, which has been previously used to segment log sequences into meaningful segments. Experiments on real-life examples show that our method reduces the effort to find the most crucial logs by up to 64% compared with the traditional timestamp approach. We demonstrate that context is highly useful in advancing fault localization, showing the possibility of further speeding up the process.