Log Anomaly Detection by Adversarial Autoencoders With Graph Feature Fusion

The exponential growth of scale and complexity in distributed systems necessitates significant maintenance efforts. Logs play an indispensable role in system operation and maintenance since they record crucial runtime information. However, recent studies on log anomaly detection have primarily focus...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on reliability 2024-03, Vol.73 (1), p.637-649
Main Authors: Xie, Yuxia, Yang, Kai
Format: Article
Language:English
Subjects:
Ablation
Adversarial machine learning
Adversarial training
Anomalies
Anomaly detection
autoencoder
Autoencoders
Behavioral sciences
Complexity
Computer networks
Deep learning
event graph
Feature extraction
Graphs
Heuristic algorithms
Hidden Markov models
knowledge graph
Knowledge graphs
Knowledge representation
log analysis
Machine learning algorithms
Maintenance
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The exponential growth of scale and complexity in distributed systems necessitates significant maintenance efforts. Logs play an indispensable role in system operation and maintenance since they record crucial runtime information. However, recent studies on log anomaly detection have primarily focused on deep learning methods, which entail high computational complexity for learning temporal and semantic features from logs. Moreover, most deep learning-based approaches for log anomaly detection require supervised training, which is labor intensive. To address these challenges, this article proposes a framework called GAE-Log . GAE-Log leverages event graphs and knowledge graphs to model logs comprehensively. By integrating temporal dynamics through event graphs and incorporating contextual information from knowledge graphs, GAE-Log enhances the understanding of the system's status. Moreover, GAE-Log employs adversarial training of autoencoders for anomaly detection on logs. The effectiveness of GAE-Log is evaluated through an ablation study and comprehensive comparisons using both public and synthetic log datasets. The results demonstrate that GAE-Log outperforms state-of-the-art methods in log anomaly detection, achieving significant performance improvements.
ISSN:0018-9529
1558-1721
DOI:10.1109/TR.2023.3305376