HEDGE: Heterogeneous Semantic Dynamic Graph Framework for Log Anomaly Detection in Digital Service Network

Log anomaly detection in digital service networks is challenging due to the heterogeneity and complexity of log formats and semantics. Traditional log anomaly detection methods struggle with two main challenges: the inability to directly correlate heterogeneous logs and the semantic heterogeneity ac...

Full description

Saved in:
Bibliographic Details
Main Authors: Qian, Bohao, Zhu, Mengying, Yang, Mengyuan, Wu, Enze, Xie, Guojie, Liang, Yuebing, Zheng, Xiaolin
Format: Conference Proceeding
Language:English
Subjects:
Anomaly detection
Complexity theory
Context modeling
Correlation
Digital service network
Dynamic graph
Heterogeneous graph
Large language model
Log anomaly detection
Monitoring
Protection
Robustness
Semantics
Soft sensors
Web services
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Log anomaly detection in digital service networks is challenging due to the heterogeneity and complexity of log formats and semantics. Traditional log anomaly detection methods struggle with two main challenges: the inability to directly correlate heterogeneous logs and the semantic heterogeneity across and within logs. To address these challenges, we propose a novel framework, HEDGE, which constructs a dynamic heterogeneous log graph to capture spatio-temporal relationships between logs, reflecting fine-grained semantic correlations and evolutionary properties of sequential logs comprehensively and detecting log anomalies effectively. To capture log representations under heterogeneity from both semantic and spatio-temporal perspectives, HEDGE not only pre-trains a dual-tower SemanticFormer based on BERT to align global and local semantic information for heterogeneous nodes but also adopts a dynamic heterogeneous graph model to learn spatio-temporal topological features within inner-snapshot and intra-snapshot contexts. Extensive experiments on public datasets demonstrate the superiority of our framework compared to state-of-the-art baselines.
ISSN:2836-3868
DOI:10.1109/ICWS62655.2024.00041