DeepQCD: An end-to-end deep learning approach to quickest change detection

• Published in: Journal of the Franklin Institute 2024-12, Vol.361 (18), p.107199, Article 107199
• Main Authors: Kurt, Mehmet Necip, Zheng, Jiaohao, Yilmaz, Yasin, Wang, Xiaodong
• Format: Article
• Language: English
• Subjects: Deep learning; Internet of things; Quickest change detection; Surveillance videos; Temporal correlation; Transient change
• ISSN: 0016-0032
• DOI: 10.1016/j.jfranklin.2024.107199

This paper aims to generalize the quickest change detection (QCD) framework via a data-driven approach. To this end, a generic neural network architecture is proposed for the QCD task, composed of feature transformation, recurrent, and dense layers. The neural network is trained end-to-end to learn the change detection rule directly from data without needing the knowledge of probabilistic data models. Specifically, the feature transformation layers can perform a broad range of operations including feature extraction, scaling, and normalization. The recurrent layers keep an internal state summarizing the time-series data seen so far and update the state as new information comes in. Finally, the dense layers map the internal state into a decision statistic, defined as the posterior probability that a change has taken place. Comparisons with the existing model-based QCD algorithms demonstrate the power of the proposed data-driven approach, called DeepQCD, under several scenarios including transient changes and temporally correlated data streams. Experiments with real-world data illustrate superior performance of DeepQCD compared to state-of-the-art algorithms in real-time anomaly detection over surveillance videos and real-time attack detection over Internet of Things (IoT) networks. •A novel data-driven algorithm, DeepQCD, is proposed for quickest change detection.•DeepQCD is the first method to learn QCD through an end-to-end trained neural network.•DeepQCD unifies the QCD framework, adapting to diverse data and change-point models.•Experiments show DeepQCD outperforms state-of-the-art in video anomaly and IoT attack detection.