Hybrid deep neural networks to infer state models of black-box systems

• Main Authors: Mashhadi, Mohammad Jafar, Hemmati, Hadi
• Format: Conference Proceeding
• Language: English
• Subjects: Black-box Model Inference; Classification algorithms; Computing methodologies > Machine learning > Machine learning approaches > Neural networks; Convolutional Neural Network; Deep Learning; Instruments; Recurrent Neural Network; Software and its engineering > Software creation and management > Designing software > Requirements analysis; Software and its engineering > Software creation and management > Software post-development issues > Software reverse engineering; Software engineering; Specification Mining; Task analysis; Testing; Time measurement; Time series; Time series analysis
• ISSN: 2643-1572
• DOI: 10.1145/3324884.3416559

Inferring behavior model of a running software system is quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, instrumenting the entire source code is not possible (e.g., when using black-box third-party libraries) or might be very costly. Unfortunately, most black-box techniques that detect states over time are either univariate, or make assumptions on the data distribution, or have limited power for learning over a long period of past behavior. To overcome the above issues, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input/output signal of the system, and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns, over time. We have applied our approach on a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent system-level test cases and inferred states, over time. Our comparison with several traditional time series change point detection techniques showed that our approach improves their performance by up to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms by up to 17%.