Uncertainty-aware specification and analysis for hardware-in-the-loop testing of cyber-physical systems

• Published in: The Journal of systems and software 2021-01, Vol.171, p.110813, Article 110813
• Main Authors: Shin, Seung Yeob, Chaouch, Karim, Nejati, Shiva, Sabetzadeh, Mehrdad, Briand, Lionel C., Zimmer, Frank
• Format: Article
• Language: English
• Subjects: Cyber-physical systems; Machine learning; Model checking; Simulation; Test case specification and analysis; UML profile
• ISSN: 0164-1212; 1873-1228
• DOI: 10.1016/j.jss.2020.110813

Hardware-in-the-loop (HiL) testing is important for developing cyber-physical systems (CPS). HiL test cases manipulate hardware, are time-consuming and their behaviors are impacted by the uncertainties in the CPS environment. To mitigate the risks associated with HiL testing, engineers have to ensure that (1) test cases are well-behaved, e.g., they do not damage hardware, and (2) test cases can execute within a time budget. Leveraging the UML profile mechanism, we develop a domain-specific language, HITECS, for HiL test case specification. Using HITECS, we provide uncertainty-aware analysis methods to check the well-behavedness of HiL test cases. In addition, we provide a method to estimate the execution times of HiL test cases before the actual HiL testing. We apply HITECS to an industrial case study from the satellite domain. Our results show that: (1) HITECS helps engineers define more effective assertions to check HiL test cases, compared to the assertions defined without any systematic guidance; (2) HITECS verifies in practical time that HiL test cases are well-behaved; (3) HITECS is able to resolve uncertain parameters of HiL test cases by synthesizing conditions under which test cases are guaranteed to be well-behaved; and (4) HITECS accurately estimates HiL test case execution times. •HITECS language for specifying HiL test cases of cyber-physical systems.•HITECS model checking for ensuring that HiL test cases are well-behaved.•HITECS uncertainty resolution for identifying well-behaved HiL test conditions.•HITECS simulation for estimating the execution time of HiL test cases.•HITECS framework is evaluated on an industry case study from the satellite domain.