Efficient Determination of Safety Requirements for Perception Systems

• Main Authors: Katz, Sydney M., Corso, Anthony L., Yel, Esen, Kochenderfer, Mykel J.
• Format: Conference Proceeding
• Language: English
• Subjects: Aerospace electronics; black-box estimation; Closed box; Estimation; Gaussian processes; perception; Safety; Smoothing methods; System performance
• ISSN: 2155-7209
• DOI: 10.1109/DASC58513.2023.10311157

Perception systems operate as a subcomponent of the general autonomy stack, and perception system designers often need to optimize performance characteristics while maintaining safety with respect to the overall closed-loop system. For this reason, it is useful to distill high-level safety requirements into component-level requirements on the perception system. In this work, we focus on efficiently determining sets of safe perception system performance characteristics given a black-box simulator of the fully-integrated, closed-loop system. We combine the advantages of common black-box estimation techniques such as Gaussian processes and threshold bandits to develop a new estimation method, which we call smoothing bandits. We demonstrate our method on a vision-based aircraft collision avoidance problem and show improvements in terms of both accuracy and efficiency over the Gaussian process and threshold bandit baselines.