SAFE-OCC: A novelty detection framework for Convolutional Neural Network sensors and its application in process control

• Published in: Journal of process control 2022-09, Vol.117 (C), p.78-97
• Main Authors: Pulsipher, Joshua L., Coutinho, Luke D.J., Soderstrom, Tyler A., Zavala, Victor M.
• Format: Article
• Language: English
• Subjects: Computer vision; Deep learning; Novelty detection; Process control
• ISSN: 0959-1524; 1873-2771
• DOI: 10.1016/j.jprocont.2022.07.006

We present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments. •Present novelty detection framework for convolutional neural net (CNN) sensors.•CNN sensors enable incorporation of image data into control architectures.•Novelty framework enables robust predictions by CNN sensors.