Deep Learning-Based Raindrop Quantity Detection for Real-Time Vehicle-Safety Application

• Published in: IEEE transactions on consumer electronics 2021-11, Vol.67 (4), p.266-274
• Main Authors: Wang, Szu-Hong, Hsia, Shih-Chang, Zheng, Meng-Jie
• Format: Article
• Language: English
• Subjects: Accuracy; ADAS; AI accelerator; artificial intelligence; Artificial neural networks; Automatic control; Cameras; Computational modeling; Computer architecture; computer vision; convolution neural network; Deep learning; embedded system; Embedded systems; Frames per second; Image classification; Machine learning; Neural networks; Rain; Raindrop detection; Raindrops; Real time; ResNet; Training; Training data; Vehicle safety; Video data; Windshields
• ISSN: 0098-3063; 1558-4127
• DOI: 10.1109/TCE.2021.3127494

The raindrops on the glass will affect driving safety, such as rear-view camera, outside mirror and windshield, etc. This article proposed a robust raindrop detection using deep learning on embedded platform with AI accelerator for real-time implementation. A training model is established through a convolution neural network (CNN)-like architecture to classify the images by the vehicle camera into three classes: no rain, heavy rain, and light rain. The classification results are used to control the speed of the motor to implement an automatic wiper control system. The training model, ResNet, is used to classify the image with good tradeoff between the computational cost and accuracy. For real-time application, the camera module on the Google Coral Dev board on embedded system platform is used to test the video stream and to estimate the performance of this system. Results show that the recognition accuracy reaches 95%, and the processing speed can achieve 20 frames per second (fps) on the embedded system.