Using Deep Learning to classify road surface conditions and to estimate the coefficient of friction

• Main Authors: Langstrand, Jens-Patrick, Randem, Hans Olav, Thunem, Harald, Hoffmann, Mario
• Format: Conference Proceeding
• Language: English
• Subjects: Cameras; Deep learning; Estimation; Friction; friction coefficient estimation; road surface condition classification; Roads; vehicle safety; Visualization; Wheels
• ISSN: 2642-7214
• DOI: 10.1109/IV55152.2023.10186717

Accurate estimation of the coefficient of friction between tyres and the road surface is important for vehicle safety. It can be used to improve active vehicle assistance systems and warn drivers of difficult road conditions. This work explores the use of RGB images and deep learning to estimate the road surface condition, as well as the coefficient of friction. A vehicle with a camera mounted in the front windshield and a friction wheel was used to collect a dataset of RGB images and friction coefficient measurements. The dataset consisted of 20,000 images of mainly wintry conditions and some dry asphalt conditions. The collected data was used to train two deep learning systems that each take RGB images as input. Firstly, a binary classifier which estimates the road surface condition that achieved a 99% accuracy. Secondly, a friction coefficient regressor that estimates the friction coefficient between the tyres of the vehicle and the road surface. The regressor uses the ResNet architecture and has an average relative error of 15% and a worst-case relative error of 194%. Our results show that images from visual light sensors may contain enough information to allow for accurate estimation of road surface condition and the underlying coefficient of friction. We believe that a larger, more comprehensive dataset would improve the performance of the estimators.