Self-Supervised Learning of Camera-based Drivable Surface Friction

The visual predictor of a drivable surface friction ahead of the vehicle is presented. The image recognition neural network is trained in self-supervised fashion, as an alternative to tedious, error-prone, and subjective human annotation. The training images are labelled automatically by surface friction estimates from vehicle response during ordinary driving. The Unscented Kalman Filter algorithm is used to estimate tire-to-road interface friction parameters, taking into account the highly nonlinear nature of tire dynamics. Finally, the overall toolchain was validated using an experimental subscale platform and real-world driving scenarios. The resulting visual predictor was trained using about 3 000 images and validated on an unseen set of 800 test images, achieving 0.98 crosscorrelation between the visually predicted and the estimated value of surface friction.