Infrared Polarization-Empowered Full-Time Road Detection via Lightweight Multi-Pathway Collaborative 2D/3D Convolutional Networks

Automatic roads detection is an essential task for traffic safety and intelligent transportation systems. Recently the long-wave infrared (LWIR) polarization imaging-based road detection technique has obtained significant progresses. However, the joint analysis among multiple polarization characteristics, sparse inter-channel information (along the $z$ -axis), and dense intra-channel information (inside the $x$ - $y$ plane), have not been considered effectively, hindering the effective detection of many road areas. Additionally, most of the existing methods often encounter a challenging trade-off between achieving high precision and maintaining a lightweight design. To tackle these issues, this paper presents a novel Lightweight Multi-Pathway Collaborative 2D/3D Convolutional Networks (LMPC2D3DCNet) with a small number of parameters for full-time road detection. Our LMPC2D3DCNet is the first attempt to incorporate 2D and 3D convolutional networks to balance extraction for sparse inter-channel polarization information and dense intra-channel polarization information, in which a new Cross 2D-3D Non-Local Attention (C2D3DNLA) network is proposed to derive respective latent features by exploiting both local and global polarization correlations. Meanwhile, it also follows the design of a multipath network structure that elegantly fuses plenty of low-frequency, high-frequency, and multiscale polarization information, thus obtaining more accurate modeling for road regions. Extensive experiments on one public infrared polarization dataset of road scenes demonstrate that our proposed LMPC2D3DCNet (The code will release soon on https://github.com/XueqiangF) achieves PRE of 96.96%, REC of 96.71%, OA of 99.45%, F1 of 96.72, BER of 1.80% and IoU of 93.85%, and outperforms significantly state-of-the-art methods.