Lightweight Context-Aware Network Using Partial-Channel Transformation for Real-Time Semantic Segmentation

Optimizing the computational efficiency of the artificial neural networks is crucial for resource-constrained platforms like autonomous driving systems. To address this challenge, we proposed a Lightweight Context-aware Network (LCNet) that accelerates semantic segmentation while maintaining a favorable trade-off between inference speed and segmentation accuracy in this paper. The proposed LCNet introduces a partial-channel transformation (PCT) strategy to minimize computing latency and hardware requirements of the basic unit. Within the PCT block, a three-branch context aggregation (TCA) module expands the feature receptive fields, capturing multiscale contextual information. Additionally, a dual-attention-guided decoder (DD) recovers spatial details and enhances pixel prediction accuracy. Extensive experiments on three benchmarks demonstrate the effectiveness and efficiency of the proposed LCNet model. Remarkably, a smaller model LCNet $_{3\_7}$ achieves 73.8% mIoU with only 0.51 million parameters, with an impressive inference speed of $\sim$ 142.5 fps and $\sim$ 9 fps using a single RTX 3090 GPU and Jetson Xavier NX, respectively, on the Cityscapes test set at $1024\times 1024$ resolution. A more accurate version of the LCNet $_{3\_11}$ can achieve 75.8% mIoU with 0.74 million parameters at $\sim$ 117 fps inference speed on Cityscapes at the same resolution. Much faster inference speed can be achieved at smaller image resolutions. LCNet strikes a great balance between computational efficiency and prediction capability for mobile application scenarios. The code is available at https://github.com/lztjy/LCNet.