Lightweight Context-Aware Network Using Partial-Channel Transformation for Real-Time Semantic Segmentation
Min Shi, Shaowen Lin, Qingming Yi, Jian Weng, Aiwen Luo, Yicong Zhou
Abstract
Optimizing the computational efficiency of the artificial neural networks is crucial for resource-constrained platforms like autonomous driving systems. To address this challenge, we proposed a Lightweight Context-aware Network (LCNet) that accelerates semantic segmentation while maintaining a favorable trade-off between inference speed and segmentation accuracy in this paper. The proposed LCNet introduces a partial-channel transformation (PCT) strategy to minimize computing latency and hardware requirements of the basic unit. Within the PCT block, a three-branch context aggregation (TCA) module expands the feature receptive fields, capturing multiscale contextual information. Additionally, a dual-attention-guided decoder (DD) recovers spatial details and enhances pixel prediction accuracy. Extensive experiments on three benchmarks demonstrate the effectiveness and efficiency of the proposed LCNet model. Remarkably, a smaller model LCNet <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{3\_7}$</tex-math> </inline-formula> achieves 73.8% mIoU with only 0.51 million parameters, with an impressive inference speed of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim$</tex-math> </inline-formula> 142.5 fps and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim$</tex-math> </inline-formula> 9 fps using a single RTX 3090 GPU and Jetson Xavier NX, respectively, on the Cityscapes test set at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1024\times 1024$</tex-math> </inline-formula> resolution. A more accurate version of the LCNet <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{3\_11}$</tex-math> </inline-formula> can achieve 75.8% mIoU with 0.74 million parameters at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim$</tex-math> </inline-formula> 117 fps inference speed on Cityscapes at the same resolution. Much faster inference speed can be achieved at smaller image resolutions. LCNet strikes a great balance between computational efficiency and prediction capability for mobile application scenarios. The code is available at https://github.com/lztjy/LCNet.