Densely Connected Transformer With Linear Self-Attention for Lightweight Image Super-Resolution
Kun Zeng, Hanjiang Lin, Zhiqiang Yan, Jinsheng Fang
Abstract
Image super-resolution (SR) is the process of restoring high-resolution (HR) images from low-resolution (LR) ones. Recent Transformer-based SR methods have achieved impressive results by utilizing the self-attention (SA) mechanism, which allows modeling long-range dependencies among input features in spatial dimensions. However, the computational complexity of SA increases quadratically with respect to the feature size, which makes Transformer-based methods inefficient. Additionally, despite the success of dense connections in CNN-based methods, they have not been fully explored in Transformer-based methods. In this paper, we propose a novel approach for lightweight SR, called Densely Connected Transformer with Linear Self-Attention (DCTLSA) network. Our method addresses the efficiency issue of SA by designing a new linear SA, which calculates the similarities in spatial dimension with linear complexity. Moreover, we leverage dense connections to integrate multiple levels of features and provide rich information for SR. Our experimental results demonstrate that DCTLSA outperforms state-of-the-art lightweight SR methods in terms of SR performance, model complexity, and inference speed. The code of the proposed method is available at https://github.com/zengkun301/DCTLSA.