Deep-Learning-Based Super-Resolution of Video Satellite Imagery by the Coupling of Multiframe and Single-Frame Models
Huanfeng Shen, Zhonghang Qiu, Linwei Yue, Liangpei Zhang
Abstract
Image super-resolution (SR) is an effective solution to the limitation of the spatial resolution of video satellite images, which is caused by the degradation and compression in the imaging phase. For the processing of satellite videos, the commonly employed deep-learning-based single-frame SR (SFSR) framework has limited performance without using complementary information between the video frames. On the other side, the multiframe SR (MFSR) can utilize temporal subpixel information to super-resolve the high-resolution (HR) imagery. However, although deeper and wider deep learning network provides powerful feature representations for SR methods, it has always been a challenge to accurately reconstruct the boundaries of ground objects in video satellite images. In this article, to address these issues, we propose an edge-guided video SR (EGVSR) framework for video satellite image SR, which couples the MFSR model and the edge-SFSR (E-SFSR) model in a unified network. The EGVSR framework is composed of an MFSR branch and an edge branch. The MFSR branch is used to extract the complementary features from the consecutive video frames. Concurrently, the edge branch acts as an SFSR model to translate the edge maps from the low-resolution modality to the HR one. At the final SR stage, the DBFM is built to focus on the promising inner representations of the features of the two branches and fuse them. Extensive experiments on video satellite imagery show that the proposed EGVSR method can achieve superior performance compared to the representative deep-learning-based SR methods.