Deep Rectangling for Image Stitching: A Learning Baseline
Lang Nie, Chun-Yu Lin, Kang Liao, Shuaicheng Liu, Yao Zhao
Abstract
Stitched images provide a wide field-of-view (FoV) but suffer from unpleasant irregular boundaries. To deal with this problem, existing image rectangling methods devote to searching an initial mesh and optimizing a target mesh to form the mesh deformation in two stages. Then rectangu-lar images can be generated by warping stitched images. However, these solutions only work for images with rich linear structures, leading to noticeable distortions for por-traits and landscapes with non-linear objects. In this paper, we address these issues by proposing the first deep learning solution to image rectangling. Con-cretely, we predefine a rigid target mesh and only estimate an initial mesh to form the mesh deformation, contributing to a compact one-stage solution. The initial mesh is predicted using a fully convolutional network with a resid-ual progressive regression strategy. To obtain results with high content fidelity, a comprehensive objective function is proposed to simultaneously encourage the boundary rect-angular, mesh shape-preserving, and content perceptually natural. Besides, we build the first image stitching rectan-gling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.