Transferring Transformer-Based Models for Cross-Area Building Extraction From Remote Sensing Images
Chunping Qiu, He Li, Wenyue Guo, Xin Chen, Anzhu Yu, Xiaochong Tong, Michael Schmitt
Abstract
Extracting buildings from remote sensing images is an important task with a variety of applications. Considerable attention has focused on achieving new SOTA accuracy with more and more advanced deep learning models. However, the developed models still hardly generalize across geographical areas, hindering the practical use of SOTA approaches. To attack this problem, we established a baseline for model cross-area generalization ability using available datasets for BE. In addition to two popular FCN-based models, we first adapted two novel transformer-based models, Swin Transformer and SegFormer, which are all able to output SOTA accuracy with no big difference when tested within one area. However, experimental results show that all models fail to generalize to a different area. We then propose to fine-tune pre-trained models from one area on a small subset of an unseen area, the effectiveness of which depends on the model choice and the data size for tuning. By jointly taking advantage of the transfer learning idea and the multiscale feature learning ability of SegFormer, a distinct improvement has been achieved compared to results from Swin Transformer and FCN-based models trained on the same amount of data. Commonly used metric, IoU, can be increased from 38.97% to 70.86%, and from 48.36% to 74.51%, when using 10% and 30% subset of the targeting area, respectively. The influence of model choice and data size for tuning has also been investigated. Our work contributes to complementing the algorithm development and within-area model evaluation in the hot field of BE from RS images.