Optical and Synthetic Aperture Radar Image Fusion for Ship Detection and Recognition: Current state, challenges, and future prospects
Zenghui Zhang, Limeng Zhang, Juanping Wu, Weiwei Guo
Abstract
The detection and recognition of ships hold crucial significance across various domains, including transportation, fishing, smuggling prevention, and rescue operations. To achieve accurate and efficient ship detection and recognition, remote sensing technologies, specifically optical and synthetic aperture radar (SAR) sensors, have been widely employed. However, each sensor modality presents its own limitations, such as cloud cover in optical images and interference in SAR images, resulting in restricted data availability and false alarms during detection. This paper provides a comprehensive summary of recent advancements in the fusion of optical and SAR images for ship detection and recognition. Meanwhile, the fusion of remote sensing images and <i>in situ</i> data, such as automatic identification system (AIS) data, is briefly summarized, considering their significant benefits and technical trends. This paper highlights the immense potential of integrating these two sensor modalities to overcome individual challenges and enhance overall system performance. One of the primary challenges addressed in this work pertains to accurately matching ship targets between optical and SAR images when ships change positions. However, recently developed multi-modal fusion networks and transfer learning techniques have the potential to tackle the challenges, even with unmatched optical and SAR image pairs. In addition to summarizing existing techniques, the paper delves into the future perspective of fusion-based ship detection and recognition. In conclusion, this survey highlights the current state of optical and SAR image fusion techniques, pinpoints existing challenges, and outlines future research prospects, emphasizing the significance of fusion-based approaches in shaping the future of maritime applications.