SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation
Changhong Fu, Liangliang Yao, Haobo Zuo, Guangze Zheng, Jia Pan
Abstract
Domain adaptation (DA)-based training has demonstrated significant promise for obtaining real-time nighttime unmanned aerial vehicle (UAV) trackers. However, the state-of-the-art (SOTA) DA-based training still lacks the potential object with accurate pixel-level location and boundary to generate the high-quality target domain training sample. This key issue constrains the transfer learning of the real-time daytime SOTA trackers for challenging nighttime UAV tracking. Recently, the notable Segment Anything Model (SAM) has achieved a remarkable zero-shot generalization ability to discover abundant potential objects due to its huge data-driven training approach. To solve the aforementioned issue, this work proposes a novel SAM-powered DA training framework for real-time nighttime UAV tracking tasks, i.e., SAM-DA. Specifically, an innovative SAM-powered target domain training sample swelling is designed to determine enormous high-quality target domain training samples from every single raw nighttime image. This novel one-to-many generation significantly expands the high-quality target domain training sample for DA-based training. Comprehensive experiments of the tracker trained by DA, i.e., SAM-DA-Track, on extensive nighttime UAV tracking videos prove the robustness and domain adaptability of SAMDA for nighttime UAV tracking tasks. Especially, compared to the SOTA DA-based training, SAM-DA can achieve better performance with fewer raw nighttime images, i.e., the fewer-better training. This economized training approach facilitates the quick validation and deployment of algorithms for UAVs. The code is available at https://github.com/vision4robotics/SAM-DA.