TFST: Two-Frame Ship Tracking for SAR Using YOLOv12 and Feature-Based Matching
Muhammad Yasir, S. Liu, Mingming Xu, Fernando J. Aguilar, Jianhua Wan, Shiqing Wei, Saied Pirasteh, Hong Fan, Qamar Ul Islam
Abstract
Tracking objects in SAR imagery is critical for maritime surveillance, traffic monitoring, and security applications, but remains a major challenge due to speckle noise, sea clutter, and limited temporal continuity. Most existing tracking-by-detection methods process frames independently, often resulting in weak associations and frequent identity switches. To overcome these limitations, we propose TFST, a two-frame SAR ship tracking framework that integrates detection, feature encoding, and optimal assignment. In this way, the goal of this work is to address the current gaps in SAR ship tracking by strengthening cross-frame partnerships and reducing identity switches through an integrated two-frame tracking framework. In our approach, a deep detector first processes consecutive frames to generate candidate bounding boxes. A lightweight feature extractor encodes both appearance and structural cues, while a matching module constructs a cost matrix that combines feature similarity and positional consistency. Gating is applied to remove infeasible associations, and the Hungarian algorithm is employed to achieve a globally optimal assignment. Quantitative evaluations performed on three widely known and publicly available SAR-Ship datasets (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SSTD, SSDD</i>, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SAR-Ship</i>) further highlight the advantages of TFST. In terms of ship detection performance, TFST achieved an average <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mAP@50</i> improvement of 2.2% over the YOLOv12 baseline model on all three tested datasets. Regarding tracking results, the superiority of TFST over state-of-the-art multi-object trackers becomes even more evident. In fact, the proposed model achieved the highest MOTA accuracy (86.9%) and the best <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IDF1</i> score (82.7%), thus outperforming strong baselines such as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Siam-SORT</i> (82.1% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MOTA</i>, 79.8% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IDF1</i>) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TrackFormer</i> (80.7% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MOTA</i>, 78.7% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IDF1</i>). In conclusion, TFST demonstrated improved robustness, fewer <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ID</i> switches, and higher tracking accuracy compared to baseline methods, underscoring its effectiveness in complex maritime environments.