Hierarchical Feature Pooling Transformer for Efficient UAV Object Tracking
Haijun Wang, Wenlai Ma, Shengyan Zhang, Wei Hao
Abstract
Recently, owing to the long-range feature dependencies, transformers have achieved considerable progress in the field of visual tracking. One of the most challenging problems in transformer-based tracking is that the large feature length of tokens leads to high computational cost. Meanwhile, only the single feature map from the last layer of convolutional neural networks (CNN) is employed as the input of transformer, which reduces the tracking accuracy and robustness in complex scenarios. Thus, in this letter, we present an efficient and effective hierarchical feature pooling transformer (HFPT) for UAV object tracking, which is able to inherit the merits from both CNN and transformer architectures. Firstly, we introduce a single pooling operation into multi-head self-attention (MHSA) in transformer to build a new backbone network for reducing the concatenated feature length and capturing rich contextual information. Secondly, hierarchical feature maps generated by multi-level convolutional layers are fed into the pooling transformer to learn interdependencies between high-resolution features and low-resolution features. Thirdly, a feature correction layer is designed to enrich the encoded detailed information for handling the small targets. Finally, we evaluate our proposed method in three well-known UAV benchmarks such as DTB70, UAV20L and UAV123@10fps. Numerous experimental results demonstrate that our HFPT method is able to achieve better performance than the current top-performing trackers with an average speed of 30.3fps on the edge platform of Nvidia Jetson AGX Orin.