DFTNet: Deep Fish Tracker With Attention Mechanism in Unconstrained Marine Environments
Shilpi Gupta, Prerana Mukherjee, Santanu Chaudhury, Brejesh Lall, Hemanth Sanisetty
Abstract
Multiple fish tracking in an unconstrained marine videos is a highly challenging task. Trajectories of fishes convey critical information for the analysis of fish behaviour. In this paper, we have proposed DFTNet which incorporates Siamese network for encoding the appearance similarity and attention-long short term memory network to capture the motion similarity across subsequent frames. Finally, intersection over union matching score is computed to amalgamate spatial similarity cue in the final score. The proposed framework can provide joint optimization score to maintain the tracklet information encoding appearance, motion and spatial similarity cues. We perform exhaustive experiments and compare the proposed approach with competing techniques over Fish4knowledge videos and achieve significant average reduction in ID switches by 60.9%. The source code is made publicly available at: https://github.com/hemanth-s17/Deep-Fish-Tracker-Network.