A Real-Time and Efficient Optical Flow Tracking Accelerator on FPGA Platform
Yifan Gong, Jinshuo Zhang, Xin Liu, Jialin Li, Ying Lei, Zhe Zhang, Chen Yang, Li Geng
Abstract
Optical flow is a highly efficient visual tracking algorithm, which is commonly used to estimate pixel movement between two consecutive images in a video sequence. However, its high computational complexity and large number of computations become a bottleneck that hinders the performance of embedded vision systems. When applied to simultaneous localization and mapping (SLAM), it is necessary to consider not only time consumption, but also the overall accuracy of the system, causing even greater difficulties. In this paper, a real-time multi-scale Lucas Kanade (LK) optical flow hardware accelerator with parallel pipeline architecture is proposed. The designed circuit meets the high precision and real-time performance required by SLAM while fully considering the limitations of hardware resources. It is deployed on Xilinx Zynq SoC and achieves a frame rate of 93 fps for feature tracking of continuous frame images at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$752\times 480$ </tex-math></inline-formula> resolution. Compared with the implementation on ARM CPU, the average speed is increased by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.5\times $ </tex-math></inline-formula> . Finally, the feasibility and applicability of the hardware accelerator system designed in this paper are verified on the SLAM system. Experimental results on a public dataset show that the average Root Mean Square Error (RMSE) of this work is 0.189 m, indicating that the hardware accelerator has comparable precision with existing state-of-the-art software algorithms, achieving a great balance of performance and precision.