Autonomous port traffic safety orientated vehicle kinematic information exploitation via port-like videos
Xinqiang Chen, Qianli Ma, Huafeng Wu, Wen‐Long Shang, Bing Han, Salvatore Antonio Biancardo
Abstract
Abstract This study proposes a framework for extracting automatic guided vehicle (AGV) kinematic information from port-like videos, which provides a solution for situation awareness of port surveillance videos. Firstly, vehicle pixel-wise positions in port-like videos are determined by the visual object tracking (SeqTrack) model. Secondly, the extrinsic parameters of the query images are estimated by the generalizable model-free 6-DoF object (Gen6D) pose estimation method. More specifically, a point cloud of AGV is reconstructed with multi-view AGV reference images and the image extrinsic parameters are obtained through structure form motion. The reference image which has the most similar viewpoint as the query image is identified with the Gen6D selection module; as a result, the extrinsic parameters of the reference image can be used to estimate the extrinsic parameters of the query image. After that, the extrinsic parameters of the query image are identified with the support of the Gen6D refinement module. Thirdly, we obtain vehicle displacement by mapping the vehicle point cloud coordinate into the camera coordinate, and then we estimate the vehicle movement information with the help of the generative adversarial network model. Experimental results suggest that the pose estimation metrics average discrepancy distance and 2D re-projection error of our method reach 0.76 and 0.75, respectively. The mean absolute error and root mean squared error of the estimated vehicle displacement reach 0.023, 0.030 for scene #1 (i.e., AGV moves along x-axis of camera coordinate) and 0.182, 0.298 for scene #2 (i.e., camera follows AGV to move along x-axis of camera coordinate).