A Tightness Detection Method for Railway Fasteners Based on RGB-P Bimodal Data
Xiaocui Yuan, Zhiming Lei, Hongtao Zhu, Yongtao Wang, Miaomiao Zhang
Abstract
Rail fasteners are critical components in railways and are inspected periodically to ensure train safety. Visual inspections using machine technology are commonly employed to identify any abnormal fasteners in rail tracks. However, it can be challenging to determine the tightness of fasteners based on unimodal data. This is where multi-modal data fusion technology comes into play, offering a viable solution. In this paper, a novel method based on RGB-P bimodal data fusion is proposed to detect the tightness of fasteners. Firstly, a 3D laser sensor is employed to collect the profiles of the railway, and a bimodal data which consist of RGB depth images and point clouds (RGB-P for short) is constructed based on the profiles. Then, the clip of fastener is rapidly segmented and clip’s skeleton is extracted from the RGB depth image. The 2D skeleton is fused with the point cloud to quickly obtain the 3D skeleton of the clip. Lastly, the features of the 3D clip skeleton are extracted to calculate the clip gap, and the tightness of fastener is detect based on the clip gap. Experiments are conducted on ballast and ballast-less railways, including WJ-7, WJ-8, and WJ-2 types of fasteners, to evaluate performance, robustness and precision of the fastener tightness detection. The experimental results demonstrate that our proposed method can accurately measure the clip gap with the measurement error and root mean square error (RMSE) both less than 0.1mm, and is robustness to the railway environment. When the measurement error is within ±0.1mm, the overall detection rate of fastener tightness detection exceeds 99.4%, and the detection speed is not less than 10.05m/s. The detection system can be installed on rail inspection vehicles to detect the tightness of fasteners in real-time and improve efficiency of the railway maintenance.