Eye-in-Hand Visual Servoing Enhanced With Sparse Strain Measurement for Soft Continuum Robots
Xiaomei Wang, Ge Fang, Kui Wang, Xiaochen Xie, Kit-Hang Lee, Justin Ho, Wai Lun Tang, James Lam, Ka‐Wai Kwok
Abstract
In the feature/object tracking of eye-in-hand visual servoing, 2D motion estimation relying only on image plane feedback is easily affected by vision occlusion, blurring, or poor lighting. For the commonly-used template matching method, tracking performance greatly depends on the image quality. Fiber Bragg gratings (FBGs), a type of high-frequency flexible strain sensor, can be used as an assistant device for soft robot control. We propose a method to enhance motion estimation in soft robotic visual servoing by fusing the results from template matching and FBG wavelength shifts to achieve more accurate tracking in applications such as minimally invasive surgery. Path following performance is validated in a simulated laparoscopic scene and LEGO-constructed scene, demonstrating significant improvement to feature tracking and robot motion, even under external forces.