Depth estimation from monocular endoscopy using simulation and image transfer approach

Abstract

Obtaining accurate distance or depth information in endoscopy is crucial for the effective utilization of navigation systems. However, due to space constraints, incorporating depth cameras into endoscopic systems is often impractical. Our goal is to estimate depth images directly from endoscopic images using deep learning. This study presents a three-step methodology for training a depth-estimation network model. Initially, simulated endoscopy images and corresponding depth maps are generated using Unity based on a colon surface model obtained from segmented computed tomography colonography data. Subsequently, a cycle generative adversarial network model is employed to enhance the realism of the simulated endoscopy images. Finally, a deep learning model is trained using the synthesized endoscopy images and depth maps to estimate depths accurately. The performance of the proposed approach is evaluated and compared against prior studies utilizing unsupervised training methods. The results demonstrate the superior precision of the proposed technique in estimating depth images within endoscopy. The proposed depth estimation method holds promise for advancing the field by enabling enhanced navigation, improved lesion marking capabilities, and ultimately leading to better clinical outcomes. • A novel three-step approach was proposed for training depth estimation networks in endoscopy. • The depth-estimation network trained with this approach outperformed other supervised learning and unsupervised learning methods. • The sim-to-real approach outperformed the real-to-sim approach in training depth estimation networks. • The sim-to-real approach allowed direct application of real endoscopy images to the depth estimation network.

Topics & Concepts