Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation
Xue Hu, Ferdinando Rodriguez y Baena, Fabrizio Cutolo
Abstract
Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 2.55 arcmin for VST and 7.72 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.04 mm, 5.96 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 2.22<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> for the VST system, and 4.36 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.80 mm, 5.65 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.42<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> for the OST system.