Virtual Resection Specimen Interaction Using Augmented Reality Holograms to Guide Margin Communication and Flap Sizing
Fabian N. Necker, Marcello Chang, Christoph Leuze, Michael C. Topf, Bruce L. Daniel, Fred M. Baik
Abstract
Head and neck surgeons often have difficulty in relocating sites of positive margins due to the complex 3-dimensional (3D) anatomy of the head and neck. We introduce a new technique where resection specimens are 3D scanned with a smartphone, annotated in computer-assisted design software, and immediately visualized on augmented reality (AR) glasses. The 3D virtual specimen can be accurately superimposed onto surgical sites for orientation and sizing applications. During an operative workshop, a surgeon using AR glasses projected virtual, annotated specimen models back into the resection bed onto a cadaver within approximately 10 minutes. Colored annotations can correspond with pathologic annotations and guide the orientation of the virtual 3D specimen. The model was also overlayed onto a flap harvest site to aid in reconstructive planning. We present a new technique allowing interactive, sterile inspection of tissue specimens in AR that could facilitate communication among surgeons and pathologists and assist with reconstructive surgery.