Mixed Reality Intraoperative Navigation in Craniomaxillofacial Surgery

We report our early experience with the HoloLens 2 (Microsoft Corp., Redmond, Wash.) mixed reality device1 in the management of patients with craniomaxillofacial trauma. Preoperative planning is performed using iPlan Cranial Version 2.6 (Brainlab, Munich, Germany). The contralateral uninjured facial skeleton is mirrored for use as a template for reconstruction. This skull reference model is exported as a Standard Tessellation Language file. This is uploaded onto the HoloLens 2 system by means of the software Virtual Surgery Intelligence (apoQlar GmbH, Hamburg, Germany), which generates a skull reference hologram. Intraoperatively, superimposition of the skull reference hologram onto the patient is performed after fracture exposure (Fig. 1). Predetermined exposed bony landmarks on the patient were matched. These were visually verified on the frontal, lateral, and worm's-eye views. Having the hologram projected directly on the operative field allows surgeons to have a better appreciation of the vector and magnitude of movement required for fracture reduction. Surgeons can visualize structures beyond the area of exposure. Adequacy of fracture reduction can be validated immediately. Adjustments can be made and validations repeated. Fixation is performed once reduction is satisfactory.Fig. 1.: Alignment of skull reference hologram is performed using multiple exposed bony landmarks to improve accuracy.The main advantage of mixed reality is its ability to allow visualization and manipulation of superimposed holograms on patients in real time. The experience is akin to a live "in-flight" image-guided surgery. There is immediate feedback on the adequacy of reduction. Surgeons are able to directly visualize the skull reference hologram superimposed on the patient's skull through the lens, without changes in line of sight. Limitations include the arbitrary nature of aligning the skull reference hologram onto the patient. Mitsuno et al.2 described a three-point technique of alignment. A more precise technique is required in craniomaxillofacial surgery. Our described technique is adapted from the registration process with current navigation devices. Greater accuracy can be attained by aligning a larger number of landmarks directly on exposed bones that are to be operated on. Advances that we are developing include the following: Automated stereoscopic registration and alignment of skull reference hologram: This removes the subjectivity and time involved in the process of superimposing the hologram onto the patient. Automated and objective validation of fracture reduction: A system that automatically detects the extent of deviation of the reduced fracture fragments from the skull reference hologram. Adjustments and reduction can be made more accurately. Soft-tissue segmentation and visualization: Critical soft-tissue structures such as nerves can be superimposed alongside the skull reference model, visualized, and preserved intraoperatively. Instrument recognition and early warning system: Instruments can be registered onto the system and be recognized intraoperatively. When positioned in close proximity to critical structures such as the optic nerve, the system detects this and sends out an early warning signal such that inadvertent injury to these structures can be avoided. Mixed reality devices provide surgeons with real-time intraoperative guidance. It has the potential to decrease operative time and improve patient outcomes. DISCLOSURE The authors have no financial interest in any of the products or devices mentioned in this article.