Geometric distortion on video see‐through head‐mounted displays

Abstract Video see‐through (VST) head‐mounted displays (HMDs) reconstruct the physical world using outward‐facing cameras and depth sensors. One of the major challenges for VST HMDs is the geometric distortion of the physical pass‐through content. In this study, we present the experimental setup and metrology for VST geometric distortion measurement on single‐camera and dual‐camera VST HMDs. The VST geometric distortion is compared to the digital distortion by the HMD. Finally, we investigate the impacts of camera misalignment and depth distance on VST geometric distortion without and with angular‐based distortion correction. The results demonstrate that camera misalignment with respect to the eye position is the primary root cause for the distortion. Dual‐camera setup is necessary to minimize the VST geometric distortion by placing the cameras close to the eye positions. Although distortion correction can partially mitigate VST geometric distortion in a 2D plane, it remains a challenge for VST distortion correction at multiple depth distance in the 3D space. The method and findings presented of this study may provide guidelines for evaluation of spatial accuracy on VST HMDs and hardware design.