A Machine Learning Approach to Predicting Personalized Head Related Transfer Functions and Headphone Equalization from Video Capture Data

In the realm of extended reality (XR) applications, personalized Head-Related Transfer Functions (p-HRTFs) have emerged as a critical element for achieving exceptional spatial audio quality. Nevertheless, the process of acquiring personalized HRTFs is intricate and time-intensive. This paper introduces an innovative technique that predicts personalized HRTFs using 2D images or video captures. The proposed method encompasses several key components, including 3D ear reconstruction from 2D images or video, followed by HRTF estimation through Boundary Element Methods or HRTF prediction using Neural Networks. Furthermore, a novel approach for estimating Personalized Headphone Equalization (p-HPEQ) curves, leveraging optical data of the ear, is presented. This approach enhances the accessibility and convenience of personalized spatial audio, leading to tailored listening experiences through customized headphone audio. Objective and subjective experiments are conducted to validate the accuracy of both p-HRTFs and p-HPEQs.