Learning Semantic Plasticity Relationship via Hierarchical Correlational Modeling for Robust Head Pose Estimation

Head pose estimation (HPE) has attracted remarkable attention in the field of human and robot interaction. However, practical head pose methods are hindered by challenges, such as occlusion, extreme angle, and complex light. To overcome these challenges, we propose a novel plasticity relationship learning framework (PlaHPE) for robust head pose estimation via hierarchical correlational modeling. The proposed model leverages two important insights: facial features across different levels and rotation-induced head plasticity relationship. On the basis of these insights, we design a semantic plasticity generation model to describe facial plasticity features at multiple granularities. Moreover, we construct a novel framework with two branches to fuse plasticity features, and this is achieved by designing two modules, namely, the plasticity relationship mining (PRM) module and the plasticity feature interaction (PFI) module. In PRM, text and visual features are aligned to enable the model to learn high-level semantic knowledge. The PFI module aggregates multigranularity features for final HPE. Extensive experiments conducted on the AFLW2000 and BIWI benchmark datasets show that PlaHPE consistently outperforms existing state-of-the-art methods, substantially improving performance while maintaining high efficiency.