Integrating Multi-Modal Data for Accurate Real-Time Pose Estimation

Posing human is a fundamental computer vision task with applications in robotics, healthcare, and human-computer interaction. This paper presents a new multi-modal fusion approach for real-time pose estimation from Part Affinity Fields (PAFs) and introduces novel solutions to key challenges in multi-person scenes. The main contributions include an optimized PAF-based architecture with reduced computational complexity while maintaining high accuracy in dynamic environments, a unified detector for body and foot keypoints that improves lower-body pose estimation using an enhanced dataset with over 16,000-foot annotations, and a CUDA implementation of the proposed architecture achieving real-time performance at an average of 33.65 FPS. Experimental results demonstrate 83% accuracy, surpassing all other state-of-the-art methods in both speed and precision. This work presents a robust online solution for multi-person pose estimation in complex environments.