DanceFormer: Hybrid transformer model for real-time dance pose estimation and feedback

Dance pose estimation holds significant value in teaching, training, and performance within the dance domain. However, challenges such as multi-target tracking in complex dynamic scenes, real-time demands, and computational resource constraints have posed difficulties for traditional methods. This paper introduces DanceFormer, a Transformer-based model for dance pose estimation that integrates the Vision Transformer (ViT), Time Series Transformer (TST), and an edge computing layer to achieve deep fusion of multimodal features, enhancing accuracy and real-time feedback capabilities. Experiments on the AIST and DanceTrack datasets show DanceFormer achieves pose estimation accuracy (MPJPE) of 18.4 mm and 20.1 mm, and multi-object tracking accuracy (MOTA) of 92.3% and 89.5%, outperforming other models. The edge computing design reduces average latency to 35.2ms, providing robust real-time processing suitable for low-resource edge devices. DanceFormer offers an efficient, precise, and real-time solution for complex dance scenarios, with broad application potential in dance education and real-time motion analysis. Future work will focus on optimizing the edge computing module and enhancing model generalization through data augmentation and dataset diversification.