Multimodal Thermal and Depth Imaging for Back Acupoint Localization in Massage Robots

Accurate localization of human acupotints remains a fundamental challenge in the development of massage robot systems. While deep learning algorithms leveraging the classical Transformer architecture have improved recognition accuracy, a notable gap still exists between current performance and the robustness needed to handle complex lighting conditions and individual variability. To fill this gap, this paper focuses on the scenarios involving low-light conditions and subjects wearing close-fitting clothing. To address this issue, the paper proposes an acupoint localization algorithm tailored to the scenario by employing multimodal data fusion techniques that intergrate thermal imaging with depth map information. The algorithm is based on YOLOv8 and has been architecturally improved, referred to as YOLOv8-TCC. Specifically, a dual-branch parallel backbone network architecture is designed, incorporating the Channel Prior Convolutional Attention (CPCA) module to enhance multimodal feature representation. Next, a Cross-Modal Feature Fusion (CTF) module is introduced to facilitate information integration, while a dynamic template alignment approach is proposed to accommodate variations in body shape and posture. Experimental results show that the YOLOv8-TCC method achieves higher accuracy compared to the traditional single-modal model under varying lighting conditions. With the introduction of the CTF and CPCA modules, the average precision at Intersection over Union (AP@0.75) increased by 2.8% and 0.8%, respectively. Moreover, the success rate under the clinically meaningful 0.5 cun constraint (SR@0.5 cun) reached 95%. It is worth mentioning that our method remains effective when subjects wear light and close-fitting clothing. The proposed method exhibits strong generalizability and scalability, offering an accurate and efficient solution for practical applications.