A New 3D Human Pose Estimation Network for Knee Posture Estimation

In lower extremity bone surgery, the surgeons need to adjust the patient's posture to expose the surgical site, which is usually done manually by the operator. However, manually adjusting is time-consuming and labor-intensive. An automatic positioning system (APS) is more preferable over the manual operation. The APS includes an electromechanic setup, a monocular camera and a computer. A monocular camera is used to detect the patient's lower limb postures, and the positioning device adjusts the patient's knee to the desired postures. This study employs a 3D human pose estimation (HPE) network, which contains a dual-attention module, to estimate the patient's knee joint angle and provide physicians with a measure of the patient's knee pose automatically from monocular images. First, a trained cascaded pyramid network (CPN) is used to obtain the 2D joint points of human images. Then, the detected two-dimensional coordinates, as the input for the proposed 3D HPE network, are defined in the three-dimensional space. The flexion and varus-valgus angles of knee joint are computed by establishing the knee coordinate system eventually. The 3D HPE network proposed in this study considers both spatial and temporal information to obtain the correlation between the spatial and temporal dimensions of the feature map. Subsequently, dilated convolution is used to extract features from the spatial and temporal attention feature maps, which realize a better estimation of the human body in 3D posture. The new model was tested on the Human3.6M dataset and showed that the mean per joint position error (MPJPE) is 48.6mm, and the errors of the flexion and varus-valgus angles are 7.9° and 12.7°, which are acceptable results for knee posture prediction compared to the manually placement of the patient postures.