mm-Pose: Real-Time Human Skeletal Posture Estimation Using mmWave Radars and CNNs

In this paper, mm-Pose, a novel approach to detect and track human skeletons\nin real-time using an mmWave radar, is proposed. To the best of the authors'\nknowledge, this is the first method to detect >15 distinct skeletal joints\nusing mmWave radar reflection signals. The proposed method would find several\napplications in traffic monitoring systems, autonomous vehicles, patient\nmonitoring systems and defense forces to detect and track human skeleton for\neffective and preventive decision making in real-time. The use of radar makes\nthe system operationally robust to scene lighting and adverse weather\nconditions. The reflected radar point cloud in range, azimuth and elevation are\nfirst resolved and projected in Range-Azimuth and Range-Elevation planes. A\nnovel low-size high-resolution radar-to-image representation is also presented,\nthat overcomes the sparsity in traditional point cloud data and offers\nsignificant reduction in the subsequent machine learning architecture. The RGB\nchannels were assigned with the normalized values of range, elevation/azimuth\nand the power level of the reflection signals for each of the points. A forked\nCNN architecture was used to predict the real-world position of the skeletal\njoints in 3-D space, using the radar-to-image representation. The proposed\nmethod was tested for a single human scenario for four primary motions, (i)\nWalking, (ii) Swinging left arm, (iii) Swinging right arm, and (iv) Swinging\nboth arms to validate accurate predictions for motion in range, azimuth and\nelevation. The detailed methodology, implementation, challenges, and validation\nresults are presented.\n