The Dawn of Synthetic Era: Synthesizing mmWave Radar Data from 2D Videos for Human Sensing

The lack of large-scale and rich millimeter wave (mmWave) radar datasets impedes progress in developing generalized human sensing models. To remedy this, researchers resort to designing a software pipeline that utilizes wealthy 2D videos to generate synthetic radar training data, but there is a noticeable lack of comprehensive reviews focusing on radar data generation from 2D videos. Addressing this gap, we first provide an extensive exploration of radar human sensing, detailing its importance, hardware and software aspects, principles, and data types. Second, we illustrate radar data generation architecture via modular strategy and propose two categories of generation techniques. Third, we collect a real-world radar dataset to evaluate existing radar data generation methods for both activity recognition and object detection. Finally, we conclude by identifying potential future research directions, highlighting the immense potential of this field for further exploration and innovation.