Automatic Contact-Less Monitoring of Breathing Rate and Heart Rate Utilizing the Fusion of mmWave Radar and Camera Steering System

The demand for noncontact breathing and heart rate measurement is increasing. In addition, because of the high demand for medical services and the scarcity of on-site personnel, the measurement process must be automated in unsupervised conditions with high reliability and accuracy. In this article, we propose a novel automated process for measuring breathing rate and heart rate with mmWave radar and classifying these two vital signs with machine learning. A frequency-modulated continuous-wave (FMCW) mmWave radar is integrated with a pan, tilt, and zoom (PTZ) camera to automate camera steering and direct the radar toward the person facing the camera. The obtained signals are then fed into a deep convolutional neural network to classify them into breathing and heart signals that are individually low, normal, and high in combination, yielding six classes. This classification can be used in medical diagnostics by medical personnel. The average classification accuracy obtained is 87% with precision, recall, and an F1 score of 0.93.