A Feasible Heartbeat Rate Monitoring Model From Facial Videos Using Weighted Feature Fusion‐Based Adaptive Long Short‐Term Memory With Attention Mechanism

ABSTRACT Clinicians can determine cardiac symptoms with the aid of heart rate detection and continuous surveillance. One of the necessary indicators utilized to assess the physiological wellness of the human body is heartbeat rate. Conventional heartbeat rate monitoring systems necessitate skin contact; nevertheless, Remote Photoplethysmography (RPPG) allows for contactless heartbeat rate measurement by using a video camera that records minute changes in skin tone. The medical field has increased interest in the surveillance of physiological signals due to technological developments in future healthcare. In this paper, a novel heartbeat rate monitoring system is developed to significantly track heart functioning. Initially, the facial videos are gathered from the online resources, and they are given to the face detection phase for generating detected face images with the help of Vision Transformer‐based You Only Look Once v5 (ViT‐Yolov5). Further, the detected face images are given to the feature extraction process. Here, the 3D Residual Attention Network (3D‐RAN) is used to retrieve the features to generate the feature set 1. Similarly, Red, Green, and Blue (RGB) features and Deep Belief Network (DBN)‐based features are retrieved from the input video, which is considered as feature set 2. Further, the heartbeat monitoring is done using the Weighted feature fusion‐based Adaptive Long Short‐Term Memory with Attention Mechanism (WALSTM‐AM), and the weighted feature fusion is carried out on the feature sets 1 and 2, in which the weight and parameters are optimized using Enhanced Shell Game Optimization (ESGO). The numerical results showed that the developed model attained a Normalized Mean Square Error Value (NMSE) of 0.001868 that indicates better and more reliable validation of heartbeat rate, leading to higher representation of heart's activity. Thus, the proposed heartbeat monitoring mechanism attains better outcomes than the conventional methods to prove its optimal performance.