A Privacy-Preserving AIoT Framework for Fall Detection and Classification Using Hierarchical Learning With Multilevel Feature Fusion

Privacy and false fall detection pose a significant challenge within the current camera-based human activity monitoring research. In response, we propose a solution that leverages the inherent relationship between binary fall detection and activity classification through hierarchical learning for low false fall classification. Our solution involves employing edge computing for data preprocessing, with a specific focus on extracting key points of the human body and motion features. Subsequently, the classification process based on multi-stream hierarchical learning takes place at the cloud level. This approach prevents sending raw RGB videos to the cloud to improve data privacy. Moreover, our network incorporates the hierarchical relationship between binary and multi-class classification using stream-to-stream skip connections and multi-level feature fusion. We created our Multiscale Convolutional Fusion Block (MSCFB) for multi-level feature extraction and fusion with an inner block skip connection. Additionally, we added an auxiliary fall detection loss to the first stream to regulate and control the feature extraction process for fall detection. Our network attained state-of-the-art performance in activity classification on the UP-Fall dataset, securing an F1-Score of 97.27%. Notably, there were no misclassifications between regular activities and fall sub-classes, leading to a perfect F1-Score of 100% for binary fall detection on this dataset. Furthermore, our fall detection network achieved state-of-the-art on the PRECIS HAR dataset with an F1-Score of 98.34%.