Fused deep learning based Facial Expression Recognition of students in online learning mode

Summary In this research work, Facial Expression Recognition (FER) is used in the analysis of facial expressions during the online learning sessions in the prevailing pandemic situation. An integrated geometric and appearance feature extraction is presented for the FER of the students participating in the online classes. The integrated features provided a low‐dimensional significant feature area for better facial data representation. Feasible Weighted Squirrel Search Optimization (FW‐SSO) algorithm is applied for selecting the optimal features due to its efficient exploration of the search space and enhancement of the dynamic search. The output of the FW‐SSO algorithm is used for tuning the autoencoder. Autoencoder is used for combining the G&A features, for feature optimization process. Classification is done by using Long Short‐Term Memory (LSTM) network with Attention Mechanism (ALSTM), as it is highly efficient in capturing the long‐term dependency of the facial landmarks in the image/video sequences. The proposed fused deep learning method focuses on the fusion of the G&A features for high discrimination. Experimental analysis using FER‐2013 and LIRIS datasets demonstrated that the proposed method achieved maximum accuracy of 85.96% than the existing architectures and maximum accuracy of 88.24% than the VGGNet‐CNN architecture.