Compound Facial Emotion Recognition based on Facial Action Coding System and SHAP Values

Human facial emotion recognition is a difficult task in computer-human interaction. Facial emotion recognition is required in many applications like medical, security, video games, e-physiotherapy, and counselling. Literature hasmany studies that have focused only on 6 basic emotions but advanced studiessuggest human emotions are not limited to these 6 basic emotions. A humanface can exhibit many other emotions, which are generated by combining thetwo basic emotions, these derived emotions are known as compound emotions.Recognition of compound emotions is also a very important task; hence thisstudy proposes the use of the Facial Action Coding System (FACS) to identify12 compound emotions. The authors identified and derived the intensities of17 AUs with Openface library. Finally, two machine learning classifiers SVM(Support Vector Machine) and KNN (K-nearest neighbour) were implementedto identify 12 compound emotions, and results were compared. The experimental results show that the SVM classifier outperformed with an emotion recognition rate of 98.31% while the recognition rate of K-NN was 93.66%. Theauthors also implemented SHAP values to observe the AUs association witheach compound emotion.