Facial Emotion Recognition from Feature Loss Media: Human Versus Machine Learning Algorithms

The automatic identification of human emotion, from low-resolution web or CCTV cameras is important for remote monitoring, interactive software, pro-active marketing, and dynamic customer experience management. Even though facial identification and emotion classification are both active fields of research, no studies, to the best of our knowledge, have attempted to compare the performance of humans and Machine Learning Algorithms (MLAs) when classifying facial emotions from media suffering from stepped feature information loss. To explore information loss in a controlled manner, in this study, we used singular value decomposition to systematically reduce the number of features contained within facial emotion images. Human participants were then asked to identify the facial emotion contained within the onscreen images; where image granularity was varied in a stepwise manner (from low to high). By clicking a button, participants added feature vectors until they were confident that they could categorise the presented emotion. The results of the human performance trials were compared against those of a Convolutional Neural Network (CNN), which classified facial emotions from the same media images. Findings showed that human participants were able to cope with significantly greater levels of information loss; achieving 85% accuracy with only three singular image vectors. Humans were also faster at classifying happy faces. CNNs are as accurate as humans when given mid and high resolution images; with 80% accuracy at twelve singular image vectors or above. The authors believe that this comparison concerning the differences and limitations of human and MLAs is critical to i) the effective use of CNN with lower-resolution video, and ii) the development of useable facial recognition heuristics.