Computer aided pain detection and intensity estimation using compact CNN based fusion network

Abstract Pain is an indication of uneasiness, and its assessment is crucial for medical diagnosis and treatment of patients. Subjective methods like visual inspection and self-report of pain, are prone to human errors. Hence, objective methods i.e. behavioural, physiological are being focused on by researchers in the recent past. In this paper, a behavioural indicator i.e. facial expression-based fully automated pain assessment system is proposed. This system includes a novel fusion structure of Convolutional Networks for assessing various pain intensities from raw facial images. Here, we have proposed joint learning of robust pain-related facial expression features from fused RGB appearance and shape-based latent representation. We have suggested that the joint learning from both feature representations results in a more robust pain assessment model as compared to learning from either of the representation independently. Given this, the proposed system used two shallower networks, i.e., Spatial Appearance Network and Shape Descriptor Network. The proposed model has been evaluated extensively on the UNBC-McMaster dataset for both pain classification and intensity estimation. For pain level classification the proposed technique achieved 0.94 of F1-score. Moreover, for pain intensity estimation the proposed model has achieved a mean absolute error of 0.22 and accuracy of 0.92.