The Application of Capsule Neural Network Based CNN for Speech Emotion Recognition

Speech emotion recognition (SER) is an important and challenging task. It requires that a machine learning model process a person's speech signals and to judge his emotional state accurately. Due to the high dimensionality of the audio data, the extracted features are always noisy. Besides, the abstraction of audio features makes it impossible to fully use the inherent relationship among audio features. This paper proposes a model that combines a convolutional neural network (CNN) and a capsule network (CapsNet), named as CapCNN. The advantage of CapCNN lies in that it provides a solution for time sensitivity, and gives the overall characteristics. In this study, it is found that CapCNN can well handle the SER task. Compared with other state-of-the-art methods, our algorithm shows high performances on the CASIA and EMODB datasets. The detailed analysis confirms that our method provides balanced results on various classes.