Human activity recognition using wearable sensors by heterogeneous convolutional neural networks

Recent researches on sensor based human activity recognition (HAR) are mostly devoted to designing various network architectures to enhance their feature representation capacity for raw sensor data. In this paper, we focus on strengthening the vanilla convolution without adjusting the model architectures in HAR scenario. Inspired by the idea of grouped convolution, we propose a novel heterogeneous convolution for activity recognition task, where all filters within a specific convolutional layer are separated into two uneven groups. Specifically, the sensor input is down-sampled into a low-dimensional embedding, which is then convolved by one filter group to recalibrate normal filters within the other group. The two filter groups can complement each other, which is very beneficial for augmenting the receptive field of sensor signals for HAR task. Extensive experiments are conducted on several benchmark HAR datasets, which consists of OPPORTUNITY, PAMAP2, UCI-HAR, USC-HAD as well as the Weakly Labeled HAR dataset. The results show that the baseline models can be significantly improved. Our heterogeneous convolution is simple and can easily be integrated into standard convolutional layers without increasing extra parameters and computational overhead. Finally, the actual operation of heterogeneous convolution is evaluated on an embedded Raspberry Pi platform.