Wild Mammal Behavior Recognition Based on Gated Transformer Network

Automatically recognizing animal behaviors in zoos and in national natural reserves can provide valuable insight into their welfare for facilitating scientific decision-making processes in animal management. This paper proposes a wild mammal behavior recognition model based on Gated Transformer Network. The model can respectively capture temporal and spatial information by two parallel Transformers, the channel-wise Transformer and the step-wise Trans-former. Thus, the hidden correlation between different channels of multivariate time series of wild mammal behavior classification can be exploited, meanwhile, the self-attention mechanism in the proposed network is used to model dependencies in sequences. we detect the animal contours in images as spatial features. The skeleton-based animal action recognition model is used to extract the joint coordinates during consecutive frames, then the fluctuate of the joint coordinates is used to distinguish the diversity of different behaviors of wild mammal in temporal space, which help to characterize the difference of joint point movement speed of different behaviors. In addition, we also compute leg joint angle for distinguishing the behaviors galloping and standing. Finally, the temporal features and spatial features are fused into the Gated Transformer Network for action recognition of wild mammal. The experiments show that the proposed model can effectively recognize four representational behaviors of animals: galloping, sitting, ambling, and standing. The average accuracy of the proposed scheme for recognizing behavior of wild mammal achieve 96.8%.