Vision-based dual network using spatial-temporal geometric features for effective resolution of fish behavior recognition with fish overlap

In this study, a novel visualization framework for fish behavior recognition based on Slowfast networks and spatial-temporal graph convolutional networks (ST-GCN) is proposed. The framework can directly recognize fish behaviors in continuous videos and classify fish states in cases of severe fish stacking. A self-constructed fish behavior dataset containing 10 single fish HD videos and 300 fish schooling video clips covering three action categories and two state categories was collected. The evaluation was performed on this behavioral dataset. The results show that the framework achieves accuracies of 95.00% and 88.61% for state recognition and action recognition, respectively, exceeding those of several benchmark methods. Robustness and generalization experiments, as well as fish feeding experiments, were also conducted to demonstrate the potential application of the framework for guiding smart feeding in real production activities. The framework provides a novel solution for fish behavior analysis in the visual domain and can be extended to other aquatic animals or scenarios.