Recognizing fish behavior in aquaculture with graph convolutional network

Analyzing fish shoal behaviors is one of the concerned problems for scientists who study fish welfare and stress. However, most shoal behavior exploring methods with manual parameters are subjective and not widely available in various conditions. Therefore, this study introduced graph technology, built 29,505 shoal behavioral graphs and presented a graph neural network for analyzing four shoal behaviors (normal, resting, abnormal, and circular state) by calculating the multiple swimming indexes and swimming posture from videos. In the proposed model, motion characteristics of the shoal and swimming posture of individuals in shoal were utilized to construct a shoal graph, and then the graph convolution network (GCN) model was trained and tested. Results indicated that the model could effectively improve the identification rate of fish shoals’ special behaviors, with an overall accuracy of 97.3% under the ideal condition, 92.3% for the practicable scheme that track fish by machine learning technology, compared with the artificial neural network, modified kinetic energy model and simulation feature point selection model, the accuracy of special behaviors increased by 1.6%, 57.7%, and 34.0%, respectively. Besides, the main factors that affected the accuracy of the analyzer were explored. The analyzer is sensitive to (1) the precision of tracking results, (2) edge connection in the graph and (3) features of the model’s input. In addition, by interpreting the principle of the GCN model, it assigns greater weights for dispersion in normal swimming state recognition, and swimming postures are the most significant indicators to determine whether a shoal is in an abnormal state or not. In summary, the model can be used to help researchers explore the basal behavioral mechanisms in aquaculture.