CMFTNet: Multiple fish tracking based on counterpoised JointNet

The analysis of fish motion is remarkably applied to investigate physiological behavior and water quality status. Multiple fish tracking methods based on computer vision have the advantages of contactless, information interpretability, single equipment, and high durability. However, the existed approaches cannot cope with complex scenarios, occlusions, and inconstant scales well. To solve the issues, we propose a multi-object video tracking model specifically for fish schools in aquaculture ponds, called CMFTNet. Firstly, we deploy the Joint Detection and Embedding paradigm to share the features for multiple fish detection and tracking tasks. It utilizes the anchor-free method to solve the problem of mutual occlusion of fish schools. Then, we embed the deformable convolution in the updated backbone to intensify the context features of fish in complex environments. Finally, we evaluate the influence of feature dimensions and propose a weight counterpoised loss that outperforms the previous aggregation methods on dual-branch. Extensive experiments show that CMFTNet achieves the best result both on precision and efficiency. The model reaches 65.5% MOTA and 27.4% IDF1 on the OptMFT dataset. The source codes and pre-trained models are available at: https://github.com/vranlee/CMFTNet.