Integrating and assessing machine learning acoustic target classification models for fish survey estimations

Abstract Scientific acoustic-trawl surveys collect data that are used to track fish and zooplankton populations over time. Most rely on manual annotation during acoustic target classification, but automated methods have been proposed. Here, we report on a framework for testing deep learning-based acoustic classification models and integrating them into the survey estimation process. The approach was applied to North Sea lesser sandeel (Ammodytes marinus) surveys from 2009 to 2024. Three U-Net-based models were tested: a baseline model, a depth-aware model, and a model trained with similarity-based sampling for the foreground class. A threshold based on the training years was applied to the models’ SoftMax outputs. The official sandeel estimation process was used as a starting point, replacing input data with model predictions. The biomass estimates were generally similar between manual annotations and model-based estimates, but variation existed across years. The baseline model misclassified a surface layer as sandeel and was prone to bottom contamination, causing larger deviations from official estimates. Discrepancies between the similarity-based model and the official estimates resulted from an incorrectly applied SoftMax threshold, leading to missing school interiors and indicating threshold sensitivity. Unlike traditional F1 score evaluations commonly used in image-based classification, our comparison assessed predictions in a survey-relevant context. The evaluation indicated that full automation was not yet feasible, but the predictions could be used as starting points for manual scrutiny. Annotating a subset of the data to refine thresholds or employing more advanced active learning approaches could enhance efficiency. These methods could enable faster, more consistent survey annotation.