Automated whistle extraction for precise scaled annotations

Whistles produced by odontocete species can be used as indicators of species, density or individual identification, as well as for communication studies. These vocalisations in spectrogram representations vary in a wide range of time-frequency shapes. Their annotation is a challenge that is often time-consuming and labour-intensive. Existing automated contour extraction solutions are improving, but still struggle to be accurate in the context of overlapping vocalisations, which is common when studying groups of free-ranging small cetaceans. To address this problem, we developed a 2-step method in Python for detecting and extracting whistles within an audio recording. This method requires a relatively small number of manual annotations (< 2500) and was tested on recordings of free-ranging common dolphins (Delphinus delphis) in the Northwest Atlantic. For whistle detection, we selected the YOLOv8 model (a popular state-of-the-art object detection model) to predict bounding boxes around whistles in spectrograms. YOLOv8 is designed to detect complex objects in landscape images. It therefore performs well on simple spectrogram images, even in noisy situations. YOLOv8 has been trained to detect and classify between two categories: isolated whistles and overlapping whistles. Bounding boxes containing overlaps are given to the user for manual contour extraction using a custom-made annotation tool. Bounding boxes containing isolated whistles are fed into a deep learning regression model that uses the isolated image of each whistle to predict the contours. The performance of the regression depends heavily on the quality of the manual annotations, but it can generalise from them to predict any whistle shape. Overall, this method achieves a satisfactory compromise between annotation speed and prediction accuracy: simple whistles are extracted automatically and only the most complex annotations (i.e. overlapping whistles) are handled by the user.