Quantifying Floating Litter Fluxes with a Semi-Supervised Learning-Based Framework

Supervised deep learning methods have been widely employed by researchers and practitioners to detect floating macroplastic litter (plastic items >5 mm) in (fresh)water bodies. However, their potential to quantify litter fluxes in rivers with wide cross-sections remains underexplored. Additionally, supervised learning (SL) models also face practical challenges, including the dependency on extensive labeled data, and low detection performance for small litter items.To overcome these issues, we propose a semi-supervised learning (SSL)-based framework for quantifying cross-sectional floating litter fluxes. This framework includes four steps: (1) developing a robust litter detection model using SSL methods, (2) collecting images of river surfaces from multiple locations along the target river cross-section using cameras, (3) applying the developed model to detect and count litter items in images, and (4) post-processing the detection results to quantify cross-sectional litter fluxes. In the first step, we first pre-trained a Residual Network with 50 layers (ResNet50) on a large amount of unlabeled data (≈500k images) using a self-supervised learning method, Swapping Assignments between multiple Views of the same image (SwAV). Then, we fine-tuned a Faster Region-based Convolutional Neural Network (Faster R-CNN) with the ResNet50 backbone on a limited amount of labeled data (1.1k images with 1.3k annotated litter items). We introduced a Slicing Aided Hyper Inference (SAHI) method to enhance accuracy of Faster R-CNN in detecting small litter.We evaluated the in-domain detection performance of SSL models using images from canals and waterways of the Netherlands, Indonesia and Vietnam. Additionally, we assessed the zero-shot out-of-domain detection performance of SSL models, and litter flux quantification performance of the proposed framework on a case study in the Saigon river in Vietnam (including the Thu Thiem and Binh Loi locations). The assessment of out-of-domain detection performance was conducted with and without SAHI method. We benchmarked our results against the SL methods using the same Faster R-CNN architecture with ImageNet pre-trained weights. The results show that the SSL models significantly outperform baseline benchmarks, with an in-domain F1-score increase of 0.2, and a zero-shot out-of-domain median F1-score increase of 0.14 for Thu Thiem and 0.07 for Binh Loi. The SSL-based framework quantifies litter fluxes nearly twice as high as the baseline SL-based framework, offering estimates that align more closely with human-measured litter fluxes. Furthermore, the SAHI method correctly identifies 54 additional small litter items (with areas below 1,000 cm²) in the case study, compared to the results obtained without the SAHI method.Our findings underscore a promising pathway for developing a robust framework for macroplastic flux measurement by integrating a foundation model, a transformative approach driving the current artificial intelligence revolution across diverse domains. By scaling our proposed framework with larger and more diversified datasets, we can make significant progress in developing advanced monitoring systems to tackle the global challenge of plastic pollution.