Efficient obstacle detection based on prior estimation network and spatially constrained mixture model for unmanned surface vehicles

Abstract Recently, spatially constrained mixture model has become the mainstream method for the task of vision‐based obstacle detection in unmanned surface vehicles (USVs), and has shown its potential of modeling the semantic structure of the marine environment. However, the expectation maximization (EM) optimization of this model is quite sensitive to initial values and easily falls into a local optimal solution in the presence of significant rolling and pitching in rough seas. In addition, existing methods based on spatially constrained mixture model are susceptible to false positives in the presence of sun glitter. In this paper, a prior estimation network (PEN) is proposed to improve the mixture model, which together enable reliable monocular obstacle detection for USVs. We develop a weakly supervised E‐step to train the PEN for learning the semantic structure of marine images and estimating initial class priors in obstacle detection. To mitigate the influence of poor initial parameters on the convergence of EM optimization, we use the priors estimated by the PEN to calculate the initial parameters of the mixture model and automatically adjust the hyper priors on the semantic components in the mixture model. The output of the PEN is also applied to set the probability values of the outlier component in the mixture model, aiming to reduce false positives caused by sun glitter. Experimental results show that our approach outperforms the current state‐of‐the‐art monocular method by 15% improvement in sea edge estimation and a 3.3% increase in F‐score on the marine obstacle detection data set, as well as 69.5% improvement in sea edge estimation and a 39.2% increase in F‐score on our data set, while running over 40 fps.