Deep learning for autonomous ship-oriented small ship detection

• A deep learning method is proposed for autonomous ship-oriented small ship detection. • A modified Generative Adversarial Network is applied for training data augmentation. • An improved YOLO v2 algorithm is used for small ship detection. • Extensive experiments are conducted to show the effectiveness of the proposed method. Small ship detection is an important topic in autonomous ship technology and plays an essential role in shipping safety. Since traditional object detection techniques based on the shipborne radar are not qualified for the task of near and small ship detection, deep learning-based image recognition methods based on video surveillance systems can be naturally utilized on autonomous vessels to effectively detect near and small ships. However, a limited number of real-world samples of small ships may fail to train a learning method that can accurately detect small ships in most cases. To address this, a novel hybrid deep learning method that combines a modified Generative Adversarial Network (GAN) and a Convolutional Neural Network (CNN)-based detection approach is proposed for small ship detection. Specifically, a Gaussian Mixture Wasserstein GAN with Gradient Penalty is utilized to first directly generate sufficient informative artificial samples of small ships based on the zero-sum game between a generator and a discriminator, and then an improved CNN-based real-time detection method is trained on both the original and the generated data for accurate small ship detection. Experimental results show that the proposed deep learning method (a) is competent to generate sufficient informative small ship samples and (b) can obtain significantly improved and robust results of small ship detection. The results also indicate that the proposed method can be effectively applied to ensuring autonomous ship safety.