A Guide to Image- and Video-Based Small Object Detection Using Deep Learning: Case Study of Maritime Surveillance

Detecting small objects in optical images and videos is a significant challenge in numerous intelligent transportation and autonomous systems. State-of-the-art generic object detection methods fail to accurately localize and identify such small objects (e.g., pedestrians, small vehicles, obstacles). Because small objects occupy only a small area in the input image (e.g., 32x32 pixels or less), the information extracted from such a small area is not always rich enough to support decision-making. Multidisciplinary strategies are being developed by researchers working at the interface of deep learning and computer vision to enhance the performance of Small Object Detection (SOD). In this paper, we provide a comprehensive review of over 160 research papers published between 2017 and 2022 in order to survey this growing subject. This paper summarizes the existing literature and provides a taxonomy that illustrates the broad picture of current research. We further explore methods to boost the performance of small object detection in maritime settings, where enhanced performance is crucial for ensuring safety and managing traffic. Detecting small objects in the maritime environment requires additional considerations and the current survey aims to review the advanced techniques addressing those aspects. In addition, the popular SOD datasets for generic and maritime applications are discussed, and also well-known evaluation metrics for the state-of-the-art methods on some of the datasets are provided.