Driver awareness collision/proximity detection system for heavy vehicles based on deep neural network

Abstract Heavy vehicles are essentially used to perform several critical tasks in construction industry. These heavy vehicles are different in shape and structure according to their capabilities and needs on the construction sites. Bigger size and different design of these vehicles result in extended blind spot compare to the normal vehicles, thus risking the equipment and workers operating around them. Each year, a large number of incidents are reported involving human injuries or loss of life due to improper handling of blind spots in heavy vehicles. In this paper we developed a deep learning based collision detection and driver assistance methodology by removing the blind spots in haul truck. In the proposed methodology, vision sensor data is used for automatic detection and classification of objects, employing the YOLO-v5 network architecture. For distance measurement of the objects we used stereo imaging camera Intel RealSense D455. The proposed algorithm detects and classify the object around the vehicle and provide the captioning to describe the nature of object to the driver. In addition, the algorithm also measures the distance of objects from the vehicle. The experimental results validate the performance of our proposed methodology in real time.