A Multimodal Embedded System for Real-Time UAV Detection using Vision and Acoustic Signals

The increase of unmanned aerial vehicle (UAV) usage has created difficulties in surveillance, security monitoring and airspace safety. It is necessary to detect unauthorized UAV(s) in real-time in order to protect sensitive areas including critical infrastructure, restricted zones and public events. This paper presents a low-power, embedded; UAV Detection System with vision-based object detection and Acoustic signal detection for real-time UAV activity monitoring and detection. Using the YOLOv8 deep learning model to detect UAVs in video images from a surveillance camera as well as a MAX4466 microphone sensor for the detection of UAV-specific rotor noise, a complete UAV detection system can be accomplished. Through the use of data fusion of both visual and acoustic UAV detection data, our system improves reliability in the performance of UAV detection as well as reduces false alarms. Acoustic signal acquisition and preprocessing will occur using an STM32 microcontroller while an ESP32 module will handle communication and alert transmission. Alert notifications will occur in real-time when a UAV is detected through the use of LED, buzzer, GSM-based SMS alerts, and alerts via Telegram. Experimental results show that the UAV Detection System provides reliable UAV detections with the performance of real-time detection capabilities through the fusion of deep learning-based using sound to detect vision is a specialized approach for embedded UAV surveillance and is effective for UAV surveillance systems.