Efficient transportation management is essential for the sustainability and safety of modern urban infrastructure. Traditional road inspection and transport management methods are often labor-intensive, time-consuming, and prone to inaccuracies, limiting their effectiveness. This study presents a UAV-based transport management system that leverages machine learning techniques to enhance road anomaly detection and severity assessment. The proposed approach employs a structured three-tier model architecture: A unified obstacle detection model identifies six critical road hazards—road cracks, potholes, animals, illegal dumping, construction sites, and accidents. In the second stage, six dedicated severity classification models assess the impact of each detected hazard by categorizing its severity as low, medium, or high. Finally, an aggregation model integrates the results to provide comprehensive insights for transportation authorities. The systematic approach seamlessly integrates real-time data into an interactive dashboard, facilitating data-driven decision-making for proactive maintenance, improved road safety, and optimized resource allocation. By combining accuracy, scalability, and computational efficiency, this approach offers a robust and scalable solution for smart city infrastructure management and transportation planning.