Solving the vehicle-drone pickup and delivery problem in road congestion: A heuristic and its deep reinforcement learning-based improvement

In the context of last-mile logistics, the vehicle-drone delivery system increasingly snone a light on its commercial potential in logistics. It offered advantage of dispatching drones which were not influenced by road congestion, and even the parcels were transported from various origins to different destinations. Efficiently determining the appropriate synchronized vehicle-drone routes and times for pickup-and-delivery pairs in a traffic congestion scenario presented a significant challenge. We first presented a mixed-integer linear programming model and designed a tailored heuristic. Then, an improvement based on deep reinforcement learning (DRL) was proposed through learning a policy of selecting requests to remove and choose operators for the optimization of combined routes. Computational results showed the efficiency of the improved heuristic in computational time and solution quality. We have also applied the DRL-based improvement in realistic road congestion scenarios in Beijing and estimated potential annual cost savings of 10.49 billion CNY through the implementation of a vehicle-drone delivery system.