Q-Learning-Based Multi-Objective Heuristic Algorithm for Solving Multi-Resource Co-Scheduling Problem of Ship-Lock-Channel Considering Water Discharge

Approach channel, ship lock and ship lift are critical resources of Three Gorges-Gezhou Dams (TGGD) along the Yangzi River. Encouraged by the operational scenarios of TGGD, this paper investigates a multi-resource co-scheduling problem of ship-lock-channel, where the synchronous velocities of vessels are influenced by freeboard height and steering skill. The vessel’ sailing interval, sailing sequence and sailing point are closely related with the water discharge in flood season. A mixed integer programming model is established to simultaneously minimize the water resource usage, average waiting time and carbon emissions over all vessels. The vessel grouping, synchronous moving and waiting process are incorporated into the scheduling model. Thereafter, a problem-specific Q-learning-based multi-objective heuristic algorithm, named QLMHA, is proposed to optimize the multi-resource co-schedule. With a well-tailored bi-level encoding scheme, QLMHA makes use of hybrid multi-stage decoding mechanism and fuzzy correlation entropy analysis to respectively decode and assess each candidate solution. To enhance the searchability of the QLMHA, Q-learning technique is coupled to select the optimal action for solution set obtained by external archive. Numerical experiments are implemented by using the practical data from TGGD. The statistical result corroborates the higher effectiveness and practicality of the QLMHA in terms of solving the concerned problem. In addition, our findings demonstrate that QLMHA is able to generate more satisfactory schemes, offering TGGD and shipping managers a broader array of preferred choice.