Integrated rescheduling of train timetables and rolling stock circulation for metro line disturbance management: a Q-learning-based approach

Disturbance occurs inevitably on a metro line, resulting in train delays and low service quality. To improve the efficiency of disturbance management, this article investigates the integrated rescheduling problem of train timetable and rolling stock circulation. To balance service quality and operational cost, the considered problem is formulated as a multi-objective optimization model by capturing train operation processes as running arcs, dwell arcs and coming-out/back arcs based on the classical space–time network. For the purpose of improving computational efficiency, a Q-learning-based solution approach is proposed to derive a hierarchical policy for integrated rescheduling. The decisions on train timetable follow a lower-level policy that can be learned in the pre-training process, and the decisions on rolling stock circulation follow a higher-level policy that can be generated in the real-time planning process. Finally, two sets of numerical examples are carried out to evaluate the proposed method.