Evolutionary algorithm incorporating reinforcement learning for energy-conscious flexible job-shop scheduling problem with transportation and setup times

Flexible job-shop scheduling is considerably important in the modern intelligent manufacturing factory. In a real job shop, transportation and setup times account for a large percentage of the total processing flow, and with today's companies demanding higher delivery times, the feasibility and punctuality of scheduling will be considerably reduced if these time constraints are ignored. Recently, several companies have become green in their manufacturing processes. However, transportation, setup, and delivery times have rarely been combined with energy efficiency. To solve this problem, we employed an integer programming approach to develop a complete mathematical model of the problem and simultaneously optimized four objectives: maximum completion time, total energy consumption, workload of critical machines, and penalties for earliness/tardiness. Subsequently, an evolutionary algorithm incorporating reinforcement learning was proposed to solve the model. The algorithm had the following features: (1) four initialization strategies were designed to obtain high-quality populations; (2) a reinforcement learning-based parameter-adaptive strategy was proposed to guide the population to select the best parameters; (3) a critical path-based neighborhood structure with transportation and setup times was designed, and according to the objectives of this study, four additional neighborhood structures were designed; (4) a reference point-based non-dominated sorting selection was presented to guide the solution toward the Pareto-optimal front; and (5) an external archive was proposed to enhance the utilization of abandoned historical solutions. Finally, the effectiveness of this algorithm was demonstrated using 33 benchmark instances of variants and comparison experiments.