Deep Reinforcement Learning-Based Energy-Efficient Flexible Job Shop Scheduling with Multi-Speed Machines

The flexible job shop scheduling problem with multi-speed machines requires simultaneous selection of machine, sequence position, and discrete speed level for each operation, creating non-linear tradeoffs between throughput and energy consumption that existing deep reinforcement learning approaches have not jointly addressed. Prior value-based methods either treat machine speed as a fixed parameter outside the learning loop, or expand the action space without principled pruning, making training intractable at industrial scale. This paper presents a framework with three coordinated contributions: a joint job-machine-speed Markov Decision Process formulation that embeds discrete speed selection as a first-class decision variable; a Heuristic-Guided Candidate Action Set mechanism that prunes the combinatorially expanded action space using dispatching-rule-inspired priority scores; and Staged Reward Annealing, which progressively introduces an energy-regret shaping term to stabilize multi-objective policy learning. Evaluated on the Brandimarte benchmark suite against classical dispatching heuristics and NSGA-II, the learned policies achieve the highest hypervolume on four of ten instances. A counterfactual speed ablation confirms that agents dynamically exploit multispeed actuation, achieving outcomes that fixed-speed baselines cannot replicate.