Optimal Feedback Controls in Dynamic Stochastic Jobshops

We consider a production planning problem for a general jobshop subject to breakdown and repair of machines and subject to lower and upper bound constraints on work-in-process. The machine capacities and demand processes are assumed to be finite state Markov chains. The problem is to choose the rates of production on the various machines over time to minimize the expected discounted costs of production and inventory/backlog over an infinite horizon. It is formulated as a stochastic dynamic programming problem. We show that the value function of the problem is locally Lipschitz and is a solution to a dynamic programming equation together with a certain boundary condition. We provide an interpretation of the boundary condition, provide a verification theorem, and derive the optimal feedback control policy in terms of the directional derivatives of the value function. The results are proved via reduction to a deterministic optimal control problem that is equivalent to the stochastic produ...