Markov Decision Processes with a Borel Measurable Cost Function—The Average Case

We consider a Markov decision process with a Borel measurable cost function. We introduce a p-step contraction property for the average cost case. By use of this method, the validity of the optimality equation and the existence of ϵ-optimal stationary policies are proved. As some applications, the sequential replacement model and the inventory model are considered.