An EM Algorithm for Markov Modulated Markov Processes

An expectation-maximization (EM) algorithm for estimating the parameter of a Markov modulated Markov process in the maximum likelihood sense is developed. This is a doubly stochastic random process with an underlying continuous-time finite-state homogeneous Markov chain. Conditioned on that chain, the observable process is a continuous-time finite-state nonhomogeneous Markov chain. The generator of the observable process at any given time is determined by the state of the underlying Markov chain at that time. The parameter of the process comprises the set of generators for the underlying and conditional Markov chains. The proposed approach generalizes an earlier approach by Ryden for estimating the parameter of a Markov modulated Poisson process.