Estimation via Markov chain Monte Carlo

Markov chain Monte Carlo (MCMC) is a powerful means for generating random samples that can be used in computing statistical estimates, numerical integrals, and marginal and joint probabilities. The approach is especially useful in applications where one is forming an estimate based on a multivariate probability distribution or density function that would be hopeless to obtain analytically. In particular, MCMC provides a means for generating samples from joint distributions based on easier sampling from conditional distributions. Over the last 10 to 15 years, the approach has had a large impact on the theory and practice of statistical modeling. On the other hand, MCMC has had relatively little impact (yet) on estimation problems in control. The paper is a survey of popular implementations of MCMC, focusing especially on the two most popular specific implementations of MCMC: Metropolis-Hastings and Gibbs sampling.