Markov Chains in Discrete Time—Basic Theory

This chapter discusses the properties of a special class of discrete time chains. An intuitive derivation can be obtained using probabilistic arguments and the fact that the chain is homogeneous. The n -step transition probabilities help to examine the properties of a Markov chain (MC) at two distinct trials, n steps apart, without considering the behavior of the chain at any intermediate trial. Sometimes, the location of the chain at an in-between trial might be an issue of concern, in particular whether the final state has already been entered. By examining the transition graphs of more complicated MCs, the transition matrix is reduced to its canonical form.