Long term distribution of Markov Chains

[Richard_07]

Say we have a transition matrix with entries P11= 5/6, P12=1/6, P21=0 and P22=1. This is clearly not irreducable chain and aperiodic. So according to Notes this chain does not have equilibrium distribution ( because to have it chain has to be finte, irreducable and aperiodic).

However, as I understand it this chain does have equilibrium distribution (0,1), because the process will end up in state 2 in the long term.

So how to reconcile these two results?

Thanks

 

[Michael Hosking]

The example you give is not irreducible, but it is aperiodic.

The results relating to these ideas are as follows

(1) If it is finite, it has a long term stationary distribution.

(2) If it is finite and irreducible, it has a unique long term stationary distribution.

(3) If it is finite, irreducible and aperiodic, it has a unique long term stationary distribution, which it does actually reach.

But note that these implications are not reversible, for example if there is unique long term stationary distribution, this does not necessarily imply that the chain is finite and irreducible.

This is a case in point. It's not irreducible, but it does have a single long term stationary distribution.

In fact, if the chain is not irreducible, we usually reduce it, ie ignore all the states which can't be "got back to" in the long term. It is then usual to consider the subchain which only contains the states which can be occupied in the long term, and to find the long term probabilities for this subchain.