Finding stationary distributions using symmetry, and the importance of being in a certian state

[Molly]

Hi all,

I'm struggling with the end of chapter 2 summary question 2.9 (page 124), please can someone help?
All parts are fine, except from the last part "Explain whether or not the process will converge to a stationary distribution given that it is in State 1 at time 0. If it does converge, determine the stationary distribution." for chain 2. there are two things i don't understand about this, which i cant seem to find in the notes:
1) I am able to find the right answer, i understand that the chain is aperiodic and so will converge BUT the solutions use a method of "symmetry" to find the stationary distribution - has anyone heard of this that could explain this to me?
2) i don't understand what the relevance of being in "state 1 at time 0" is - can we not just find the stationary distribution of process as we normally do?

 

[Andrew Martin]

Hello

For 1), the solutions are using the fact that there is not really any difference between any of the states. They are all linked together in this big loop / chain with the same probabilities going back and forth between each pair. Because of this, we can propose that the stationary distribution is (1/5,1/5,1/5,1/5,1/5), ie the same probability in each state. It is relatively straightforward to check that this indeed is the unique stationary distribution (we know it is unique as we have a finite number of states and the chain is irreducible) by showing that pi * P = pi where pi is the vector above.

For 2) The long term behaviour of any particular MC may depend on where it starts. In the specific case of a MC with a finite state space that is irreducible and aperiodic, the long term behaviour of the chain is the same regardless of where it starts. So, for this particular chain, it is true that the fact it starts in State 1 is not particularly relevant.

However, for example, consider the first chain in this question. This is irreducible and has a finite state space, however it is not aperiodic. The starting distribution plays an important role in determining the long-run probabilities of being in particular states. In the case of a starting distribution of (1,0,0,0), then, as given in the solutions, we oscillate between the distributions (0, 0.5, 0, 0.5) and (0.5,0,0.5,0). However, this chain does have a unique stationary distribution, which is (0.25,0.25,0.25,0.25). If the starting distribution is this distribution, the expected distribution for all future steps is the same, ie (0.25,0.25,0.25,0.25).

As another example, consider a markov chain with 4 states:

\( 1 \hspace{0.25cm} \rightleftharpoons 2 \hspace{0.25cm} \leftarrow 3 \rightarrow \hspace{0.25cm} 4 \hspace{0.25cm} \rightleftharpoons \)

Where the final arrows are meant to indicate that state 4 can only go to itself.

If we start in state 4, for example, the long term behaviour is the stationary distribution given by (0,0,0,1).

If we start in state 1, for example, the long term behaviour is oscillation between (1,0,0,0) and (0,1,0,0).

Hope this helps!

Andy

 

[Molly]

Hi Andy,

Thank you so much, that's so helpful - especially the example at the end!!

Hello

For 1), the solutions are using the fact that there is not really any difference between any of the states. They are all linked together in this big loop / chain with the same probabilities going back and forth between each pair. Because of this, we can propose that the stationary distribution is (1/5,1/5,1/5,1/5,1/5), ie the same probability in each state. It is relatively straightforward to check that this indeed is the unique stationary distribution (we know it is unique as we have a finite number of states and the chain is irreducible) by showing that pi * P = pi where pi is the vector above.

For 2) The long term behaviour of any particular MC may depend on where it starts. In the specific case of a MC with a finite state space that is irreducible and aperiodic, the long term behaviour of the chain is the same regardless of where it starts. So, for this particular chain, it is true that the fact it starts in State 1 is not particularly relevant.

However, for example, consider the first chain in this question. This is irreducible and has a finite state space, however it is not aperiodic. The starting distribution plays an important role in determining the long-run probabilities of being in particular states. In the case of a starting distribution of (1,0,0,0), then, as given in the solutions, we oscillate between the distributions (0, 0.5, 0, 0.5) and (0.5,0,0.5,0). However, this chain does have a unique stationary distribution, which is (0.25,0.25,0.25,0.25). If the starting distribution is this distribution, the expected distribution for all future steps is the same, ie (0.25,0.25,0.25,0.25).

As another example, consider a markov chain with 4 states:

\( 1 \hspace{0.25cm} \rightleftharpoons 2 \hspace{0.25cm} \leftarrow 3 \rightarrow \hspace{0.25cm} 4 \hspace{0.25cm} \rightleftharpoons \)

Where the final arrows are meant to indicate that state 4 can only go to itself.

If we start in state 4, for example, the long term behaviour is the stationary distribution given by (0,0,0,1).

If we start in state 1, for example, the long term behaviour is oscillation between (1,0,0,0) and (0,1,0,0).

Hope this helps!

Andy