Sep 2020 Q7 ii.

[Srijana Raghunath]

How to find stationary distribution of the given markov chain?
Also pls help with (iii) too.

 

[Andrew Martin]

Hello

You can find this in the usual way. Set pi * P = pi and solve the resulting equations (also using pi_i >=0 and sum(pi_i) = 1). For this particular chain, there are infinitely many solutions. You'll need to pick a value for either pi_1 or pi_4 (say a) and then work in terms of a. You should see why this is necessary when writing out the equations.

For part (iii), think about what the probability is when starting in states 1 or 4.

When starting in states 2 or 3, think of all the possible sample paths that lead to hitting state 4 and add up the resulting infinite series of probabilities.

Hope this helps

Andy

 

[Priyanka Malhotra]

Hi , could you please explain the alternate solution of part iii in examiner's report, thanks in advance!

 

[Andrew Martin]

If we start in State 2, then to reach State 4, we first move back and forth between States 3 and 2 a certain number of times before transitioning from State 2 to State 4.

So this type of path look like:

2 -> 3 -> 2 -> 3 -> 2 ... -> 3 -> 2 -> 3 -> 4

The probability of this is given by:

p23 * p32 * p23 * p32 * .... * p32 * p23 * p34

= (p23 * p32)^n * p23 * p34

where we can have n = 0, 1, 2, 3, ....

The overall probability we want is the sum of the probabilities for all these paths. Ie:

(p23 * p32)^0 * p23 * p34 + (p23 * p32)^1 * p23 * p34 + (p23 * p32)^2 * p23 * p34 + (p23 * p32)^3 * p23 * p34 + ...

= p23 * p34 * (1 + r + r^2 + r^3 + ...)

where r = p23 * p32 = 0.25 and p23 * p34 =0.25. The sum of such a geometric progression (the bit in brackets) is 1/(1-r) = 1/0.75 = 4/3.

So overall we have 0.25 * 4/3 = 1/3.

Hope this helps!

Andy