Hello You can find this in the usual way. Set pi * P = pi and solve the resulting equations (also using pi_i >=0 and sum(pi_i) = 1). For this particular chain, there are infinitely many solutions. You'll need to pick a value for either pi_1 or pi_4 (say a) and then work in terms of a. You should see why this is necessary when writing out the equations. For part (iii), think about what the probability is when starting in states 1 or 4. When starting in states 2 or 3, think of all the possible sample paths that lead to hitting state 4 and add up the resulting infinite series of probabilities. Hope this helps Andy
Hi , could you please explain the alternate solution of part iii in examiner's report, thanks in advance!
If we start in State 2, then to reach State 4, we first move back and forth between States 3 and 2 a certain number of times before transitioning from State 2 to State 4. So this type of path look like: 2 -> 3 -> 2 -> 3 -> 2 ... -> 3 -> 2 -> 3 -> 4 The probability of this is given by: p23 * p32 * p23 * p32 * .... * p32 * p23 * p34 = (p23 * p32)^n * p23 * p34 where we can have n = 0, 1, 2, 3, .... The overall probability we want is the sum of the probabilities for all these paths. Ie: (p23 * p32)^0 * p23 * p34 + (p23 * p32)^1 * p23 * p34 + (p23 * p32)^2 * p23 * p34 + (p23 * p32)^3 * p23 * p34 + ... = p23 * p34 * (1 + r + r^2 + r^3 + ...) where r = p23 * p32 = 0.25 and p23 * p34 =0.25. The sum of such a geometric progression (the bit in brackets) is 1/(1-r) = 1/0.75 = 4/3. So overall we have 0.25 * 4/3 = 1/3. Hope this helps! Andy
