Chapter 3 qns
Hi JamaicanJem,
Yes, you've answered your own question! The distribution (a,b,c) shows the proportions of people on each discount level. At time n, everyone is on 0% discount, so the distribution is (1,0,0).
n(ijk) is the number of times we observe the triple of going from state i to state j to state k. So n(132) is the number of times we go from state 1 to state 3 to state 2. We would need a 3D matrix to record all the possible triples so, instead, we have a matrix for n(1jk), a matrix for n(2jk) and a matrix for n(3jk). So if we go (from state 1 to state 3 to state 2) five times say, we have n(132) = 5 and this would be recorded in the 3rd row and 2nd column of the n(1jk) matrix.
To be Markov we would need that p(32) is not affected by the fact that we arrived in state 3 from state 1. ie if we set up the possible triple 132 by going from state 1 to state 3, the probability of completing it is p(32):
ie n(132) follows Bin {n(13), p(32)}
However, we need to be careful - n(13) should only record the number of times we set up the possible triple. So, if the very last transition is from state 1 to state 3, it would be unfair to include this in n(13) because we never get to find out whether the triple is completed. Watch out for this as you count doubles n(ij)
Happy Xmas
John
