Can anyone explain this paragraph at the bottom of page 38: "The reason that the definition of n_i only allows t to go up to N-1 so that it equals to the number of chances for a transition out of i, rather than the number of times it is in i" Thanks.
Suppose you want to estimate p(12). You want to calculate the observed proportion of times that the process jumps out of state 1 and into state 2. Suppose you have ten observed times in your sample path, and the states observed were these: 1 1 1 2 1 1 2 2 1 2 There were 3 jumps observed from 1 to 2, so this is your numerator. The number of times you could have jumped from 1 to 2 is 6 (equal to the number of times the process is in state 1), so the estimate is 3/6 = 0.5. But, now consider the estimate of p(21). The estimate is 2/3, because there were 2 observed jumps from 2 to 1, and the number of possible observed jumps is 3. This is not 4, because we don't know where the process went after the tenth observation. So we exclude this last observation from the denominator of this estimate. This is what the notes are referring to here.
