Hi all, I understand the concept of periodic/aperiodic states when analyzing a markov chain but I am struggling to think of real-life applications, as that will help really cement the concept for me. I haven't come across this is in my line of work, if someone can describe an application, I would appreciate it very much. Thanks!
Hello Aperiodic Consider a simple no claims discount model where we have 3 levels, 0%, 10% and 20%. If a policyholder has one or more claims, they move down a single level of discount (or stay at 0%) - say this happens with probability p > 0. If a policyholder has no claims then they move up a level of discount (or stay at 20%), say this happens with probability 1 - p. All three states in this model are aperiodic. Mathematically, this means that the highest common divisor of return times is 1. Although not stated explicitly in the notes (I don't think) another handy result for aperiodic states is as follows: If state i is aperiodic then there exists an N such that N is the smallest such integer such that P_ii(n) > 0 for all n > N. In other words, there exists an N such that it is possible to return to state i in all possible step counts over N. For example, if N was 4 for a particular state i then this means that when starting in state i, it is possible to be back in state i after times 5, 6, 7, 8, 9, 10,... So, practically speaking, what this means is that if a policyholder starts in the state 0%, there exists an N such that it is possible to be back in state 0% after any number of steps > N. In this simple model, N here is 0 because state 0% has a loop to itself so the possible return times 1,2,3,4,.... However, for state 10%, the possible return times are 2,3,4,5,.... (as you have to leave the 10% state to either 0% or 20%). So here we can think of N as being 1. So, one way to think about this is that if we look at a number of steps far enough into the future, it is possible to be back at that aperiodic state (having started there) after any number of steps past that point. Periodic Consider the symmetric random walk, which is defined by: X0 = 0 Xn = Sum(i = 1, n) Yi Yi = -1 with probability p 1 with probability 1-p This could be modelling gambling outcomes where each time the gambler wins, they win £1 and each time they lose, they lose £1. This is an example of a markov chain where the probability of going from n to n+1 = p and the probability of going from n to n-1 is 1-p. All the states here are periodic with period 2. Mathematically, this is because the highest common divisor of the return times is 2 (where the return times are 2,4,6,8,10 etc). Practically speaking, this means that if we start at a particular number, say 21, then we can only get back to 21 after an even number of steps. A practical conclusion we can make about this fact is that after an odd number of gambles we must either be worse or better off than we started, whereas with an even number of gambles, it is possible for us to be even overall. Overall One of the really important practical results for the case when we have an irreducible, aperiodic MC with a finite state space is that the long-term behaviour of the MC is determined by the unique stationary distribution. So, taking the NCD model as an example, this means that the distribution of policyholders remains constant over the states (technically the expected distribution is constant as there will always be random variation. However, with a large enough number of policyholders it should be pretty close). So if the operation has been running long enough, the insurer will know what proportion are expected in each of the states 1, 2 and 3 (from which they can calculate their expected premium income over the whole group). Hope this helps! Andy
