conditioning

In a Markov problem I have noticed that some problems are solved by "conditioning on the first move out of the state". This has led to the following "general" questions.

1. Let us say we have three states A, B and C. We can go from A to B, B to C, C to B, B to A and also from B to B, A to A and C to C as well as A to C and C to A. Essentially, all states communicate directly with each other.

Conditioning on the first move out of state B = P_BA*#A + P_BC*#C + P_BB*#B. Is this the right equation?

2. When can we using conditioning as defined above and when we should not?

3. In the equation above - Is a transition from same state to same state considered transitioning out (specifically is the term P_BB*#B valid in the expression above)

Thanks

 

It appears that you're considering a Markov chain, since you mention direct transitions from A to A, B to B and C to C, which wouldn't be marked on a jump process transition diagram.

But I must admit I'm a bit uncertain what you're asking about - I think a concrete example would help (eg a question reference).

In particular, in 1 what type of thing are you trying to calculate using that expression? And what's #A etc?

 

September 2007, Q9
In a game of tennis, when the score is at “Deuce” the player winning the next point holds “Advantage”. If a player holding “Advantage” wins the following point that player wins the game, but if that point is won by the other player the score returns to “Deuce”.
When Andrew plays tennis against Ben, the probability of Andrew winning any point is 0.6. Consider a particular game when the score is at “Deuce”.
Calculate the probability that Andrew wins the game.

Abridged Solution as provided by the examiners -
By definition AGame A = 1 and AGame B = 0.
Conditioning on the first move out of state Adv A:
AAdv A = 0.6× AGame A + 0.4× ADeuce = 0.6 + 0.4× ADeuce.

The question is when do we know to condition?

 

I'd say there's not really a general rule on when you should condition. Nor is there a general rule on when you shouldn't. It's really a matter of doing it when it's useful to find your answer, which comes down to experience.

You see here that it's useful as the quantity that you're trying to calculate (the probability of winning) would take a different value depending on the next move made. So, conditioning is useful when what happens next is important.

This question is about a Markov chain. You see a similar idea for Markov jump processes when calculating the expected time to reach a furture state (see the formula in the Tables on page 38, and section 8 of Chapter 5).

 

can someone explain me this problem?

 

Can apply below method for this problem?
If yes how?
Since Same problem in Indian exam solved using equations.

Conditioning on the 1st move out of state i, i.e. starting in i, P [not visting any state ] = 0
Then forming equations for probabilities from and in to the remaining states.
Solving these equations gives required probability.

 

Yes.
I tried and finally got... Can apply the mentioned method and calculated correct probabilities.