Hitting Probability (Transient)

Definition

The probability that the Markov chain ever enters transient state $j$ given that it starts in transient state $i$ is:

${\pi }_i(j)=\frac{{\mu }_{ij}-{\delta }_{ij}}{{\mu }_{jj}}$

where ${\delta }_{ij}=1$ if $i=j$ and $0$ otherwise.

Notation

• ${\mu }_{ij}$: expected number of visits to state $j$ starting from $i$
• ${\mu }_{jj}$: expected number of visits to state $j$ starting from $j$

Example

For $N=7,p=0.4$ (lose probability $0.6$), starting with 3 units:

• ${\mu }_{3,5}=0.9228$ (Expected time at 5 units)
• ${\mu }_{3,2}=2.3677$ (Expected time at 2 units)
• Probability of ever hitting 1: ${\pi }_3(1)={\mu }_{3,1}/{\mu }_{1,1}=1.4206/1.6149=0.8797$

Related

• Mean Time Spent in Transient States
• Limiting vs Hitting Probability
• First Step Analysis