Recurrent vs Transient

Definition

Let $f_i$ = probability that, starting from state $i$, the process will ever return to state $i$.

• Recurrent: $f_i=1$ (will return with probability 1)
• Transient: $f_i<1$ (positive probability of never returning)

Interpretation

• Transient: Will eventually leave and never come back
• Recurrent: Will always come back, no matter how many times it leaves

Unlike transition probability, this talks about whether or not a state will return to itself in the future.

Properties

• If $i$ is recurrent: returns infinitely many times
• If $i$ is transient: number of visits to $i$ has geometric distribution with finite mean $\frac{1}{1-f_i}$
• State $i$ is recurrent iff expected number of time periods in state $i$ is infinite

Example

When an absorbing state occurs, it’s absorbed there forever and never goes to other states, which makes the other states “transient”.

Related

• State
• Absorbing State
• Ergodic State