Existence of CTMC Limit Probabilities

Theorem

If a CTMC $\{X(t):t\ge 0\}$ on state space $S$ is:

1. Irreducible, and
2. Positive recurrent

then for all $j\in S$:

${\pi }_j={\lim }_{t\to \infty }{P}_{ij}(t)$

exists and is independent of the initial state $i$. Such a process is called ergodic.

Interpretation

The conditions mirror those for discrete-time Markov chains: irreducible (all states communicate) and positive recurrent (expected return time is finite) guarantee a unique limiting distribution. Since CTMCs have no periodicity issues, these two conditions are sufficient.

Implication: Balance Equations

When the limit distribution exists, the steady-state probabilities ${\pi }_j\ge 0$ satisfy the system of linear equations (balance equations):

${\nu }_j{\pi }_j={\sum }_{k\ne j}{q}_{kj}{\pi }_k,\forall j$ ${\sum }_{k\in S}{\pi }_k=1$

Also: ${\pi }_j={\nu }_j/({\sum }_{i\in S}{\nu }_i)$ in terms of the sojourn rates.

NOTE

In practice, it is very difficult to prove positive recurrence for chains with infinitely many states. Instead, we solve the balance equations and determine the condition under which these probabilities exist.

Related

• CTMC Limit Probabilities
• CTMC Balance Equations
• Irreducible
• Ergodic State

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Cek konvergensi BD process. Untuk BD process dengan ${\lambda }_n=\lambda$, ${\mu }_n=n\mu$ (M/M/∞), periksa apakah limit probabilitas ada.

Jawaban: ${\theta }_n=\frac{{\lambda }_0\cdots {\lambda }_{n-1}}{{\mu }_1\cdots {\mu }_n}=\frac{{\lambda }^n}{n!{\mu }^n}=\frac{(\lambda /\mu {)}^n}{n!}$. ${\sum }_{n=0}^{\infty }{\theta }_n=\sum \frac{(\lambda /\mu {)}^n}{n!}=e^{\lambda /\mu }<\infty$. Jadi limit probabilitas ada: ${\pi }_n=\frac{(\lambda /\mu {)}^n{e}^{-\lambda /\mu }}{n!}$ (Poisson).

Kondisi perlu ergodisitas. Sebutkan dua syarat agar CTMC ergodik.

Jawaban: (1) Irreducible — semua state berkomunikasi. (2) Positive recurrent — expected return time ke setiap state finite. CTMC tidak memiliki masalah periodisitas (selalu aperiodik).