CTMC Limit Probabilities

Definition

For a CTMC $\{X(t):t\ge 0\}$ with transition probability function $P_{ij}(t)$, the limit probabilities (or steady-state probabilities) are:

${\pi }_j={\lim }_{t\to \infty }{P}_{ij}(t),\forall j\in S$

The process will be in state $j$ at time $t$ no matter where it initially started. If such limits exist, they are called the limit probabilities of $\{X(t)\}$.

Interpretation

${\pi }_j$ is the long-run proportion of time the process spends in state $j$. If ${\pi }_j=0.3$, then in the long run, the process is in state $j$ for 30% of the time.

Key Properties

When limit probabilities exist (${\pi }_j>0$, $\sum {\pi }_j=1$), they are also stationary probabilities:

${\pi }_j={\sum }_i{\pi }_i{P}_{ij}(t),\forall t\ge 0$

If the initial state is chosen according to $\{{\pi }_j\}$, then the probability of being in state $j$ at time $t$ is ${\pi }_j$ for all $t$.

From Kolmogorov to Limit Probabilities

When steady state is reached, the Kolmogorov forward equation yields:

$0=-{\pi }_j{\nu }_j+{\sum }_{k\ne j}{\pi }_k{q}_{kj}$

which is the balance equation — rate at which the process leaves $j$ equals rate at which it enters $j$.

Matrix Form

Let $1$ be a column vector of 1’s and $\mathbf{\pi }$ be the row vector of limit probabilities. Then:

$P\equiv {\lim }_{t\to \infty }P(t)=1\mathbf{\pi }$

And: $\mathbf{\pi }P=\mathbf{\pi },\mathbf{\pi }Q=0$

NOTE

The process is called ergodic when the limiting probabilities exist.

Related

• Existence of CTMC Limit Probabilities
• CTMC Balance Equations
• Continuous-Time Markov Chain
• Stationary Distribution

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Kuis 2 2025 No. 9. Diberikan $Q=\left[\begin{array}{ccc}-2& 2& 0\\ 1& -3& 2\\ 0& 4& -4\end{array}\right]$ pada $S=\{1,2,3\}$. Tentukan ${\lim }_{t\to \infty }P[X_t=3\mid X_0=1]$.

Jawaban: Selesaikan $\mathbf{\pi }Q=0$ dengan $\sum {\pi }_i=1$: $\begin{array}{rlrl}-2{\pi }_1+{\pi }_2& =0& \Rightarrow {\pi }_2& =2{\pi }_1\\ 2{\pi }_1-3{\pi }_2+4{\pi }_3& =0& \Rightarrow 2{\pi }_1-6{\pi }_1+4{\pi }_3=0& \Rightarrow {\pi }_3={\pi }_1\\ 2{\pi }_2-4{\pi }_3& =0& \Rightarrow 4{\pi }_1-4{\pi }_1=0& \text{(konsisten)}\end{array}$ ${\pi }_1+2{\pi }_1+{\pi }_1=1\Rightarrow {\pi }_1=1/4$. Jadi ${\pi }_3=1/4$. ${\lim }_{t\to \infty }P[X_t=3\mid X_0=1]={\pi }_3=1/4$.

Kuis 2 2025 No. 6. M/M/1/2: $\lambda =3$/jam, $\mu =4$/jam. Tentukan rata-rata jumlah tugas dalam sistem ($L$).

Jawaban: $\rho =3/4$. Untuk M/M/1/2 (kapasitas 2): ${\pi }_1=\rho {\pi }_0$, ${\pi }_2=\rho {\pi }_1={\rho }^2{\pi }_0$. ${\pi }_0(1+\rho +{\rho }^2)=1\Rightarrow {\pi }_0=\frac{1}{1+3/4+9/16}=\frac{16}{49}$. $L=0\cdot {\pi }_0+1\cdot {\pi }_1+2\cdot {\pi }_2=\frac{12}{49}+\frac{18}{49}=\frac{30}{49}$.