CTMC Transition Probability Function

Definition

The transition probability function (TPF) of a CTMC is:

$P_{ij}(t)=\mathrm{Pr}\{X(t+s)=j\mid X(s)=i\},t\ge 0$

where $P_{ij}(t)$ denotes the probability that a process presently in state $i$ will be in state $j$ a time $t$ later.

Initial Conditions

$P_{ij}(0)=\{\begin{array}{ll}1,& i=j\\ 0,& i\ne j\end{array}$

Infinitesimal Behavior

For small time $h>0$, the TPF is related to the transition rate matrix $Q=(q_{ij})$:

$P_{ij}(h)=q_{ij}h+o(h),i\ne j$ $P_{ii}(h)=1-{\nu }_{i}h+o(h)$

where ${\nu }_i={\sum }_{j\ne i}{q}_{ij}$.

Interpretation

The TPF fully characterizes the probabilistic behavior of a CTMC over any time interval $t$. While the Q-matrix describes instantaneous behavior, the TPF describes behavior over arbitrary duration.

Two Specifications

A CTMC can be specified by either:

1. The Q-matrix (transition rates) — describes instantaneous dynamics
2. The TPF $P_{ij}(t)$ — describes behavior over any time $t$

The TPF can be obtained from $Q$ by solving the Kolmogorov differential equations.

Related

• Transition Rate Matrix
• Kolmogorov Differential Equations
• CTMC Chapman-Kolmogorov Equation

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Kuis 2 2025 No. 8. Diberikan $Q=\left[\begin{array}{ccc}-2& 2& 0\\ 1& -3& 2\\ 0& 2& -2\end{array}\right]$. Manakah yang benar?

(A) $P_{00}(t)=e^{-2t}$, $P_{01}(t)=2(1-e^{-2t})$

Jawaban: Salah. $P_{00}(t)=e^{-2t}$ hanya benar jika dari state 0 tidak bisa kembali — tetapi dari $Q$ terlihat state 0 selalu bertransisi ke 1, dan dari 1 bisa kembali ke 0. TPF harus memperhitungkan semua kemungkinan jalur.

(B) $P_{02}(t)=0.5$ untuk $t$ besar — Salah, karena ${\pi }_2\approx 0.5$ bukan berarti $P_{02}(t)=0.5$ untuk semua $t$.

(C) $P_{00}(t)=e^{-2t}$, $P_{02}(t)=0$ — Salah, state 2 bisa dicapai dari 0 melalui state 1.