Kolmogorov Differential Equations

Theorem

The Kolmogorov differential equations describe the evolution of the transition probability function $P_{ij}(t)$ of a CTMC.

Forward Equation

Starting from the Chapman-Kolmogorov equation and taking $h\to 0$:

$\frac{d}{dt}{P}_{ij}(t)={\sum }_{k\ne j}{P}_{ik}(t)q_{kj}-P_{ij}(t){\nu }_j$

where $q_{kj}$ are entries of the Q-matrix and ${\nu }_j={\sum }_{k\ne j}{q}_{jk}$.

Interpretation (Forward)

The rate of change of $P_{ij}(t)$ equals the rate at which transitions arrive at $j$ from other states $k$ (via $q_{kj}$) minus the rate at which the process leaves $j$ (at rate ${\nu }_j$). The process runs forward: from $0$ to $t$ to $t+h$.

Backward Equation

$\frac{d}{dt}{P}_{ij}(t)={\sum }_{k\ne i}{q}_{ik}{P}_{kj}(t)-{\nu }_i{P}_{ij}(t)$

Interpretation (Backward)

The process runs backward: it starts $h$ later, first making an instantaneous transition from $i$ to $k$ (at rate $q_{ik}$), then proceeding to $j$ in time $t$. Alternatively, it stays in $i$ for small time $h$ then moves to $j$ in time $t$ (at rate $-{\nu }_i$).

Matrix Form

Let $P(t)=(P_{ij}(t))$ and $Q$ be the Q-matrix.

• Forward: $P^{\prime }(t)=P(t)Q$
• Backward: $P^{\prime }(t)=QP(t)$

Solution

The solution is $P(t)=e^{Qt}$, where $e^{Qt}$ is the matrix exponential.

Related

• CTMC Transition Probability Function
• Transition Rate Matrix
• Matrix Exponential
• CTMC Chapman-Kolmogorov Equation

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Kuis 2 2025 No. 8. Untuk $Q=\left[\begin{array}{ccc}-2& 2& 0\\ 1& -3& 2\\ 0& 2& -2\end{array}\right]$, tuliskan persamaan forward untuk $P_{00}(t)$.

Jawaban: $\frac{d}{dt}{P}_{00}(t)=P_{00}(t)q_{00}+P_{01}(t)q_{10}+P_{02}(t)q_{20}=-2P_{00}(t)+P_{01}(t)\cdot 1$.

CTMC 2-state. Untuk $Q=\left[\begin{array}{cc}-\lambda & \lambda \\ \mu & -\mu \end{array}\right]$, selesaikan persamaan backward untuk $P_{00}(t)$.

Jawaban: $\frac{d}{dt}{P}_{00}(t)=-\lambda P_{00}(t)+\lambda P_{10}(t)$. Dengan $P_{10}=1-P_{00}$: $\frac{d}{dt}{P}_{00}=-\lambda P_{00}+\lambda (1-P_{00})=\lambda -(\lambda +\mu )P_{00}$. Solusi: $P_{00}(t)=\frac{\mu }{\lambda +\mu }+\frac{\lambda }{\lambda +\mu }{e}^{-(\lambda +\mu )t}$.