CTMC Transition Time Distribution

Properties

Let $T_i$ denote the amount of time that a continuous-time Markov chain $\{X(t):t\ge 0\}$ stays in state $i$ before making a transition to a different state.

$T_i$ satisfies the memoryless property:

$\mathrm{Pr}\{T_i>t+s\mid T_i>s\}=\mathrm{Pr}\{T_i>t\},\forall s,t\ge 0$

By the Markov Property and the homogeneity of the CTMC:

$$\begin{array}{rl}\mathrm{Pr}\{T_i>t+s\mid T_i>s\}& =\mathrm{Pr}\{X(0,t+s]=i\mid X[0,s]=i\}\\ & =\mathrm{Pr}\{X(s,t+s]=i\mid X[0,s]=i\}\\ & =\mathrm{Pr}\{X(s,t+s]=i\mid X(s)=i\}\\ & =\mathrm{Pr}\{X(0,t]=i\mid X(0)=i\}\\ & =\mathrm{Pr}\{T_i>t\}\end{array}$$

Since $T_i$ is memoryless, it must be exponentially distributed:

$T_i\sim \text{Exp}({\nu }_i)$

where ${\nu }_i$ is the transition rate out of state $i$.

Interpretation

The exponential transition time is a fundamental property of CTMCs. It follows directly from the Markovian assumption: the probability that the process stays in state $i$ for at least $t$ more time units is independent of how long it has already been in that state.

Implications

• The probability that the process leaves state $i$ in a small time interval $h$ is approximately ${\nu }_{i}h$.
• The mean sojourn time in state $i$ is $\mathbb{E}[T_i]=1/{\nu }_i$.
• The exponential distribution’s memoryless property is what makes CTMCs analytically tractable.

Related

• Exponential Distribution
• Memoryless Property
• Continuous-Time Markov Chain

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Kuis 2 2025 No. 5. Pada reaksi kimia di atas, jika terdapat $j$ molekul A, setiap molekul berubah menjadi B dengan laju $q$. Tentukan ekspektasi waktu hingga semua molekul A habis.

Jawaban: ${\sum }_{j=1}^N\frac{1}{qj}$. Waktu transisi dari $j$ ke $j-1$ adalah $T_j\sim \text{Exp}(qj)$, sehingga $\mathbb{E}[T_j]=1/(qj)$. Jumlahkan untuk $j=N,N-1,\dots ,1$.

Kuis 2 2024 No. 2a. Pure birth process dimulai dari $X(0)=0$ dengan ${\lambda }_0=1$, ${\lambda }_1=3$, ${\lambda }_2=2$, ${\lambda }_3=5$. Tentukan $\mathbb{E}[W_3]$ dan $\text{Var}(W_3)$.

Jawaban: $\mathbb{E}[W_3]=\frac{1}{1}+\frac{1}{3}+\frac{1}{2}=\frac{11}{6}$. $\text{Var}(W_3)=\frac{1}{1^2}+\frac{1}{3^2}+\frac{1}{2^2}=1+\frac{1}{9}+\frac{1}{4}=\frac{49}{36}$.