Time Reversible CTMC

Definition

An ergodic CTMC is time reversible if the process run backward in time has the same probabilistic structure as the original process.

Condition for Time Reversibility

$P_i{q}_{ij}=P_j{q}_{ji},\forall i\ne j$

where $P_i$ are the limiting probabilities and $q_{ij}$ are transition rates.

Interpretation

The rate at which the process goes directly from state $i$ to state $j$ equals the rate at which it goes directly from $j$ to $i$. The process “looks the same” whether time runs forward or backward.

Embedded Chain Perspective

Time reversibility of the CTMC is equivalent to time reversibility of the embedded DTMC:

${\pi }_i{P}_{ij}={\pi }_j{P}_{ji},\forall i,j$

where ${\pi }_i$ are the embedded chain’s limiting probabilities.

Relation Between CTMC and Embedded Probabilities

$P_i=\frac{{\pi }_i/{\nu }_i}{{\sum }_j{\pi }_j/{\nu }_j}$

Reversed Process

Going backward in time:

• The time spent in state $i$ is still exponentially distributed with rate ${\nu }_i$
• The sequence of states forms a DTMC with transition probabilities $Q_{ij}=\frac{{\pi }_j{P}_{ji}}{{\pi }_i}$

Finding Limiting Probabilities via Time Reversibility

Proposition: If we can find probabilities $\{P_i\}$ with $\sum P_i=1$, $P_i\ge 0$, satisfying:

$P_i{q}_{ij}=P_j{q}_{ji},\forall i\ne j$

then the chain is time reversible and $\{P_i\}$ are the limiting probabilities.

This often provides an easier way to find limiting probabilities than solving balance equations directly.

Related

• Ergodic Birth and Death Process is Time Reversible
• Truncated Time Reversible CTMC
• Continuous-Time Markov Chain
• Time Reversible Markov Chain

Exercises

Back to Roadmap 📖 → 🃏 → ✏

Verifikasi time reversibility. Untuk BD process ergodik, buktikan bahwa $P_i{q}_{i,i+1}=P_{i+1}{q}_{i+1,i}$ (syarat time reversibility) ekuivalen dengan ${\lambda }_i{P}_i={\mu }_{i+1}{P}_{i+1}$.

Jawaban: $q_{i,i+1}={\lambda }_i$ (birth rate), $q_{i+1,i}={\mu }_{i+1}$ (death rate). Jadi $P_i{\lambda }_i=P_{i+1}{\mu }_{i+1}$, yang merupakan persamaan detailed balance yang sudah terbukti dari balance equation BD process. Maka semua BD process ergodik bersifat time reversible.

M/M/1 truncated. M/M/1 dengan $\lambda =3$, $\mu =4$ ditruncate ke state $\{0,1,2\}$ (M/M/1/2). Jika original ${\pi }_n=(1-\rho ){\rho }^n$ dengan $\rho =3/4$, tentukan ${\pi }_n$ untuk truncated chain.

Jawaban: ${\pi }_n^A=\frac{{\pi }_n}{{\sum }_{i=0}^2{\pi }_i}=\frac{(1-\rho ){\rho }^n}{(1-\rho )(1+\rho +{\rho }^2)}=\frac{{\rho }^n}{1+\rho +{\rho }^2}$. ${\pi }_0=\frac{16}{49}$, ${\pi }_1=\frac{12}{49}$, ${\pi }_2=\frac{9}{49}$.