Gambling Model

Example: Gambling Model

Consider a gambler who, at each play of the game, either wins $1 with probability $p$ or loses $1 with probability $1-p$. If the gambler quits playing either when going broke or attaining a fortune of $N$, then the gambler’s fortune is a Markov chain having transition probabilities:

$P_{i,i+1}=p=1-P_{i,i-1},i=1,2,\dots ,N-1$

$P_{00}=P_{NN}=1$

States 0 and $N$ are absorbing states since once entered they are never left.

Transition Matrix

The transition matrix $\mathbf{P}$ is $(N+1)\times (N+1)$. For $N=10$, $p=0.6$:

Transition probabilities:

• $P_{i,i+1}=p=0.6$ for $i\in \{1,2,...,9\}$
• $P_{i,i-1}=1-p=0.4$ for $i\in \{1,2,...,9\}$
• $P_{0,0}=1$ (absorbing)
• $P_{10,10}=1$ (absorbing)

Simulation Examples

Parameters: $p=0.6$ (win probability), $N=10$ (target), starting fortune = $5

Simulation 1 (Success):

• Start: $5 → Win(0.6) → $6 → Lose(0.4) → $5 → Win(0.6) → $6 → Win(0.6) → $7 → Lose(0.4) → $6 → Win(0.6) → $7 → Win(0.6) → $8 → Win(0.6) → $9 → Win(0.6) → $10 (absorbed)
• Result: Reached target in 9 games

Simulation 2 (Failure):

• Start: $5 → Lose(0.4) → $4 → Lose(0.4) → $3 → Lose(0.4) → $2 → Win(0.6) → $3 → Lose(0.4) → $2 → Lose(0.4) → $1 → Lose(0.4) → $0 (absorbed)
• Result: Went broke in 7 games

Related

• Random Walk Markov Chain
• Absorbing State
• First Step Analysis - Gambler’s Ruin