Random Walk Markov Chain

Definition

A Random Walk is a Markov chain whose state space is given by the integers $i=0,\pm 1,\pm 2,\dots$ where at each point in time the process either moves:

• One step to the right with probability $p$
• One step to the left with probability $1-p=q$

$P_{i,i+1}=p,P_{i,i-1}=1-p$

where $0<p<1$.

Interpretation

Think of it as a model for an individual walking on a straight line who at each point in time either takes one step to the right with probability $p$ or one step to the left with probability $1-p$.

Example

See Random Walk Model Example and Gambling Model Example.

Related

• Discrete-time Markov Chain
• Gambling Model