What Is a Regular Markov Chain?

A regular Markov chain is a finite Markov chain whose transition matrix $T$ has some positive integer power $T^k$ in which every entry is strictly positive. A regular chain converges to a unique steady-state distribution, regardless of its initial state distribution.

How Regularity Works

In IB Mathematics: Applications and Interpretation, the convention is

$$\mathbf{s}_n=T^n\mathbf{s}_0,$$

where $\mathbf{s}_0$ is the initial column state matrix and $T_{ij}$ is the probability of moving from state $j$ to state $$. Therefore, each column of the must sum to .

Consider

$$T=\begin{pmatrix}0&0.5\\1&0.5\end{pmatrix}.$$

This matrix contains a zero, so not every transition is possible in one step. However,

$$T^2=\begin{pmatrix}0.5&0.25\\0.5&0.75\end{pmatrix}.$$

Every entry of $T^2$ is greater than zero. Thus, every state can lead to every other state in exactly two transitions, so the chain is regular with $k=2$.

Its steady state satisfies

$$T\mathbf{s}=\mathbf{s},$$

with the entries of $\mathbf{s}$ summing to $1$. Here,

$$\mathbf{s}=\begin{pmatrix}\frac{1}{3}\\\frac{2}{3}\end{pmatrix}.$$

Regularity is stronger than merely having a steady state. It ensures that, after sufficiently many transitions, the influence of the initial distribution disappears. Numerically, the columns of a large power $T^n$ approach the same vector $\mathbf{s}$. This gives the long-term proportion expected in each state, although it does not predict the exact state of any single individual.

A common misconception is that a chain is regular merely because every state is reachable. For example, $T=\begin{pmatrix}0&1\\1&0\end{pmatrix}$ is connected, but its powers always contain zeros because the chain alternates periodically.

Exam Technique

This is HL-only content from AHL 4.19: Transition matrices and Markov chains. On any GDC-active AI paper, calculate successive powers of $T$, identify a power with all entries positive, and explicitly conclude that the chain is regular; do not stop after displaying the matrix.