Question Type 1: Construction transition matrices from diagrams or context descriptions Exercises

Construct the transition matrix for a Markov chain with states Sunny ($S$) and Rainy ($R$), given that if it is Sunny today it stays Sunny tomorrow with probability $0.8$ and becomes Rainy with probability $0.2$, and if it is Rainy today it becomes Sunny with probability $0.5$ and stays Rainy with probability $0.5$.

Interpret the third row of the transition matrix $P$:

$P = \begin{pmatrix} 0.2 & 0.5 & 0.3 \\ 0.1 & 0.7 & 0.2 \\ 0.3 & 0.2 & 0.5 \end{pmatrix}$

Explain what the entries in row 3 represent.

Given the transition matrix

$P = \begin{pmatrix} 0.6 & 0.4 \\ 0.3 & 0.7 \end{pmatrix},$

interpret the entry $p_{12} = 0.4$.

A machine can be either On ($O$) or Off ($F$). If it is On, it stays On with probability $0.9$ and switches Off with probability $0.1$. If it is Off, it stays Off with probability $0.7$ and switches On with probability $0.3$.

Write the transition matrix with states in the order $(O, F)$.

In a cohort of students, transitions each year between courses Mathematics ($M$), Physics ($P$), and Chemistry ($C$) occur as follows:

• Of those in $M$: 50 stay in $M$, 30 move to $P$, and 20 move to $C$.
• Of those in $P$: 20 move to $M$, 60 stay in $P$, and 20 move to $C$.
• Of those in $C$: 10 move to $M$, 30 move to $P$, and 60 stay in $C$.

Construct the transition matrix $\mathbf{P}$ for this system, with states ordered ($M, P, C$).

A gambler’s fortune has three levels: Low ($L$), Medium ($M$), and High ($H$).

If at $L$, he stays at $L$ with probability $0.4$ or moves to $M$ with probability $0.6$. From $M$ he stays at $M$ with probability $0.5$, moves to $H$ with probability $0.3$, or to $L$ with probability $0.2$. From $H$ he stays at $H$ with probability $0.3$ or moves to $M$ with probability $0.7$.

Construct the transition matrix $P$ in the order ($L, M, H$).

For the transition matrix
$P = \begin{pmatrix} 0.2 & 0.5 & 0.3 \\ 0.1 & 0.6 & 0.3 \\ 0.4 & 0.4 & 0.2 \end{pmatrix},$

what is the probability that the chain moves from state C (3) to state B (2) in one step?

A customer moves among four shops $A$, $B$, $C$, and $D$.

From $A$, they stay in $A$ with probability $0.1$, go to $B$ with $0.4$, $C$ with $0.3$, and $D$ with $0.2$. From $B$, they stay in $B$ with probability $0.2$, go to $A$ with $0.2$, $C$ with $0.5$, and $D$ with $0.1$. From $C$, they stay in $C$ with probability $0$, go to $A$ with $0.3$, $B$ with $0.3$, and $D$ with $0.4$. From $D$, they stay in $D$ with probability $0.25$, go to $A$ with $0.25$, $B$ with $0.25$, and $C$ with $0.25$.

Construct the transition matrix $P$ for this customer's movement in the order $(A, B, C, D)$.

Verify that the matrix $P = \begin{pmatrix} 0.5 & 0.3 & 0.2 \\ 0.4 & 0.4 & 0.2 \\ 0.2 & 0.5 & 0.3 \end{pmatrix}$ is a valid transition matrix.