Question Type 1: Construction transition matrices from diagrams or context descriptions Exercises

A machine can be either On (O) or Off (F). If it is On it stays On with probability 0.9 and switches Off with probability 0.1. If it is Off it stays Off with probability 0.7 and switches On with probability 0.3. Write the transition matrix with states in order (O,F).

Construct the transition matrix for a Markov chain with states Sunny (S) and Rainy (R), given that if it is Sunny today it stays Sunny tomorrow with probability 0.8 and becomes Rainy with probability 0.2, and if it is Rainy today it becomes Sunny with probability 0.5 and stays Rainy with probability 0.5.

Given the transition matrix
$P=\begin{pmatrix}0.6 & 0.4\\0.3 & 0.7\end{pmatrix},$
interpret the entry $p_{12}=0.4$.

For the transition matrix
$P=\begin{pmatrix}0.2 & 0.5 & 0.3\\0.1 & 0.6 & 0.3\\0.4 & 0.4 & 0.2\end{pmatrix},$
what is the probability that the chain moves from state C (3) to state B (2) in one step?

Verify that the matrix
$P=\begin{pmatrix}0.5 & 0.3 & 0.2\\0.4 & 0.4 & 0.2\\0.2 & 0.5 & 0.3\end{pmatrix}$ is a valid transition matrix.

A gambler’s fortune has three levels: Low (L), Medium (M), High (H). If at L, he stays at L with probability 0.4 or moves to M with probability 0.6. From M he stays at M with probability 0.5, moves to H with probability 0.3, or to L with probability 0.2. From H he stays at H with probability 0.3 or moves to M with probability 0.7. Construct the transition matrix in the order (L,M,H).

In a cohort of 100 students each year, transitions between courses Math (M), Physics (P) and Chemistry (C) occur as follows: of those in M, 50 stay in M, 30 move to P, 20 to C; of those in P, 20 to M, 60 stay, 20 to C; of those in C, 10 to M, 30 to P, 60 stay. Construct the transition matrix with states (M,P,C).

Interpret the third row of the transition matrix
$P=\begin{pmatrix}0.2&0.5&0.3\\0.1&0.7&0.2\\0.3&0.2&0.5\end{pmatrix}$; that is, explain what the entries in row 3 represent.

Given the transition matrix
$P=\begin{pmatrix}0.5&0.3&0.2\\0.2&0.5&0.3\\0.1&0.4&0.5\end{pmatrix},$ calculate the two-step transition probability from state 1 to state 3 by computing $(P^2)_{13}$ and interpret its meaning.

A customer moves among four shops A, B, C, D. From A they stay in A with 0.1, go to B with 0.4, C with 0.3, D with 0.2. From B they stay with 0.2, go to A with 0.2, C with 0.5, D with 0.1. From C they stay with 0, go to A with 0.3, B with 0.3, D with 0.4. From D they stay with 0.25, go to A with 0.25, B with 0.25, C with 0.25. Construct the transition matrix in order (A,B,C,D).

In the four‐shop chain above, what is the probability the customer moves from shop B to shop D in one step and how do you interpret it?

Consider the diagram with states X, Y, Z and transition probabilities shown:
![State diagram with X→X:½, X→Y:¼, X→Z:¼; Y→X:0.3, Y→Z:0.7; Z→X:0.6, Z→Y:0.2, Z→Z:0.2]
Construct the transition matrix with order (X,Y,Z) and interpret the entry $p_{23}$.