How Does Reinforcement Learning Work (Agent; Environment; Reward; Policy)?

Reinforcement learning works through repeated interaction: an agent takes an action in an environment, receives a reward and uses that feedback to improve its policy. Its objective is to learn behaviour that maximizes cumulative reward over time.

The Reinforcement Learning Cycle

At each time step, the agent observes the current state of the environment. It selects an action according to its policy, after which the environment produces a new state and a numerical reward.

This interaction forms a feedback loop:

1. The agent observes a state.
2. The policy selects an action.
3. The environment changes state.
4. The agent receives a reward.
5. The learning algorithm adjusts the policy.
6. The cycle repeats, often across many episodes.

A policy can be represented as $\pi(a \mid s)$, meaning the probability of selecting action $a$ when the agent is in state $s$. The agent must balance exploration, which means trying unfamiliar actions, with exploitation, which means selecting actions currently expected to produce high rewards.

A common misconception is that the agent simply chooses the action with the largest immediate reward. In fact, reinforcement learning normally aims to maximize cumulative reward, so an action with a small immediate reward may lead to a better long-term outcome.

IB Exam Technique

For A4.3 Machine learning approaches (HL only), explain reinforcement learning as a complete interaction cycle rather than defining each term separately. If asked to apply it to a scenario, identify the agent, environment, actions, states, rewards and policy, then explain how reward feedback changes future decisions.