Biased training data can cause an AI system to learn patterns that systematically disadvantage particular groups. Since the model uses past data to classify new cases, existing inequalities may be reproduced or amplified.
The Reasoning and Mechanism
Training data consists of examples used to adjust an AI model's parameters. If examples are unrepresentative, historically unequal, or incorrectly labelled, the model may produce algorithmic bias: systematic outcome differences that unfairly benefit or harm groups.
The mechanism occurs in stages:
- Data is collected from a population or past decisions.
- Bias enters through sampling, measurement, labelling, or existing inequality.
- The model identifies statistical correlations in that data.
- It applies those correlations to new cases, producing unequal outcomes.
- If its decisions become future training data, a feedback loop can reinforce the bias.
| Source of bias | Example | Possible unfair outcome |
|---|---|---|
| Selection bias | Facial-recognition data underrepresents some demographic groups | Lower recognition accuracy for underrepresented groups |
| Historical bias | Past recruitment data favours male applicants | Qualified female applicants receive lower scores |
| Label bias | Assessors label similar behaviour differently across groups | The model learns inconsistent classifications |
| Proxy variables | A postcode correlates with ethnicity or income | Credit decisions indirectly discriminate |
Accuracy alone does not guarantee fairness. A model may have high overall accuracy while producing more false positives or false negatives for one group. Developers should use representative datasets, compare outcomes across demographic groups, and conduct regular bias audits.
A common misconception is that AI is automatically objective because it uses mathematics. Models learn from human-selected data and design choices, so outputs can reflect societal biases.
IB Exam Technique
For an A4.4 ethical considerations question, explain the full causal chain: biased data, learned correlation, unequal prediction, and real-world harm. If asked to evaluate, balance benefits against discrimination, accountability, transparency, and the need for human oversight.
