Algorithmic bias is the systematic production of unfair or unequal outcomes by an algorithm for particular people or communities. It occurs when bias enters through data, design choices, objectives, testing, or the social context of deployment.
This topic sits within IB Digital Society 3.2 Algorithms and is shared by SL and HL.
Algorithms follow defined instructions, but people decide what problem to solve, which data to use, what outcome to optimize, and how outputs affect decisions. Bias can therefore enter at several points:
| Source of bias | How it produces unequal outcomes |
|---|---|
| Training data | Historical data may contain discrimination or underrepresent some groups, causing a model to reproduce those patterns. |
| Data collection | A sample may exclude people with limited access to digital systems, making it unrepresentative. |
| Design choices | Developers choose variables, categories, and optimization objectives that reflect assumptions and values. |
| Proxy variables | Postcode, for example, may indirectly represent ethnicity or socioeconomic status. |
| Deployment context | A system developed for one population may be unreliable for another. |
| Feedback loops | Biased outputs can create new data that reinforces the original pattern. |
A common misconception is that algorithms are neutral because they use mathematics. Their procedures may be consistent, but inputs, objectives, and applications are shaped by human decisions and unequal social systems.
System: Amazon developed a machine-learning recruitment tool to rank applicants. Example specifics: In 2018, Reuters reported that Amazon abandoned it after finding that it disadvantaged applications containing indicators associated with women, reflecting historical recruitment patterns. Impacts and implications: The tool risked restricting employment opportunities. Its failure also highlighted the need for representative data, auditing, and human accountability. Concepts: The case exposes unequal power between employers and applicants and raises questions of fairness and responsibility.
In exams, explain the causal chain and distinguish observed impacts from future implications for stakeholders.
