Pipelining is a CPU technique that overlaps the processing stages of several instructions so that different instructions are handled simultaneously. In a multi-core CPU, each core can have its own pipeline, allowing both instruction-level and core-level parallelism.
How pipelining works
An instruction passes through stages such as fetch, decode, execute, and store. Without pipelining, the CPU completes every stage of one instruction before fetching the next instruction.
With pipelining, once instruction 1 moves from fetch to decode, instruction 2 can enter the fetch stage. After the pipeline has filled, an ideal processor may complete approximately one instruction per clock cycle, although each individual instruction still passes through every stage.
For example:
| Clock cycle | Pipeline activity |
|---|---|
| 1 | Fetch instruction 1 |
| 2 | Decode instruction 1; fetch instruction 2 |
| 3 | Execute instruction 1; decode instruction 2; fetch instruction 3 |
| 4 | Store instruction 1; execute instruction 2; decode instruction 3 |
Pipelining primarily improves throughput, meaning the number of instructions completed in a given time. It does not necessarily reduce the latency of one instruction.
Pipelining versus multiple cores
| Feature | Pipelining | Multi-core processing |
|---|---|---|
| Type of parallelism | Overlaps stages of different instructions | Executes separate instruction streams on different cores |
| Main benefit | Increases instruction throughput within a core | Allows tasks or threads to run concurrently |
| Limitation | Pipeline hazards can cause stalls | Software must support effective parallel execution |
A pipeline hazard interrupts the smooth flow of instructions. Data dependencies, branch instructions, and competition for hardware resources can produce stalls or require the pipeline to be flushed.
A common misconception is that pipelining means one instruction is executed by several cores. Instead, pipelining normally occurs within each core, while multiple cores execute different threads or tasks.
Exam technique
For an IB explain question in A1.1, identify the overlapping stages, state that throughput increases, and distinguish pipelining from multi-core parallelism. Mention hazards when evaluating why the theoretical performance improvement may not be achieved.
