A1.1.3 Differences Between the CPU and GPU (HL only) Notes

Design Philosophy and Usage Scenarios

Think of it like this:

1. The CPU is the brain, excellent at decision-making and multitasking.
2. The GPU is the muscle team, built to do lots of the same job, fast and in parallel.

CPU Design Philosophy

Flexibility and Generalization

1. CPUs are designed to handle a wide variety of tasks, from running operating systems to executing complex algorithms.
2. They excel at sequential processing, making them ideal for tasks that require complex logic and decision-making.

Key Features

1. Fewer, More Powerful Cores: Each core is capable of handling complex instructions with high clock speeds.
2. Branch Prediction: CPUs use advanced techniques to anticipate the next instruction, minimizing delays.
3. Instruction Versatility: They support a broad set of instructions, making them suitable for general-purpose computing.

GPU Design Philosophy

High Throughput and Parallelism

1. GPUs are designed for tasks that can be broken down into smaller, independent pieces.
2. They excel at parallel processing, making them ideal for graphics rendering, scientific simulations, and machine learning.

Key Features

1. Thousands of Cores: Each core is less powerful than a CPU core but designed for simple, repetitive tasks.
2. SIMD Architecture: GPUs use Single Instruction, Multiple Data (SIMD) operations to apply the same instruction to many data elements at once.
3. High Memory Bandwidth: Designed to move data efficiently between cores and memory.

Usage Scenarios

1. CPUs
   1. Running operating systems and managing system resources
   2. Executing general-purpose software (e.g., web browsers, office applications)
   3. Handling user input and multitasking
2. GPUs
   1. Graphics rendering for gaming and video editing
   2. Accelerating scientific simulations and machine learning
   3. Processing large datasets in parallel

Core Architecture, Processing Power, and Memory Access

Core Architecture

1. CPUs
   1. Fewer, More Powerful Cores: Typically 4–16 cores, each capable of handling complex instructions.
   2. High Clock Speeds: Enable fast execution of sequential tasks.
   3. Advanced Features: Include branch prediction and out-of-order execution.
   4. Limited Parallelism: Relying more on multi-threading
2. GPUs
   1. Massive Parallelism: Hundreds to thousands of cores designed for simple, repetitive tasks.
   2. SIMD Capabilities: Allow the same operation to be performed on multiple data points simultaneously.
   3. Specialized Cores: Some GPUs include tensor cores for machine learning.

Processing Power

1. CPUs
   1. Optimized for Sequential Tasks: High instructions per cycle (IPC) and multithreading capabilities.
   2. Versatile: Can handle a wide range of instructions, from arithmetic to complex logic.
2. GPUs
   1. Optimized for Parallel Tasks: High throughput for tasks like matrix multiplications and pixel processing.
   2. Specialized Instructions: Include SIMD and texture mapping for graphics and scientific computing.

Memory Access

1. CPU accesses system RAM (large, general-purpose, slower).
2. GPU uses VRAM (Video RAM), fast and located on the graphics card.
3. VRAM is optimised for sequential, high-bandwidth access to visual data.
4. CPUs
   1. Memory Hierarchy: Utilize caches (L1, L2, L3) to minimize latency.
   2. Cache Coherence: Ensure consistent data across multiple cores.
5. GPUs
   1. High Bandwidth Memory: Prioritize throughput over low latency.
   2. Unified Memory Architecture: Some GPUs share memory with the CPU, simplifying data transfer.

Power Efficiency

1. CPUs
   1. Dynamic Voltage and Frequency Scaling (DVFS): Adjust power usage based on workload.
   2. Thermal Design Power (TDP): Balances performance and heat generation.
2. GPUs
   1. Power Efficiency in Parallel Tasks: Spread workload across many cores, reducing power per task.
   2. Advanced Power Management: Lower clock speeds or power down idle cores when full performance is not needed.
   3. Can get very hot under load (requires cooling)

CPUs and GPUs Working Together

Task Division

1. CPUs
   1. Handle sequential and control-intensive tasks, such as operating system management and input/output processing.
   2. The CPU is the coordinator, it plans, controls, and sets up tasks.
2. GPUs
   1. Offload parallelizable, data-intensive tasks, such as graphics rendering and machine learning.
   2. The GPU is the worker swarm, once given a task, it runs massive numbers of operations simultaneously.

Data Sharing

1. Memory Transfer
   1. Data is often transferred from the CPU's memory to the GPU's memory via the PCIe bus.
   2. Some systems use unified memory, allowing both the CPU and GPU to access the same memory space.
   3. This is common in:
      1. Gaming (CPU handles AI and logic, GPU draws the world)
      2. AI training (CPU handles dataset preparation, GPU trains the model)
      3. Video rendering (CPU handles timelines, GPU renders frames)

Coordinating Execution

1. Programming Languages
   1. CUDA (for Nvidia GPUs) and OpenCL are used to manage task division and synchronization.
2. Synchronization
   1. Barriers and events ensure that tasks are executed in the correct order and that data is available when needed.

Reflection