A1.1.2 Role of a GPU Notes

What is a GPU?

1. GPU stands for Graphics Processing Unit.
2. It is a specialised processor designed to handle graphics rendering and complex parallel tasks.
3. Originally built to improve image and video performance, but now used in areas like AI, scientific computing, and data analysis.

What does the GPU do?

1. Processes visual data, turning code into images you see on a screen.
2. Performs parallel computations, many small tasks at the same time.

Works alongside the CPU to take on specialised work and free up resources.

Understanding GPU Architecture

1. Parallel Processing:
   1. GPUs are designed with thousands of smaller cores, enabling them to perform parallel processing.
   2. This architecture is ideal for tasks that can be broken down into smaller, independent operations.
2. High Throughput:
   1. GPUs are optimized for high throughput, meaning they can process large amounts of data simultaneously.
   2. This is crucial for tasks like graphics rendering and machine learning.
3. Specialized Memory:
   1. GPUs use high-speed memory, such as VRAM (Video RAM), to handle large textures and data sets efficiently.

Real-World Applications of GPUs

1. Graphics Rendering:
   1. GPUs are essential for rendering complex graphics in video games.
   2. This enables high-resolution textures, realistic lighting effects, and smooth frame rates.
2. Machine Learning:
   1. GPUs accelerate the training of neural networks by performing parallel computations on large data sets.
   2. This is vital for applications like image recognition and natural language processing.
3. Scientific Simulations:
   1. In fields like climate modeling and bioinformatics, GPUs speed up simulations by processing large-scale data in parallel.
4. Cryptocurrency mining:
   1. solving hashes using repeated mathematical functions.
5. Video Editing and Graphics Design:
   1. GPUs enable real-time rendering of 3D models and effects, enhancing the workflow of designers and editors.

Why GPUs Are Suited for Complex Computations

1. Parallel Architecture: Unlike CPUs, which have a few powerful cores optimized for sequential processing, GPUs have thousands of smaller cores designed for parallel tasks.
2. SIMD Operations: GPUs excel at Single Instruction, Multiple Data (SIMD) operations, where the same instruction is applied to many data elements at once.
3. High Memory Bandwidth: GPUs are equipped with high-speed memory to support the rapid data transfer required for tasks like graphics rendering and machine learning.
4. Offload work from CPU: Modern software can offload calculations to the GPU (e.g. TensorFlow for AI). Useful in non-visual tasks because of their structure and speed.

The Evolution of GPUs Beyond Graphics

1. Machine Learning: GPUs are now integral to training neural networks, thanks to their ability to perform matrix multiplications and other parallelizable operations efficiently.
2. Cryptocurrency Mining: GPUs are used to solve complex mathematical problems in blockchain networks, leveraging their parallel processing power.
3. Scientific Research: GPUs accelerate simulations in fields like physics and genomics, enabling researchers to process vast amounts of data quickly.

The Future of GPUs

1. AI and Machine Learning: As AI continues to evolve, GPUs will play a critical role in training and deploying complex models.
2. Real-Time Ray Tracing: GPUs are advancing graphics rendering with technologies like ray tracing, which simulates realistic lighting and shadows in real time.
3. Edge Computing: GPUs are being integrated into edge devices, enabling real-time data processing in applications like autonomous vehicles and IoT devices.

VRAM (Video RAM) vs RAM