Memory Bottleneck

The Memory Bottleneck (or Memory Wall) refers to the growing disparity between the speed of Processors (CPUs/GPUs) and the speed of Main Memory (RAM/VRAM). As compute power increases exponentially, memory bandwidth and latency improvements lag significantly, causing processors to idle while waiting for data.

In Large Language Models (LLMs)

In the context of large-language-model inference, the memory bottleneck is the primary constraint on throughput and latency, particularly during the prefill and decode phases.

Key Constraints

  • KV-Cache Pressure: The Key-Value Cache grows linearly with sequence length. Storing and retrieving these tensors consumes significant Video RAM (VRAM) bandwidth.
  • Memory-Bound Inference: Unlike training, which is often compute-bound, inference (especially decoding) is heavily memory-bound. The GPU spends more time moving weights and activations than performing arithmetic operations.
  • Bandwidth Saturation: High-throughput serving requires maximizing memory bandwidth utilization. Inefficient memory access patterns lead to underutilized Tensor Cores.

Recent Optimizations

Mitigation Strategies

  • Quantization: Reducing precision (e.g., FP16, INT8) to decrease memory footprint and increase effective bandwidth.
  • PagedAttention: Managing KV-Cache memory non-contiguously to reduce fragmentation and improve utilization.
  • Model Parallelism: Distributing model weights across multiple GPUs to bypass single-device memory limits.
  • FlashAttention: Optimizing memory access patterns to minimize HBM reads/writes.

Source Notes