GPU Passthrough
GPU Passthrough (also known as GPU Virtualization or vGPU) is a virtualization technique that allows a physical GPU to be assigned directly to a Virtual Machine (VM) or container, bypassing the host OS’s graphics stack. This enables near-native performance for graphics-intensive tasks, AI/ML workloads, and rendering within isolated environments.
Core Mechanisms
- IOMMU/VT-d/AMD-Vi: Hardware support required to isolate device memory and DMA transactions.
- VFIO (Virtual Function I/O): Linux kernel subsystem used to bind devices to userspace drivers, preventing the host from claiming the GPU.
- SR-IOV (Single Root I/O Virtualization): Allows a single physical GPU to appear as multiple virtual GPUs (vGPUs), enabling sharing among multiple VMs.
- MDEV (Mediated Devices): Kernel framework for creating mediated devices, often used in conjunction with NVIDIA vGPU or AMD MxGPU.
Use Cases
- AI/Deep Learning: Running PyTorch or TensorFlow models in isolated docker containers or VMs with direct hardware access.
- Cloud Gaming: Streaming high-fidelity graphics from remote servers to clients.
- Development Environments: Providing developers with dedicated GPU resources for CUDA/OpenCL development without affecting the host system.
Integration with WSL and Containers
Recent advancements have blurred the lines between traditional VM passthrough and containerized GPU access, particularly on Windows hosts.
- WSL 2 GPU Support: WSL 2 now supports direct GPU access for Linux distributions, allowing CUDA and ROCm workloads to run without full VM overhead.
- Native WSL Containers: Microsoft has introduced native containerization within WSL, reducing dependency on external Docker daemons for Linux container management.
- See: WSL Containers: Native Linux Containerization, Docker Hub Integration, GPU Passthrough
- This technology enables native Linux containerization with integrated Docker Hub access and streamlined GPU passthrough capabilities directly within the WSL environment.
Configuration Considerations
- Host OS: Windows (Hyper-V), Linux (KVM/QEMU), or ESXi.
- Guest OS: Typically Linux for AI/ML workloads; Windows for gaming/rendering.
- Driver Management: Requires careful handling of host vs. guest drivers to avoid conflicts. In WSL scenarios, the host Windows driver often manages the hardware while the guest Linux environment accesses it via virtualized interfaces.