GPU virtualization

GPU Virtualization Scheduling in the Era of Large Models: A Deep Game Between Hardware Genes and Software Strategies

by
GPU Virtualization Scheduling in the Era of Large Models: A Deep Game Between Hardware Genes and Software Strategies

1. Hardware Isolation of Nvidia MIG: From Architectural Design to Engineering Compromise MIG implements 7 independent instances in A100/H100, each exclusively occupying32-512 SM units, 1.8-24GB of VRAM, and independent L2 cache slices. Hardware-level isolation is achieved throughinstance ID tags: the instruction queue, register file, and memory controller access of each SM carry the instance ID, … Read more

GPU Virtualization Solutions and Implementation

by
GPU Virtualization Solutions and Implementation

This article summarizes the current implementation of the GPU passthrough solution on the 360 cloud platform and the verification of the container + MIG solution. 1. Background As a key strategic goal of 360 Company, AI large models rely heavily on GPU cards, which are also considered strategic resources. If physical machines are directly allocated … Read more