CUTLASS: Fast Linear Algebra in CUDA C++
This article delves into NVIDIA’s open-source CUTLASS library, which is a collection of templates for implementing high-performance General Matrix Multiplication (GEMM) in CUDA C++. The article thoroughly analyzes the hierarchical structure of efficient GEMM computation on modern GPUs, from thread blocks, warps to thread-level blocking strategies, and covers key techniques such as mixed-precision computation, Tensor … Read more