FlashAttention

Traditional Attention, Multi-Query Attention, and FlashAttention Overview

by
Traditional Attention, Multi-Query Attention, and FlashAttention Overview

Issues with Traditional Attention Problems with Traditional Attention There are constraints on context length. It is slow and consumes a lot of memory. Optimization Directions for Attention Increase context length. Speed up and reduce memory usage. Variants of Attention Sparse Attention: Introduces sparse biases to reduce complexity. Linearized Attention: Unrolls the attention matrix and kernel … Read more

Efficient LLM Inference with Block Sparse Attention

by
Efficient LLM Inference with Block Sparse Attention

Click the card below to follow the “LiteAI” public account Hi, everyone, I am Lite. A while ago, I shared the Efficient Large Model Full-Stack Technology from Part 1 to Part 19, which includes content on large model quantization and fine-tuning, efficient LLM inference, quantum computing, generative AI acceleration, etc. The content links are as … Read more