vLLM

Deploying Multiple LoRA Adapters on a Base Model with vLLM

by
Deploying Multiple LoRA Adapters on a Base Model with vLLM

Source: DeepHub IMBA This article is approximately 2400 words long and is recommended for a 5-minute read. In this article, we will see how to use vLLM with multiple LoRA adapters. We all know that using LoRA adapters can customize large language models (LLMs). The adapters must be loaded on top of the LLM, and … Read more

vLLM Framework Source Code Analysis: Block Allocation and Management

by
vLLM Framework Source Code Analysis: Block Allocation and Management

1. Block Overview A significant innovation of vLLM is the division of the physical layer GPU and CPU available memory into several blocks, which effectively reduces memory fragmentation issues. Specifically, vLLM’s blocks are divided into logical and physical levels, with a mapping relationship between the two. The following diagram explains the relationship between the two … Read more