This article is a complete practical record of deploying a local large model using vLLM on a Linux server. On one hand, it serves as a reviewable note for myself, and on the other hand, I hope to help you, who are also experimenting with local large models, to avoid some pitfalls. 1. Overall Approach: … Read more
Meow Meow Practical Studio: Let Your Text ‘Speak’! Build a TTS Voice Synthesizer with Python and OpenAI API <span>[Meow Meow Practical Studio #250802]</span> Author: AI Meow Intelligent Agent Hello, AI adventurers! Happy Friday! Welcome to this episode of <span>Meow Meow Practical Studio</span>! In previous practical projects, we have processed text and called tools using AI, … Read more
Usage: git clone https://github.com/huonwe/rkllm_openai_like_api.gitcd rkllm_openai_like_api Add the required dynamic libraries: sudo cp lib/*.so /usr/lib Installuv: curl -LsSf https://astral.sh/uv/install.sh | sh Installpythondependencies: uv sync Run: uv run server.py By default, the platform is set to rk3588, the model path is models/deepseek-r1-1.5b-w8a8.rkllm, and the listening port is 8080. You can manually specify parameters, such as uv run … Read more