This article is a complete practical record of deploying a local large model using vLLM on a Linux server. On one hand, it serves as a reviewable note for myself, and on the other hand, I hope to help you, who are also experimenting with local large models, to avoid some pitfalls. 1. Overall Approach: … Read more
1. What is an NPU? NPU (Neural Processing Unit) is a processor specifically designed for inference and training of artificial neural networks, featuring high parallelism, low power consumption, and low latency. It is a key hardware component in scenarios such as edge AI, autonomous driving, smart cameras, and voice recognition. 2. Comparison of NPU with … Read more
Andrew Su is a senior technical expert in the AI product direction at Renesas Electronics China, responsible for product definition and operation of AI-related products and technical solutions for the Chinese market. What is RUHMI? With the launch of the new high-performance RA8P1 microcontroller integrated with the Ethos-U55 NPU and Arm Cortex-M85 CPU core, Renesas … Read more
The implementation principle of Edge AI is to deploy artificial intelligence algorithms and models on edge devices close to the data source, enabling these devices to process, analyze, and make decisions locally without the need to transmit data to remote cloud servers. The goal of Edge AI implementation is to bring AI capabilities down to … Read more
# PaddleOCRv5 C++ Inference Project Tutorial ## 1. Project Overview This project is based on PaddleOCR v5, utilizing Paddle Inference and OpenCV to implement text detection and recognition in both Chinese and English, supporting orientation classification, suitable for secondary development and deployment on the Windows platform. ## 2. Environment Preparation – Operating System: Windows 10/11 … Read more
Using the NPU to run inference on large models can more effectively plan resource allocation, achieving more efficient applications. 1 Preparation Before Deployment To deploy large language models using the NPU on RK3588, the following files need to be prepared in advance, which include the corresponding files for Rockchip’s NPU and the DeepSeek inference model … Read more
DeepSeek is here! Recently, the rising star in the AI field, DeepSeek (Chinese name: 深度求索), has quickly gained popularity online due to its low-cost and high-performance AI model. Its core is a powerful language model capable of understanding natural language and generating high-quality text. Additionally, DeepSeek is freely available to developers worldwide, accelerating the spread … Read more