In recent years, whenever the topics of “AI edge inference”, “industrial vision”, and “local large models” come up, you must have heard the name RK3588. Its reputation is simple: 6TOPS computing power + strong video encoding/decoding + multiple interfaces, which can almost cover all scenarios of edge AI. The Baile Technology BL450 is an industrial-grade … Read more
Arm Technology held a product launch conference in Shanghai, officially introducing the new generation “Zhouyi” X3 NPU. This is the first heavyweight product under its Arm China “All in AI” strategy, targeting four major fields: infrastructure, smart vehicles, mobile terminals, and smart IoT, aiming to provide efficient large model inference and multi-scenario AI acceleration capabilities … Read more
Global generative AI applications are penetrating vertical fields such as industry, healthcare, and finance at an unprecedented speed. Gartner predicts that by 2026, 80% of enterprises will use generative AI, with edge AI deployment exceeding 50%. As the demand for AI applications continues to grow, edge AI will unleash greater productivity for enterprises. Recently, Wangsu … Read more
The development of artificial intelligence applications is accelerating, and the deployment work that developers face is becoming increasingly complex.The endless array of algorithm models, various architectures of AI hardware, different deployment requirements (servers, services, embedded, mobile, etc.), and different operating systems and programming languages pose significant challenges for AI developers in project implementation. To solve … Read more
This article will introduce how to implement streaming output for model inference in the transformers module. The transformers module provides a built-in Streaming method for streaming output during model inference. Additionally, we can use model deployment frameworks such as vLLM and TGI to better support streaming output for model inference. Below, we will detail how … Read more