Inference Acceleration

Paper Interpretation | llm.npu: Achieving 43.6× Inference Acceleration on Mobile NPU with Nearly 60% Energy Efficiency Improvement

by
Paper Interpretation | llm.npu: Achieving 43.6× Inference Acceleration on Mobile NPU with Nearly 60% Energy Efficiency Improvement

Note Compilation: Wang Jiabao Paper Title: Fast On-device LLM Inference with NPUs (llm.npu) Paper Link: https://dl.acm.org/doi/abs/10.1145/3669940.3707239 Code Link: https://github.com/UbiquitousLearning/mllm Published Conference: ASPLOS 2025 Introduction The biggest challenge in deploying large language models on mobile and edge devices is the slow inference during the prefill stage, which is also very power-consuming. Even for models with around … Read more

This Week’s Developments in Large Models: Inference Acceleration, Cross-Modal Watermarking, and Self-Evolving Visual Language Navigation Frameworks

by
This Week's Developments in Large Models: Inference Acceleration, Cross-Modal Watermarking, and Self-Evolving Visual Language Navigation Frameworks

Clickthe card below to follow the “Brain Science and Intelligence Public Account“ for the latest achievements in brain science and intelligence, delivered to you promptly. 01 VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning Recently, Vision Language Models (VLMs) have improved performance by increasing the number of visual tokens, but these visual tokens … Read more