With the expansion of the artificial intelligence inference market, major global technology companies are reducing their reliance on GPUs (Graphics Processing Units). They are seeking next-generation alternatives to the expensive and heat-generating GPUs. The Neural Processing Unit (NPU) is an emerging alternative, specifically designed for AI inference. This architecture, tailored for AI computation, achieves high efficiency at low power consumption.
According to industry insiders on the 28th, the global AI inference market is expected to grow rapidly. Market research firm Markets and Markets predicts that this year, the inference market size will reach approximately $10.6 billion (about 14.7976 trillion Korean Won), and by 2030, it will grow to about $25.5 billion (approximately 35.598 trillion Korean Won), with an average annual growth rate of around 19%.
Therefore, the inference NPU market is also expected to grow. This is due to the diverse AI applications requiring higher inference throughput, lower latency, and greater energy efficiency, which the inference NPU is best suited to meet.
The Samil PwC Management Research Institute analyzed in a report that “in the semiconductor sector used for AI, the CPU and GPU markets have entered a stage of technological maturity, while the inference-oriented AI semiconductor (NPU) market, centered around optimized low-power, high-efficiency ASICs (Application-Specific Integrated Circuits), is growing.”
Notable companies in the NPU market include Sambanova and Grok, both of which are American startups.
First, Sambanova integrates its dataflow architecture-based NPU with proprietary software, covering the training and inference of large language models (LLMs). By bundling hardware with models and platforms, Sambanova has secured major clients, including the U.S. government and financial institutions. The company is known for building its own ecosystem both within and outside the industry.
Grok mass-produces chips specifically designed for inference. It utilizes proprietary chips and software to achieve real-time inference speeds for millions of tokens, with a business model based on cloud-based “LLM services.” It is considered suitable for high-speed search in large data centers and RAG (Retrieval-Augmented Generation) services. Its revenue comes from cloud inference services rather than hardware sales.
Industry insiders believe that for AI semiconductor companies to participate in global competition, they must focus on two key strategies.
First, they must ensure a significant advantage in energy efficiency. The power consumption and operational costs of data centers are increasingly becoming the biggest obstacles to AI applications, thus efficiency directly translates to competitiveness.
Second, they should target the customization market. It is quite challenging for general-purpose GPUs like those from NVIDIA to dominate all fields. A more realistic strategy is to expand into specific industries by customizing “inference NPUs” for sectors such as telecommunications, public institutions, finance, and defense.