According to reports from Electronic Enthusiasts (by Huang Jingjing), the Arm Tech Symposia annual technology conference recently held its first domestic stop in Shenzhen. As one of Arm’s most important and largest technical events, the Arm Tech Symposia annual conference focused on the theme “Arm is Building the Future of Computing,” discussing hot topics in industries such as artificial intelligence, machine learning, the Internet of Things, infrastructure, automotive, endpoints, and mobile computing. At this conference, Arm conveyed its latest insights into processor technology and collaborative ecosystems in the server market, introducing related technological products to vigorously support the accelerated construction of infrastructure.
In the AI Era, Custom Chips Transform Server Architecture
The wave of artificial intelligence has emerged, and the integration of AI into infrastructure has raised higher demands for computing power and performance in data centers and servers. Arm is experiencing rapid growth in the server market, and it is also helping partners achieve innovation in server architecture.
Image: Arm’s Senior Vice President and General Manager of Infrastructure Division, Mohamed Awad, interviewed by the media at the conference.Mohamed Awad explained that in traditional server system architecture, memory is on the left, connected to a general-purpose off-the-shelf CPU, which manages multiple accelerators. This traditional architecture was the only available architecture in the market in the past, but the problem with this architecture is that the interface between the general-purpose off-the-shelf CPU and the accelerators directly limits the final performance level of the product, as all accelerators must access additional memory through a CPU, which prevents memory consistency and means that the performance of the accelerators cannot be fully utilized, thus failing to adequately support the demands of the generative AI era.
Traditional system architecture in the infrastructure field.In a new architecture presented by Arm, the number of CPUs has been increased to provide higher performance for the entire system architecture. Specifically, in this modern system architecture, CPUs are customized based on actual needs, and each customized CPU can be connected to a single accelerator, achieving strong memory consistency. “In such an architecture, the design party customizes the CPU based on the user’s actual scenarios and use cases to maximize the efficiency of the accelerators,” said Mohamed Awad. The new architecture design can better support market demands for infrastructure, including generative AI.
Modern system architecture emerging to meet new application demands in the infrastructure field.This architecture emphasizes the customization of CPUs and the enhancement of data transmission performance on the links.Mohamed Awad explained that in past architectures, one CPU needed to correspond to multiple accelerators or multiple GPUs, but this CPU itself was not a customized CPU. This CPU was connected to the accelerators via PCIe interfaces, and its data transmission throughput and link bandwidth were directly limited by how much data could enter the accelerators. If the link is poor, only a small amount of data enters the accelerators, affecting their performance. However, in the new architecture, not only does one CPU correspond to one accelerator, but more importantly, this CPU is a customized CPU. When combined one-to-one with the accelerator, its overall link and data throughput will reach the highest level, improving the utilization of the accelerators themselves.This new architecture is not entirely new, as it is the architecture used by NVIDIA’s GH200 Grace Hopper super chip.
The NVIDIA GH200 Grace Hopper super chip uses Arm Neoverse combined with NVIDIA GPUs to meet AI performance demands.In this architecture, the combination of 72 Arm Neoverse cores and NVIDIA GPUs can enhance AI performance by ten times compared to systems based on x86 architecture.However, there are still many possibilities for future architectures. Mohamed Awad stated that it is still too early to determine whether a structure where one CPU corresponds to one GPU is the future trend. However, given that we are in an era of computing acceleration, in future architectures, there will definitely be an accelerator next to any general-purpose CPU. This means that whether through tightly coupled, moderately coupled, or loosely coupled methods, there will definitely be an accelerator. But what this accelerator looks like and what architecture it adopts is still being explored in the industry.From his words, it can be inferred that the evolution direction of CPU + accelerator is certain. As for how Arm will lead this direction, it is believed that more new technologies and products will emerge.
Simplifying Custom Chip Development: Arm Upgrades Neoverse Computing Subsystem CSS
Currently, Arm primarily focuses on the server market with the Neoverse platform. To simplify customer designs of server chips based on this IP, Arm has recently launched the Neoverse Computing Subsystem (CSS) and a comprehensive design (Arm Total Design) ecosystem project aimed at leveraging industry ecosystem strength to accelerate the development and delivery of custom system-on-chip (SoC) based on the Neoverse Computing Subsystem (CSS).Arm Neoverse CSS is pre-integrated with the Arm Neoverse platform and has completed verification configuration, which can reduce development costs and accelerate the product launch process. The Arm Neoverse CSS N2, developed based on the Arm Neoverse N2 platform, is the first generation product of Arm Neoverse CSS. It is reported that in a partner’s implementation, Arm Neoverse CSS not only reduces SoC design complexity but also saves up to 80 person-years of engineering time.
The Arm Total Design ecosystem project helps partners accelerate chip development based on Neoverse CSS.The Arm Total Design project brings together leading companies in the industry, including ASIC design companies, IP suppliers, EDA tool providers, foundries, and firmware developers, to accelerate and simplify system development based on Neoverse CSS.For example, pre-integrated and verified IP and EDA tools from partners such as Cadence, Rambus, and Synopsys; design services from partners such as ADTechnology, Alphawave Semi, Broadcom, Capgemini, Zhi Yuan Technology, Socionext, and Sondrel; technologies from foundry partners such as Intel Foundry Services (IFS) and TSMC, optimized for leading process nodes and advanced packaging technologies; and commercial software and firmware support for Neoverse CSS from infrastructure firmware providers such as AMI.Bringing so many partners into this ecosystem means that Arm has done a lot of preparatory work for IC design companies, allowing them to focus solely on their product innovation.Mohamed Awad stated that Arm Total Design further enhances the possibilities available to partners in the Neoverse CSS ecosystem, whether they are ASIC design companies, IP suppliers, EDA tool providers, foundries, or firmware developers, Arm can further support and assist their product development.Additionally, Mohamed Awad emphasized that the Arm Total Design ecosystem is non-exclusive and welcomes more relevant industry chain partners to join. Meanwhile, Neoverse CSS is continuously evolving to support emerging chiplet technology.
Microsoft Azure Cobalt 100Recently, Microsoft’s Cobalt 100 CPU was developed based on Neoverse CSS. Additionally, there are Chinese partners developing based on CSS. Many partners are expected to put the first generation CSS designs into production in 2024.
Strengthening Ecosystem: Advancing Server Chips with Domestic Manufacturers
Currently, all major public clouds offer cloud instances based on Neoverse, and the Arm Neoverse platform is experiencing significant growth in the market. Typical partners include large-scale cloud service providers such as Amazon Web Services (AWS), Alibaba, and local startups such as Hongjun Microelectronics, Yuxian Microelectronics, and Cloud Leopard Intelligence.
From large-scale cloud service providers to startups, empowering innovation on the Arm platform.It is reported that Alibaba’s Yitian 710 is a CPU built on the Arm Neoverse platform. Additionally, Hongjun Microelectronics and Yuxian Microelectronics, both local startups from China, are developing server SoC solutions based on the Arm platform. Meanwhile, the local company Cloud Leopard Intelligence is leveraging the Arm Neoverse platform to create DPU solutions for cloud computing and data centers. Lenovo is also deploying the Arm architecture more in the 5G field. Mohamed Awad stated that by partnering with strong local partners, we will further promote the development of Neoverse in China.
Arm Neoverse Computing Subsystem (CSS)Zou Ting, Global Vice President of Arm China, stated that in the Chinese market, Arm Neoverse has many customers, especially in the infrastructure field. Over the past three to four years, the entire Neoverse has developed very strongly in China. At the same time, Arm is actively participating in local ecosystems and open-source software communities, including the Longxin community, to help these communities better integrate into Arm’s global ecosystem.
Conclusion
Innovation in semiconductors often starts at the source, and Arm’s IP and ecosystem often lead the direction of a trend. In the mobile terminal field, Arm is undoubtedly the leader. Earlier this year, mobile SoC manufacturers based on the Armv9 architecture launched all big core chips, pushing high-end smartphones to new heights.Regarding such cluster configurations, Ian Smythe, Arm’s Vice President of Product Marketing, stated that in the current high-end smartphone market, we often see many different ideas. Each smartphone manufacturer selects its CPU cluster configuration based on the user scenarios of their target audience, which is part of Arm’s CPU strategy, bringing diversity and differentiation through flexible configurations of Arm IP.In mobile terminals, Arm continues to empower the realization of AI on mobile devices through its comprehensive computing solutions for mobile platforms. Additionally, in the Internet of Things field, Arm has a rich portfolio of edge AI products. With Arm Cortex-M55, Cortex-M85, and the newly launched Cortex-M52, it continues to provide partners with the various performance needed to meet the growing demand for IoT in the AI era.In the AI era, computing infrastructure is also undergoing transformation. Currently, 70% of machine learning workloads run on Arm platforms. Mohamed Awad stated that the key to AI lies in the close integration of hardware and software, as well as the enhancement of performance and computing capabilities. In the future, AI will exist at the edge, in networks, data endpoints, storage, and servers, integrating into every process. Arm is continuously accelerating and empowering the changes and innovations brought by AI.
Disclaimer: This article is original from Electronic Enthusiasts. Please indicate the source when reprinting. For group discussions, please add WeChat elecfans999. For submission inquiries, please email [email protected].
More Hot Articles to Read
-
Demand Explosion: Computing Power Rental Prices Surge by 200%! Manufacturers are Increasing Investment; What is the Future Trend?
-
Computing Demand Creates Survival Opportunities; HBM Competition Starts from Advanced Packaging
-
BMW and Mercedes-Benz Join Forces! Will Building a Supercharging Network Become a Core Advantage for Future Brands?
-
Text-to-Video Pika 1.0 Goes Viral! Generate Videos with One Sentence; Ordinary People Can Also Be “Directors”.
-
Amazon Launches New Products, Alibaba’s Damo Academy Exits; After a Significant Decline in Financing Scale, Is Quantum Computing Still a Good Business?