Skip to content
The separation of CPU design from manufacturing through a foundry model has provided AMD with a high degree of flexibility. The second and third generation EPYC processors can relatively freely choose different processes to match the specific needs of chip designs, objectively helping AMD to “achieve more with less” and continuously capture market share from Intel.
What Are the Differences Between Arm v9 and v8 Architectures?
Bridging the CPU Architecture Gap from x86 to ARM
However, the beneficiaries of this flexibility are primarily AMD itself. Large-scale users like AWS and Alibaba Cloud are not satisfied with traditional customization that mainly adjusts core counts, operating frequencies, and TDP metrics; they desire more autonomy over CPU design; or emerging CPU suppliers like Ampere need to choose applicable technology routes… Arm is virtually the only answer in the server CPU market.
If TSMC helps solve the manufacturing issues of CPUs, then Arm helps solve the design issues of CPUs.
Cortex Incubates Neoverse
For “3A” clients like Amazon (Annapurna Labs), Alibaba (Pingtouge), and Ampere who have sufficient chip design capabilities, Arm’s Neoverse platform provides the foundation for designing a server CPU, including the microarchitecture of CPU cores and supporting processes.
Arm’s positive offensive in the server CPU market can be traced back to October 2011, when Arm announced the addition of the optional 64-bit architecture (AArch64) in ARMv8-A. A year later, Arm released the microarchitectures Cortex-A53 and Cortex-A57 that implemented the ARMv8-A 64-bit instruction set, and AMD expressed its intention to launch corresponding server products—AMD’s years of experience in the server market was exactly what the Arm camp lacked at that time.
In the following years, chip suppliers like Cavium, Qualcomm, and domestic Huaxintong, as well as large-scale users like Microsoft, actively promoted the entry of 64-bit Arm into the data center market. However, the truly large-scale deployment should start from November 2018, when AWS previewed its first Arm server CPU—Graviton.
Graviton is based on the Cortex-A72 launched in 2015, using a 16nm process, featuring 16 cores and 16 threads. Compared to x86 server CPUs of the same period, it is rather ‘unremarkable’, relying heavily on Amazon’s “homegrown” optimization.
The Cortex-A family is already the most performance-oriented among the Cortex trio, but it is not designed for server platforms, and cannot loosen power consumption constraints to boost performance. Therefore, a month before Graviton was unveiled, Arm launched the Neoverse platform aimed at cloud computing and edge infrastructure, starting with the 16nm A72 and A75, codename Cosmos.
△ Neoverse Scalable Computing Platform
Just four months later, in February 2019, Arm updated the roadmap for the Neoverse platform and launched the 7nm Neoverse N1, which offers over 30% performance improvement compared to previous targets.
The Neoverse N1, codenamed Ares, is based on the Cortex-A76 launched in 2018, both having the same pipeline structure, featuring an 11-stage short pipeline design, with a front end that is 4-wide read/decode. Arm refers to it as an “accordion” pipeline because, depending on instruction length, it can overlap the second prediction stage with the first fetch stage in latency-sensitive scenarios, overlap the scheduling stage with the first release stage, and reduce the pipeline length to 9 stages. The L2 Cache also adds an optional 1MiB capacity, which is double that of A76.
△ 4 vCPU Configuration, Neoverse N1’s Integer Performance Improvement Relative to Cortex-A72
Compared to the previous generation A72 platform, the Neoverse N1 platform brought significant performance improvements, with many projects achieving double results, especially in iconic machine learning projects, where results are close to five times that of the previous generation. Although A72 is somewhat outdated, such a performance gap also indicates that Neoverse N1 indeed has made a qualitative leap.
Graviton2 and Altra Series
The Neoverse N1 platform had a huge impact on the data center market, as everyone saw its immense potential and value, as well as the opportunities behind it. If the previous A72 was just starting to make a mark in the data center market, then Neoverse N1 made more people believe that Arm has the capability to carve out a share in this field.
Two 7nm CPUs, one from a cloud service provider and the other from an independent CPU supplier, are both based on Neoverse N1.
In November 2019, AWS officially announced the Graviton2 processor:
-
The core count surged to 64, four times that of the first generation;
-
The number of transistors increased sixfold, reaching 30 billion;
-
64MiB L2 Cache, eight times that of the first generation;
-
DDR4-3200 memory interface (frequency) is double that of the first generation;
-
Operating frequency of 2.5GHz, slightly higher than the first generation’s 2.3GHz.
△ In the new EC2 instances added by AWS in 2020, Graviton2 accounted for a significant share, and the ratio of Intel and AMD is also noteworthy.
The EC2 (Elastic Compute Cloud) instance types based on Graviton2 rapidly increased, including but not limited to general-purpose (M6g, T4g), compute-optimized (C6g), and memory-optimized (R6g, X2gd). The regions and numbers deployed have steadily grown since mid-2020—statistics show that 49% of the new AWS EC2 instances in 2020 were based on AWS Graviton2.
Announced in November 2011, Armv8 brought Arm into the 64-bit era. With the joint efforts of Arm and ecosystem partners, after several product iterations, the Arm camp has established a foothold in the server market over the past decade.
At the end of March 2021, Armv9 was released, focusing on upgrades in security, machine learning (ML), and digital signal processing (DSP) capabilities based on Armv8.
Among the three major features brought by the new architecture, machine learning may be the most familiar and concerning topic for the public. With the rise of heterogeneous applications, artificial intelligence (AI) technologies represented by machine learning have penetrated various aspects of our lives, whether in backend data centers or on terminals and edge sides, machine learning has great potential.
To better enhance the computing power required for AI and DSP, ARMv9 upgraded the previously supported Scalable Vector Extension (SVE) to version 2.0. This technology can improve the performance of machine learning and digital signal processing applications, aiding the processing of workloads related to 5G systems, VR/AR, machine learning, and more.
SVE2 provides adjustable vector sizes ranging from 128 bits to 2048 bits, allowing for variable granularity of 128 bits without being affected by hardware platforms. This means that software developers will only need to compile their code once, and it can be applied to Armv9 and subsequent products, achieving “write once, run anywhere”. Similarly, the same code will be able to run on more conservative designs with lower hardware execution width capabilities, which is crucial for Arm designs from IoT and mobile to data center CPUs.
SVE2 extensions also add the capability to compress and decompress code and data within CPU cores, as moving data in and out of chips consumes a lot of power; maximizing the use of on-chip data can reduce this data movement, thereby lowering energy consumption.
More importantly, the Confidential Compute Architecture (CCA) is the most significant content in this version update by Arm. Security issues have become increasingly severe in recent years, with ransomware and hacker attacks occurring constantly. In the face of the ever-evolving network attack issues, it requires efforts from network service providers and software companies, as well as hardware infrastructure providers like Arm to block potential vulnerabilities from the source, leading to the emergence of CCA. This is a security defense capability based on architecture, creating a hardware-based secure runtime environment to execute computations, protecting certain codes and data from access or modification, and even shielding them from privileged software.
△ Arm Confidential Compute Architecture (left), Memory Tagging Extension Technology Introduced by Android 11 and OpenSUSE (right)
To this end, CCA introduces the concept of dynamically creating confidential realms—a secure containerized execution environment that supports secure data operations, isolating data from the hypervisor or operating system. The management functions of the hypervisor will be handled by a “realms manager”, while the hypervisor itself is only responsible for scheduling and resource allocation. The advantage of using “realms” is that it greatly reduces the trust chain of running a given application on a device, making the operating system largely transparent to security issues, and allowing critical task applications requiring supervisory control to run on any device.
In practical applications, memory is a very easy target for attacks, and memory security has always been a focus in the industry. How to detect issues before memory security vulnerabilities are exploited is an important step in enhancing global software security. Therefore, the “Memory Tagging Extension” (MTE) technology continuously developed by Arm in collaboration with Google has also become a component of Armv9, capable of identifying spatial and temporal safety issues in memory within software, linking pointers pointing to memory with tags, and checking the correctness of this tag when using the pointer. If access exceeds the bounds, the tag check will fail, allowing for immediate detection and blocking of memory security vulnerabilities.
What Are the Differences Between Arm v9 and v8 Architectures?
Over the past few years, Arm has improved the ISA and made various updates and expansions to the architecture. Some of these may be very important, while others may be fleeting.
Recently, as part of Arm’s Vision Day event, the company officially released the first details of its next-generation Armv9 architecture, laying the foundation for Arm to become the next 300 billion chip computing platform in the next decade.
A major question readers may ask is, what are the differences between Armv9 and Armv8 that allow such a significant upgrade in the architecture? Indeed, from a purely ISA perspective, v9 may not achieve a fundamental leap compared to v8 as v8 introduced AArch64, a completely different execution mode and instruction set that has broader microarchitecture branches compared to AArch32, such as extended registers, 64-bit virtual address space, and more improvements.
Armv9 continues to use AArch64 as the baseline instruction set, but adds some very important extensions to its functionality to ensure the increase in architecture numbering, and allows Arm to gain new capabilities for AArch64 software re-benchmarking for v9 while maintaining the extensions gained on v8 over the years.
Arm believes the new architecture Armv9 has three main pillars: security, AI, and improved vector and DSP capabilities. For v9, security is a very important theme, and we will delve into the new extensions and features in detail, but first, the DSP and AI capabilities should be straightforward.
The most notable new feature promised by the compatible CPUs of Armv9 is SVE2, the successor to NEON, which developers and users can see immediately.
The Scalable Vector Extension (SVE) first appeared in 2016 and was first implemented in Fujitsu’s A64FX CPU core, which supports Japan’s top supercomputer Fugaku. The issue with SVE is that the range of the first iteration of the new variable-length vector-length SIMD instruction set is quite limited and is more targeted at HPC workloads, lacking many more general-purpose instructions still covered by NEON.
SVE2 was released in April 2019, aimed at addressing this issue by supplementing the new scalable SIMD instruction set with the necessary instructions to serve workloads like DSP that still use NEON.
In addition to the various modern SIMD features added, the strengths of SVE and SVE2 also lie in their variable vector sizes, ranging from 128 bits to 2048 bits, allowing for variable granularity of 128 bits regardless of the hardware they run on. From the perspective of vector processing and programming, this means that software developers will only need to compile their code once, and if a future CPU features native 512b SIMD execution pipelines, that code will be able to fully utilize the entire width of the unit. Similarly, the same code will be able to run on more conservative designs with lower hardware execution width capabilities, which is crucial for Arm designs ranging from IoT and mobile to data center CPUs. While retaining the 32b encoding space of the Arm architecture, it can accomplish all of this. However, architectures like x86 require new instructions and extensions to be added based on vector sizes.
Machine learning is also seen as an important component of Armv9, as Arm believes that an increasing number of ML workloads will become commonplace in the coming years, especially in scenarios with critical performance or power efficiency requirements. This makes running ML workloads on dedicated accelerators a long-term need, while we will continue to run smaller ML workloads on CPUs.
Matrix multiplication instructions are key here, representing an important step towards broader adoption of v9 CPUs as a fundamental capability within the ecosystem.
Generally, I believe SVE2 may be the most important factor ensuring the upgrade to v9, as it is a more definitive ISA feature that distinguishes it from v8 CPUs in everyday use, and it ensures that the software ecosystem can operate normally, which differs from the existing v8 stack. For Arm in the server domain, this has become quite a significant issue, as the software ecosystem is still based on the v8.0 software package foundation, which unfortunately lacks the most crucial v8.1 large system extensions.
Advancing the entire software ecosystem and assuming that new v9 hardware possesses new architectural extension capabilities will help drive progress and may resolve certain current situations.
However, v9 involves not only SVE2 and new instructions, but it also places a strong emphasis on security, and we will see some fundamental changes in security.
Introducing Confidential Compute Architecture
In recent years, security and hardware security vulnerabilities have become top priorities in the chip industry, with the emergence of vulnerabilities like Spectre and Meltdown and all their peer side-channel attacks indicating that rethinking how to ensure security has become a fundamental requirement. Arm’s approach to solving this overarching issue is to redesign how secure applications work through the introduction of the Arm Confidential Compute Architecture (CCA).
Before proceeding, I want to highlight that today’s disclosure is merely a high-level explanation of how the new CCA operates, and Arm states that more details about the exact workings of the new security mechanism will be revealed later this summer.
The goal of CCA is to gain greater benefits from the current software stack situation, where applications running on devices must inherently trust the operating system and hypervisor they run on. The traditional security model is based on the fact that higher privilege software layers are allowed to view lower layer executions; however, this can become problematic if the operating system or hypervisor is compromised in any way.
CCA introduces the new concept of dynamically creating “realms”, which can be viewed as a secure containerized execution environment that is completely opaque to the OS or hypervisor. The hypervisor will still exist but will only be responsible for scheduling and resource allocation. Meanwhile, the “realm” will be managed by a new entity called the “realm manager”, which is considered a piece of new code, roughly one-tenth the size of the hypervisor.
Applications within the realm will be able to “prove” the realm manager to determine whether it is trustworthy, which is not possible for traditional hypervisors.
Arm has not delved into what exactly causes the isolation between the realm and the non-secure world of the operating system and hypervisor, but it does sound like hardware-supported address spaces that cannot interact with each other.
The advantage of using realms is that it significantly reduces the trust chain of running a given application on a device, and the OS becomes increasingly transparent to security issues. Instead of requiring businesses or enterprises to use dedicated devices with authorized software stacks, critical task applications requiring supervisory control will be able to run on any device.
MTE (Memory Tagging Extensions) is not a new feature of v9 but was introduced with v8.5. MTE, or memory tagging extensions, aims to help address two of the most persistent security issues in software: buffer overflows and use-after-free. These issues have been part of software design for the past 50 years and may take years to identify or resolve. MTE aims to help identify such issues by tagging pointers at allocation and checking them when used.
This is not directly related to v9 but is closely related to the upcoming v9 design’s technical roadmap. Arm also discussed some perspectives on the expected performance of v9 designs over the next two years.
Arm discussed how the mobile market this year improved performance by 2.4 times with devices featuring X1 (here we are only referring to the IPC of ISO process design), which is twice the performance of the Cortex-A73 launched years ago.
Interestingly, Arm also talked about the Neoverse V1 design and how it achieves 2.4 times the performance of A72-like designs, revealing they are looking forward to the first V1 devices being released later this year.
For the next-generation mobile IP cores codenamed “Matterhorn” and “Makalu”, the company disclosed that the expected IPC gain for these two generations combined is 30%, not including any frequency or other performance gains that SoC designers could achieve. This effectively represents a 14% generation increase for these two new designs, and as indicated by the performance curve in the slides, it shows that the pace of improvement is slowing relative to the work Arm has managed over the past few years since A76. However, the company points out that the pace of progress is still far ahead of the industry average. But the company also admits that it has been dragged down by some industry participants.
Arm continues to view CPUs as the most universal computing modules of the future. Although dedicated accelerators or GPUs will have a place, they struggle to address some important issues, such as programmability, security, universality (the ability to run them on any device), and the ability to work reliably.
Currently, the computing ecosystem is extremely fragmented in how it operates, with not only different types of devices but also different suppliers and operating systems.
SVE2 and matrix multiplication can greatly simplify the software ecosystem and allow computing workloads to move forward in a more unified manner, which will be able to run on any device in the future.
Finally, Arm also shared new information about the future of Mali GPUs, revealing that the company is developing new technologies like VRS, especially Ray Tracing. This is quite surprising and suggests that the desktop and console ecosystems driven by AMD and Nvidia’s introduction of RT are also expected to push the mobile GPU ecosystem towards RT.
Armv9 Design Set to Debut in Early 2022
Today’s announcement comes in a very high-level form, and we hope Arm will discuss various details and new features of Armv9 in more depth during the company’s usual annual technology disclosures in the coming months, such as CCA.
Overall, Armv9 seems to combine a more fundamental ISA shift (which can be seen as SVE2) with an overall re-benchmarking of the software ecosystem to summarize the last decade of v8 extensions and lay the groundwork for the next decade of the Arm architecture.
Arm has already discussed Neoverse V1 and N2 in the second half of last year, and I sincerely hope that N2 will at least be designed and released based on v9. Arm further revealed that more CPU designs based on Armv9 (possibly the follow-up products of mobile Cortex-A78 and X1) will be launched this year, and the new CPUs are likely to be adopted by the usual SoC suppliers and are expected to appear in commercial devices in early 2022.
Source:E-Enterprise Research Institute, Semiconductor Industry Observation
Disclaimer:This account focuses on sharing related technologies, and the content and views do not represent the position of this account. All traceable content is duly cited. If there are copyright issues with the published articles, please leave a message to contact for deletion, thank you.
For more architecture-related technology knowledge summaries, please refer to the related e-books of “Architect’s Full Store Technical Data Package” (details of the 41 books technical data package summary can be obtained by “reading the original text“.
Please search for “AI_Architect” or “Scan Code” to follow the public account to keep up with in-depth technical sharing. Click “Read the Original Text” to get more original technical content.
