Recently, Arm’s actions against Huawei have caused dissatisfaction among many industry insiders. The timing coincides with Arm’s annual Techday, where they unveil the next generation of GPU and CPU cores. Whether one advocates for utilizing foreign technology to counter foreign threats or adopts a stance of technology without borders, let us focus on this year’s new products: the Cortex-A77 CPU and Mali-G77 GPU. This article will first share insights into the latest Cortex-A77, which employs the Deimos architecture. Arm continues to fulfill its promise, delivering a surprising boost in floating point performance.
This article is translated from Mr. Andrei Frumusanu of AnandTech without authorization, with the intention of breaking the information flow gap between domestic and international sources. If there is any infringement, please delete it.
2018 was an exciting year for Arm’s own CPU designs. Last May, we saw the release of the Cortex-A76, which ultimately appeared in the form of the Kirin 980 and Snapdragon 855 SoCs. The performance of the A76 amazed us, as Arm successfully delivered on all its promises regarding performance, efficiency, and area, providing outstanding SoCs and devices for most flagship products in 2019.
This year’s Techday officially unveiled the next generation IP of A76—the A77. The A77 is a direct evolution of the A76’s main architecture and represents the second actual IP of Arm’s new Austin core family. Today, we will analyze how Arm drives the IPC of its new architecture and how this translates into actual computational performance to meet the upcoming SoC and device demands at the end of 2019 and early 2020.
Cortex-A77 Based on Deimos Architecture
The release of the Cortex-A77 is not surprising, as Arm continues its tradition of updating its CPU and GPU IP each year. In fact, this is not the first time Arm has talked about the A77: last August, Arm revealed the CPU core during the announcement of its 2020 performance roadmap, featuring the 7nm Deimos architecture (see below):
The new Cortex-A77, codenamed “Deimos”, continues from where the Cortex-A76 left off, following Arm’s planned trajectory of achieving a sustained performance CAGR of 20-25% for each generation of the new Austin CPU family.
Before we introduce the new Cortex-A77, we should review the performance improvements of the A76:
The A76 is undoubtedly one of Arm’s most successful licensed IPs, combining a brand new architecture with significant improvements from the 7nm TSMC process node, resulting in the largest performance and efficiency gains in the industry’s history.
Ultimately, both the Kirin 980 and Snapdragon 855 achieved significant leaps compared to their predecessors. Qualcomm announced that the Snapdragon 855’s performance improved by 45% over the previous generation with the A75 processor, marking the largest generational leap in history.
While the A76 performed excellently, it does not mean it had no rivals. Although Samsung’s proprietary architecture with M4 has narrowed the efficiency gap, the Exynos CPU still lags behind the previous generation, even as this difference is magnified by this year’s process node differences (8nm vs. 7nm). Arm’s true competitor is Apple’s CPU design team: the A11 and A12 still maintain an efficiency level that is about two generations ahead of Arm’s architecture.
However, one of Arm’s advantages remains its provision of the best PPA in the industry. Although the performance of the A76 does not match Apple’s, it achieves excellent efficiency with a very small chip area. In fact, this is a conscious design decision by Arm, as power efficiency and area efficiency are top priorities for Arm’s licensing vendors.
Overview of Cortex-A77
The Cortex-A77 is a direct successor to the A76’s architecture, meaning the new core fundamentally retains the previous features. Arm states that the core is built on a design foundation that allows manufacturers to easily upgrade SoCs.
In fact, this means that the A77 is architecturally consistent with its predecessor, still being an ARMv8.2 CPU core, which will pair with the Cortex-A55 small CPU within the dynamic shared unit (DSU) cluster.
As an evolution of the A76, the A77 does not bring many surprises, whether from an architectural perspective or in terms of absolute performance. We also do not expect significant process node improvements in the next generation SoC.
The A77 is expected to still be manufactured on the 7nm process node for most customers, with Arm announcing a similar peak frequency of 3GHz as the A76. Clearly, since the frequency is not expected to change much, the +20% core performance improvement can only be attributed to the changes brought about by the new generation IP architecture, Deimos.
To achieve clock instruction gains, Arm redesigned the microarchitecture, introducing clever new features that overall enhance the CPU’s intellectual property, resulting in a wider and higher performance design.
Cortex-A77: Frontend Bandwidth Improvement
The Cortex-A76 represented a new design in architecture, with Arm injecting years of CPU design knowledge and experience from the ground up. This has enabled the company to design a new core that is forward-thinking in its microarchitecture. The A76 was originally the benchmark for two designs under the Austin family, namely today’s 7nm node Cortex-A77 and next year’s 5nm node “Hercules” design.
The main goal of the new features introduced in the A77 is to improve the architecture’s IPC. This generation of Arm focuses on continuing to provide the best PPA in the industry, which means that designers aim to improve core performance while maintaining the outstanding efficiency and area characteristics of the A76 core.
In terms of frequency capabilities, the new core maintains the same frequency range as the A76, with Arm targeting a peak frequency of 3GHz in optimal implementations.
Arm has made changes to nearly every part of the new architecture. Starting from the front end, we see higher fetch bandwidth, a doubling of the branch predictor functionality, a new macro-operation cache structure acting as an L0 instruction cache, a 50% wider intermediate core decoder, a new integer ALU pipeline, and improved load/store queues and dispatch capabilities.
Diving deeper into the front end, a major change in the branch predictor is its advanced bandwidth, which has increased from 32B/cycle to 64B/cycle. This increase is generally due to the front end being wider and more capable, necessitating an increase in the speed of the branch predictor to keep up with the feeding into the intermediate core. Arm instructions are 32 bits wide (Thumb is 16b), meaning the branch predictor can fetch up to 16 instructions per cycle. This is 2.6 times higher than the decoder width in the intermediate core, and this imbalance allows the front end to catch up quickly when branch bubbles occur in the core.
The design of the branch predictor has also changed, reducing mispredictions and improving its accuracy. While the A76 already had a very large branch target buffer capacity with 6K entries, Arm has increased this by 33% in the new generation design to 8K. It seems Arm has abandoned the hierarchy of the BTB: the A76 had a 16-entry nano BTB and a 64-entry micro BTB—in the A77, it appears to have been replaced by a 64-entry L1 BTB with a one-cycle latency.
Another major feature of the new front end is the introduction of the macro-operation cache structure. For readers familiar with AMD and Intel x86 processor cores, this may sound familiar and is similar to the operation/operation cache structures in those cores, as their functional principles are indeed similar.
In fact, the new macro-operation cache acts as an L0 instruction cache, containing already decoded and fused instructions (macro-operations). In the case of the A77, the structure is 1.5K entries large, which, assuming macro-operations have a similar 32-bit density to Arm instructions, is equivalent to about 48KB.
The characteristic of Arm’s cache implementation is that it is deeply integrated with the intermediate core. After instruction fusion and optimization, it fills the cache in a decoupled manner after the decoding stage. In case of a cache hit, the front end feeds directly from the macro-operation cache to the renaming stage of the intermediate core, reducing the effective pipeline depth of the core by one cycle. This means that the branch misprediction penalty of the core has been reduced from 11 cycles to 10 cycles, even though it has a frequency capability designed for 13 cycles (+1 decode, +1 branch/fetch overlap, +1 dispatch/release overlap). While we currently do not have direct new data for the new core, Arm’s data in this regard is very good, as the misprediction penalties of other cores are significantly worse (Samsung M3, Zen1, Skylake: around 16 cycles).
Arm’s rationale for a 1.5K entry cache size is that they aim for an 85% hit rate for their test suite workloads. The smaller the capacity, the more dramatically the hit rate decreases, while using a larger cache reduces returns. Compared to a 64KB L1 cache, the 1.5K MOP cache occupies about half the area.
The MOP cache also allows for higher access bandwidth for the CPU. This structure can provide information to the renaming stage at a rate of 64B/cycle, again significantly higher than the renaming/scheduling capabilities of the core, and this “fatter” imbalance in front-end bandwidth also allows the core to hide quickly, effectively mitigating branch bubbles and pipeline flushes.
Arm mentioned a bit about “dynamic code optimization”: here the core will rearrange operations to better fit the backend execution pipeline. It is worth noting that the “dynamic” here does not mean it is actually programmable (similar to Nvidia’s Denver code translation); the logic is fixed in the core design.
Finally, reaching the intermediate core, we see a significant increase in core bandwidth. Arm has increased the decoder width from 4 to 6. The increase in width also ensures an increase in the reorder buffer of the core, from 128 entries to 160 entries. It is worth noting that Qualcomm’s Cortex-A76 variant has already seen such changes, although we have never been able to confirm the exact dimensions adopted. Since Arm is still responsible for RTL changes, I wouldn’t be surprised if it is indeed the same 160-entry ROB.
Similarly, as long as there is an MOP cache hit, the intermediate core can bypass its decoding stage, thus reducing one cycle.
Cortex-A77: Increased ALUs and Better Load/Store
After discussing the front end and intermediate core, we turn to the backend of the Cortex-A77 to examine what changes Arm has made to the execution units and data pipeline. On the integer execution side of the core, we saw the addition of a second branch port, accompanied by a doubling of the front-end branch predictor bandwidth.
On the integer execution side of the core, we saw the addition of a second branch port, accompanied by a doubling of the front-end branch predictor bandwidth.
We also see an additional integer ALU addition. This new unit sits between simple single-cycle arithmetic logic units and the existing complex arithmetic logic unit pipeline: it naturally retains the capabilities of single-cycle arithmetic logic unit operations but can also support more complex two-cycle operations (some shift combination instructions, logical instructions, move instructions, test/compare instructions).
Arm states that the increase in new pipelines brings about astonishing performance improvements: as the core becomes wider, the backend may become a bottleneck, which is a case where the execution units need to grow alongside the rest of the core.
Finally, the existing execution pipeline has not seen much change. One latency improvement is the pipelined operation of the integer multiplication unit on the complex arithmetic logic unit, allowing for 2-3 cycle multiplications instead of 4 cycle multiplications.
Interestingly, Arm did not mention the floating point/ASIMD pipeline of the Cortex-A77. Here, the “state-of-the-art” design of the A76 seems sufficient for them to focus on the core of this generation.
As for the load/store units, we still find two units, but Arm has added two additional dedicated storage ports to the units, effectively doubling the bandwidth of the issue. In practice, this means the left/right units are 4-wide, with 2 address generation operations and 2 store data operations.
The issue queue itself has again been unified, with Arm increasing the capacity by 25% to showcase more memory-level parallelism.
To hide the system’s memory latency, data prefetching is very important: reducing cycles by avoiding waiting for data can significantly improve performance. It is not difficult to see that the new prefetcher of the Cortex-A76 (even further) outperforms other CPUs in the industry, and the excellent performance of the A76 prefetcher is a significant advantage for Arm, as it can handle some very complex patterns. In fact, the A76 performs far better than any other architecture tested, which is an impressive achievement.
For the A77, Arm has improved the prefetcher and added a new prefetch engine to further enhance this. Arm is tight-lipped about the details here, but we are promised increased pattern coverage and better prefetch accuracy. One such change is referred to as “increased maximum distance”, meaning the prefetcher will recognize repeating access patterns over larger virtual memory distances.
One new feature in the A77 is the so-called “system-aware prefetching”. Here, Arm attempts to address the issue of having to use a single IP across different system loads; some systems may have better or worse memory characteristics than others, such as latency. To handle these differences between memory subsystems, the new prefetcher will change its behavior and aggressiveness based on the current system’s performance.
One thought I have is that this may mean some interesting performance improvements under certain DVFS conditions—the prefetcher will change its behavior based on the current memory frequency.
Another aspect of this new system awareness is a better understanding of the cache pressure in the DSU L3 cache. In cases where other CPU cores are highly active, the core’s prefetcher will see this and reduce its aggressiveness to avoid unnecessarily thrashing the shared cache, thereby improving overall system performance.
Performance Goals: IPC Increase of 20-35%
In the Cortex-A77, we see some interesting architectural changes that are expected to enhance performance. The question now is, where will the target performance improvements ultimately manifest?
In terms of announced performance improvements, Arm has chosen to continue using SPEC2006, 2017, GeekBench4, and LMBench memory bandwidth. Our focus will be on SPEC2006, as it remains the most relevant benchmark in mobile devices.
Under the SPECint2006 standard, the A77 promises to achieve a 23% IPC improvement, while SPECfp2006 actually shows a performance increase of 35%. The integer workload increase of 23% is more or less in line with our expectations for CPU cores, but I must admit that a 30-35% increase in floating point workloads is quite surprising, especially since we did not see any significant changes in the floating point execution units of the core. One explanation here is that the SPEC FP test suite is more memory-intensive than the integer suite, and the various architectural improvements of the Cortex-A77 will be more pronounced in these workloads.
Last year, I made performance and efficiency predictions for the A76 at two frequency points, and ultimately I was very close to the performance results of the Kirin 980 and Snapdragon 855. For the Cortex-A77, it should be easier to predict, as we will not see major process node changes in the upcoming 7nm SoC.
Based on the theoretically published IPC improvements of the 2.6GHz A77 and the current results of the Kirin 980, I can roughly infer its performance. It is worth noting that while Arm again talks about the 3GHz target frequency for the A77 this year, I do not expect vendors to reach this frequency in the upcoming SoCs; the prediction will still be for a peak frequency of 2.6GHz.
In terms of performance, the integer suite will see some substantial improvements, but the floating point results are much more interesting. If correct, the A77 will surpass the FP performance of Apple’s A11 and deliver a significant generational push, although we do not expect major process node improvements. It is noteworthy that the A77 will have to compete with Apple’s A13 and Samsung’s next-generation M5 core later this year.
Arm promises that the A77’s energy efficiency will remain consistent with the current A76 SoC. Therefore, at peak performance, both CPU cores will use the same energy to complete a set of workloads. However, the increase in performance of the A77 comes with a downside: power consumption increases linearly with the performance metrics. This latter increase in power consumption seems to reach a level where running more than two high-power large cores at peak frequency in mobile SoCs will become problematic. Fortunately, most vendors have transitioned from four full-speed large cores to designs with only one or two high-power large cores in 2+2 or 3+1 configurations.
It is worth noting that despite discussing large cores here, the A77 is said to be only 17% larger than the A76, still much smaller than the next best architecture from competitors.
In Conclusion
Overall, today’s release of the Cortex-A77 does not show as much change as we saw with the A76 last year, nor is it as significant as today’s release of Arm’s new Valhall GPU architecture and G77 IP.
However, what Arm successfully achieves with the A77 is the continuation of their roadmap, which is extremely important in a competitive environment. The A76 delivered on all of Arm’s promises, ultimately becoming a high-performance core while maintaining impressive efficiency and holding a clear density advantage in competition. In this regard, Arm’s main customers remain very focused on having the best PPA in their products, and Arm contributes to this.
The biggest surprise of the A77 is its floating point performance boost of 30-35%, which is much higher than I expected for the core. In the mobile space, web browsing is a critical application that has a large floating point workload, so I look forward to seeing how the A77’s SoCs will perform in the future.
Yet, even in integer workloads, the 20-25% IPC gain is a truly remarkable improvement, and we believe Arm can maintain the efficiency of the A76. Power will rise slightly, but I believe the industry has shown that today’s mobile devices can handle at least two higher power cores, so future SoCs should continue adopting large + medium + small CPU configurations.
Upcoming A77 processors from vendors are still expected to be 7nm—Qualcomm and HiSilicon are two obvious leading customers that will adopt this core, and I expect their timeframes to be similar to the previous generation chipsets. Currently, Arm is delivering on their promise of a 20-25% CAGR every year, and we believe this will continue for the foreseeable future generations.