Processor design has at least three architectural layers, each playing a crucial role. Optimizing any system is a multi-layered problem, but when it comes to processors, there are at least three levels to consider. Architects must be able to think across these boundaries, as the role of each layer must be understood and balanced. The first layer of potential optimization is at the system level. For instance, how does data enter and exit the processing core? The next layer is the architecture of the processing core itself. This might involve using existing processors or adding instructions to scalable cores. The final level of optimization is at the microarchitecture level, where the implementation of pipelines is defined. When a system is created from predefined components, the freedom of choice is limited. However, when deploying custom chips, it can easily be overwhelmed by a large amount of flexibility, leading to an understanding of details before the right time. A top-down discipline must be maintained. Siemens EDA technology researcher Gajinder Panesar states, “Our task is to optimize the execution of one or a group of tasks. This is not about the processor. It’s about the process you are trying to optimize, so you must consider the system that implements the tasks. Before you start thinking about selecting a CPU and deciding whether to customize it, how to construct and partition the system design is a key question.” Task partitioning and system architecture present a challenge. Once a processor is selected, it needs to be optimized and customized for the given task or group of tasks. Processors can be optimized for various reasons, including a combination of throughput, latency, and power. Synopsys’ ASIP tool senior director Gert Goossens says, “The specialization of processors is essentially the introduction of parallelism. It can refer to instruction-level parallelism or data-level parallelism. Processing vectors can be task-level parallelism or deploying multi-core architectures. The second trick is specialization. You want to ensure that the functional units of the processor can execute tasks in one cycle, while traditional processors require hundreds of cycles.” Performance and power are often tightly coupled. George Wall, product marketing director of Cadence’s Tensilica Extensa processor IP group, states, “A typical example is accelerating certain tasks by increasing computational resources. The goal is often to achieve lower energy execution. Energy is power multiplied by time. If you customize a processor, there will usually be a slight increase in power, expected to be offset by a significant reduction in cycle time, providing a net win in energy overall.”
System-Level Optimization
Ideally, the processor is always busy doing productive work and never has to wait for data to be available. However, this situation rarely occurs. Every cycle, if the processor is idle or executing a speculative task that is not used, it leads to wasted time and power. Michael Frank, a researcher and system architect at Arteris IP, states, “You have a processor, and they need someone to feed them. They need cache. They need peripherals like interrupt controllers to provide vectors. When you encounter an interrupt, they need SMMU for virtualization. A complete ecosystem needs to surround the processor. It won’t work on its own. Then you need to provide cache infrastructure for the processor, as your processor has become very fast. You can’t let them talk to remote memory.” In multi-processor systems, each processor needs to be carefully orchestrated. “How will this accelerator be managed?” asks Sharad Chole, chief scientist at Expedera. “Where will workload orchestration happen? What kind of bandwidth does the accelerator need? How much DDR access or shared memory is required at the workload level? When we discuss solutions with customers, this is often a hardware/software co-design issue. You need to analyze the workload, and you need to define the impact of the workload on the entire SoC. It’s important not to miss this. We are not optimizing a single CPU core. If you optimize a CPU core in isolation, you end up with an uncertain multi-core architecture, where performance depends on the compiler. How good is the compiler?”
Key Elements of Processor Optimization
Optimization occurs at multiple levels. Simon Davidmann, founder and CEO of Imperas Software, states, “The system needs to be analyzed to ensure communication works correctly and to identify any bottlenecks. Many systems adopt complex communication, which can be synchronous or asynchronous. It’s like the challenge of floor planning, ensuring data does not bottleneck as it flows through the system.” Often, this communication involves software. “You must consider the firmware and software running on the device, as this determines the product’s performance,” says Panesar of Siemens. “We can illustrate this by thinking about various types of visibility. A CPU-centric view assumes tracking instruction execution is enough, but if you can’t see what’s happening elsewhere in the system, it’s a very rigid approach. Is the size and configuration of the on-chip network (NoC) correct? Does memory bandwidth and latency affect performance? Without these system-level factors, all the CPU customization in the world won’t yield a successful product.” Some of these decisions are influenced by the type of processor core. Kristof Beets, vice president of technology innovation at Imagination Technologies, states, “CPUs and GPUs can perform random data access, but the difference with AI processing cores is that they are designed to execute a limited set of algorithms using very specific and well-known data flows. This enables stream processing and allows for smaller logic and less local cache. The supported operations and data flows are more limited than those supported by GPUs, especially compared to CPUs.” For many processors, bus interfaces may be viewed as a limiting factor. Wall of Cadence states, “In creating efficient designs, in some cases, the interface is as important as or even more important than the actual ISA. Traditional processors typically have a bus interface, such as an AMBA protocol-based interface, interfacing with other elements on the SoC, connecting to main memory storage, and connecting to various I/O devices. Depending on the number of elements competing for these devices, such interfaces are limited. Part of the current embedded design process is to consider whether it makes sense to interface the processor to specific devices via the system bus. Or, is there an alternative way to connect it? This is another way to extend the processor—creating interfaces like queue interfaces or lookup interfaces that allow other devices to connect more directly with the processor.” Ignoring these types of issues can lead to suboptimal solutions. Manuel Uhm, marketing director at AMD Versal, states, “Data movement is crucial. I/O and memory bottlenecks must be thoroughly considered. We have actually doubled the on-board tightly coupled memory—essentially the program memory—connected to each core. We have also added something called memtiles, which essentially builds buffers that support these cores. They are not part of the core itself, unlike tightly coupled program memory, but are their own tiles that support all of this. We have learned that learning is not just about computation. It’s about how to move data, how to manage memory, and how to bring all the data in. I/O is an important part of this issue.” Some recent workloads are even unsuitable for Von Neumann-like processor architectures. Chole from Expedera states, “We didn’t assume we needed a processor at first. We didn’t start from a Von Neumann architecture. We started from the building blocks of operations used in neural networks—such as matrix multiplication, convolution, activation functions, pooling layers. We began by defining how to do it best. We studied the costs of doing so and how to ensure that all operands are available when computation is needed. Then we built an architecture that is completely deterministic without any back pressure.”
Processor Architecture Optimization
Until recently, opportunities for processor architecture optimization were limited unless you built a fully custom processor. “Processor architecture has two parts,” says Zdeněk Přikryl, CTO of Codasip. First is the instruction set architecture (ISA), and second is the microarchitecture. This is the implementation of the architecture. In proprietary ISA cases, changes to the ISA are rarely allowed, so only microarchitecture changes can be made. You can think of these changes as evolutionary rather than revolutionary. On the other hand, if you start from an open ISA, it gives you a good starting point to focus on innovation and critical differentiation. This becomes your secret weapon. You can add new instructions to help improve performance, reduce memory footprint, etc. In this case, it can be described as revolutionary rather than evolutionary.” Some ISAs contain a lot of flexibility. “There is configurability and scalability,” says Rich Collins, product marketing director at Synopsys. “Many standard implementations can be configured based on bus width and memory size. All these things are configurable without requiring any customization. There may be predefined forms of extensions, like providing a set of condition codes, a set of auxiliary registers, or an extended instruction set, or even additional hardware accelerators. You don’t have to just add custom instructions. If you have your own custom secret accelerator, then you can connect it to the processor.” Using benchmark processors or fixed architecture processors does have some benefits. Goossens from Synopsys states, “People can reuse existing benchmark processor ecosystems like Arm, RISC-V, or ARC. This ecosystem can extend to peripherals, devices connected to the processor, and existing software libraries. In fact, it’s important that you can reuse these elements, which is why people may prefer to start from a RISC-V baseline or an ARC baseline.” While some in the industry lament the lack of processor architects, this is only a temporary issue. He says, “With the rise of RISC-V, there are already more computer engineering courses covering processor design, so in the coming years, the industry will see more people entering. I don’t think there will be a lack of people who know how to make processors. It takes a few people as crystallization points who gather around a team to start building processors. Building a processor doesn’t require special skills. It’s just another IP, but making a good processor is not easy.” It indeed requires a good flow to analyze architectural changes. Goossens states, “You need software tools, especially compilers. You don’t want to write extensions in low-level assembly code and then inline assemble them and link them with compiler-generated code. The compiler must be able to take advantage of all the specialization you add. Then you can get immediate feedback on architecture quality by using real application code. If you don’t have these conditions, then you can only guess at how to do extensions.” There are two approaches to solving this problem. Přikryl from Codasip says, “The traditional approach is to have one team of software engineers responsible for the software development kit (SDK), then a team responsible for RTL, and another team responsible for verification. There’s nothing wrong with this approach, but it requires many engineers, and you need to synchronize between them to ensure they are consistent. The other approach is to use automated tools. Describe the processor in a high-level language, including its ISA and microarchitecture. We call this CodAL (a C-like language) at Codasip. Then generate the SDK, RTL, and verification tools.”
Microarchitecture Optimization
Once the processor architecture is defined, it must be implemented. “Microarchitecture defines how the instruction set is executed,” says Frank of Arteris. “When people stick to a consensus instruction set architecture, you are squeezing the lemon, the microarchitecture, to provide more performance.” The implementation of the processor is often viewed as a standard RTL design process. However, this does not mean it is trivial in any way, especially if it leverages out-of-order parallelism, speculation, branch predictors, or many other techniques.
Conclusion
The design, creation, and optimization of processors involve many aspects that can execute one or a group of defined tasks, and these aspects are closely interconnected. Focusing too much on one aspect while neglecting others will cause problems. As with most systems, starting from the top yields the most benefits. “In hindsight, it was 2020,” says Uhm of AMD. “If you don’t put something on the market, you can’t learn from it and get better. Many AI companies today are like that. All those who ran out of money after the first device came out will go bankrupt. Those who achieve some level of success are developing the next chip, learning from it, and improving on it. You need a continuous feedback loop about what works and what doesn’t.”
About Shenzhen Electronic Chamber of Commerce
The Shenzhen Electronic Chamber of Commerce (SECC) was established in 2002 and is a non-profit organization with legal entity status, voluntarily formed by enterprises and groups engaged in the production, operation, research, teaching, and service of electronic products in Shenzhen. It is a 5A-level social organization in Shenzhen and a nationally, provincially, and municipally recognized “Four Good” chamber of commerce, as well as a public service demonstration platform for small and medium-sized enterprises in Shenzhen.
SECC currently has over 2,000 registered members, and member companies’ products involve the research and production of automation equipment, artificial intelligence, electronic components, automotive electronics, robotics, digital products, home appliances, and the sales of the entire industrial chain elements such as electronic markets, internet trading platforms, innovative research and development centers, testing and certification, and software development.
<< Swipe to view the next image >>
To join, please scan the code
Website: www.seccw.com
Phone: 0755-8367 7342
Email: [email protected]
WeChat ID: szdzsh
● Scan to follow us ●