Hardware Defines Software? Or Software Defines Hardware?

Source: Software-Hardware Integration

Author:Chaobowx

1 Software and Hardware

1.1 Definitions of Software and Hardware

Instructions are the medium between software and hardware, and the complexity of the instructions determines the degree of decoupling between the system’s software and hardware.

Based on the complexity of unit calculations (instructions), processor platforms can be roughly divided into CPU, co-processors, GPU, FPGA, and ASIC. From left to right, the unit calculations become increasingly complex with decreasing flexibility.

If a task runs on a CPU, it is defined as software operation; if a task runs on a co-processor, GPU, FPGA, or ASIC, it is defined as hardware-accelerated operation.

1.2 Definitions of “Hardware Defines Software” and “Software Defines Hardware”

Software definition is a currently very popular topic, such as software-defined networking, software-defined data centers, software-defined cars, etc. The scope of software definition is very broad, involving not only the underlying software and hardware technologies but also system operation management, monitoring, updating, and even some business considerations.

A system is composed of software and hardware. When we say that software defines XX, we are actually referring to the software-defined XX system. Due to the limited length of this article, we will explain the relationship and influence between software and hardware at the lowest level of related technology.

“Hardware defines software” is defined as:

When the main business logic of a system is implemented in hardware, the corresponding software is auxiliary.
The related software is built upon the interfaces provided by the hardware.

Correspondingly, “software defines hardware” is defined as:

When the main business logic of a system is implemented in software; there is no hardware engine in the system, or the hardware engine is software programmable; the hardware engine executes operations according to the logic programmed by the software.
The related hardware is built upon the interfaces provided by the software.

1.3 CPU, Software and Hardware Decoupling

The CPU is the most flexible because the instructions running on the CPU are the most basic arithmetic operations, along with some memory access and control instructions. They are like building blocks, allowing us to freely combine various forms of functionality we desire.

Many people believe that the CPU can automatically execute very complex computer programs, which is the greatest value of the CPU. However, this is not the case; the greatest value of the CPU lies in providing and standardizing a standardized instruction set, allowing software and hardware to decouple:

Hardware engineers do not need to worry about scenarios, focusing instead on rapidly enhancing CPU performance through various “extreme” methods.
Software engineers do not need to consider hardware details, focusing solely on the program itself. With the support of high-level programming languages/compilers, operating systems, and various system frameworks/libraries, a vast software ecosystem is built.

In the desktop and server domains, x86 is the most popular processor architecture. In mobile devices, ARM dominates. Open-source RISC-V aligns with future technological and commercial trends, having already secured a significant position in the MCU domain and is advancing into the desktop and server areas.

The vast software ecosystem is built on specific CPU architectures. However, generally speaking, the CPU is a sufficiently granular instruction set and a general-purpose computing platform, which decouples software and hardware:

On one hand, under specific architectures, each CPU architecture “basically” guarantees forward compatibility, allowing software and hardware to develop independently in specific architectures.
On the other hand, (ideally,) as OS, compilers, and other systems mature, they can ensure that the same high-level language program behaves consistently across different CPU architectures. This allows for the construction of a completely indifferent software ecosystem, independent of specific CPU architecture.

From a long-term development perspective, RISC-V should be a better choice in the future:

Openness. The biggest feature of RISC-V is its open-source instruction set, allowing any manufacturer to design their own RISC-V CPU according to their needs, creating a shared open ecosystem.
Standardization. Standardization is the most critical value. All architectures (x86/ARM/RISC-V) can be considered standard, but due to RISC-V’s openness, its future value in standardization will be significant. As mentioned earlier, under “ideal” conditions, we can seamlessly migrate programs from one platform to another; however, in practice, many commercial software do not provide source code. Moreover, often, some detailed issues may lead to migration failures, posing significant challenges and risks to users. Once the standard RISC-V becomes sufficiently popular, the entire ecosystem built on RISC-V will burst with powerful vitality.
Others. For example, RISC-V has no historical burdens, its instruction set is more efficient, and it has more flexible expansion capabilities (ensuring no fragmentation), etc.

1.4 Definitions of CPU in Software and Hardware

The interaction interface between the CPU and software programs is the instruction set, which consists of the most granular arithmetic operations, allowing for the combination of any desired programs like building blocks.

Whether the CPU is software-defined or hardware-defined depends on different perspectives:

Software and hardware develop in parallel. The CPU, through ISA, perfectly achieves the standardization and decoupling of software and hardware. Therefore, at this point, there is no hardware defining software or software defining hardware, and both can develop rapidly in parallel.
Hardware defines software. The vast software ecosystem built on the CPU can be considered as “hardware defining software”: first, there is CPU hardware, then compilers, followed by OS and applications.
Software defines systems. However, from the software perspective, all system implementations can be achieved through programming, without needing to consider the “slight differences” of the CPU platform, thus it can also be viewed as the system running on CPU being “software defined”.

2 Hardware Defines Software

2.1 Systems Transitioning from Software to Hardware

A system must undergo a process of rapid iteration in the early stages, gradually stabilizing in later stages. Therefore, the choice of processor platform for system operation must have the following patterns:

CPU General Software Platform: When a new application appears, the first implementation is generally software. First, the cost of implementation is lower, allowing for quick realization of ideas; second, the flexibility provided by the CPU enables it to handle almost any task without considering performance.
Co-processor Extended Instruction Acceleration Platform: With technological evolution, performance demands arise. At this point, programming and compilation optimizations can be applied to programs that consume CPU resources.
GPU Vector and Parallel Acceleration Platform: As technology becomes widely applied, and we find more parallelism in algorithms, we can seek specialized processors, such as GPUs, DSPs, NPUs, etc., to handle tasks. Through specific parallel optimizations and complex instruction compilation optimizations like SIMD (Single Instruction, Multiple Data) and MIMD/VLIW (Multiple Instruction, Multiple Data/Very Long Instruction Word), performance can be deeply optimized.
FPGA Hardware Programmable Acceleration Platform: As technology matures and application scales increase, resource consumption also rises. At this point, we should spend more effort refining highly complex algorithms (which can be considered as very complex “instructions”) and implement them through hardware logic, quickly deploying them via FPGA hardware programmability.
ASIC Customized Acceleration Platform: As technology matures and stabilizes further, and application scales become sufficiently large, it becomes necessary to customize and develop ASICs for optimal performance, lowest cost, and minimal power consumption.

When developing for a new field that requires rapid implementation or when the application scenario is uncertain, using CPUs is more suitable as the hardware platform needs sufficient adaptability. When extreme efficiency is needed, and cost and power consumption are sensitive, developing custom ASICs is more appropriate. If a compromise is needed, where some flexibility and guaranteed performance acceleration are required, and the application has sufficient parallelism, then GPUs are more suitable.

A classic example of transitioning from software to hardware is Bitcoin, which rapidly transitioned from CPU to ASIC. The core algorithm of the blockchain technology used in Bitcoin is SHA-256, and its performance comparison across platforms is as follows:

CPU: Initially, miners used CPUs, and a high-end personal computer had a processing speed of about 20MH/s (H/s, Hash per second);
GPU: Later, some people used GPUs to accelerate mining, with SHA-256 being further decomposed into ordinary arithmetic logic operations. A high-end graphics card can achieve a processing speed of up to 200MH/s;
FPGA: Subsequently, custom SHA-256 algorithm hardware logic FPGA acceleration cards emerged for mining, and carefully designed custom circuits in the FPGA can achieve computation speeds of 1GH/s;
ASIC: The ASIC mining chip BM1385 released by Bitmain in 2015 achieved a single chip computing power of 32.5GH/s.

The performance data of CPU, GPU, and FPGA mentioned above comes from “Blockchain: Technology-Driven Finance”, with the data reflecting performance around 2013.

2.2 Hardware Architecture Determines Software Design

2.2.1 ASIC Hardware Definition

SOC integrates CPU, GPU, various acceleration processing engines (ASIC), and interface modules (ASIC).

Smartphones use battery power while providing sufficiently powerful performance. Such demanding application conditions lead smartphone processors to typically opt for highly integrated SOCs. The image above shows Qualcomm’s Snapdragon 810 smartphone SOC chip, which primarily includes:

General CPU: ARM Cortex-A57 and Cortex-A53, mainly used to run Android and other smartphone operating systems and applications;
Specific scenario processors: Adreno 430 GPU, Hexagon DSP, ISP, multimedia processors, etc.; the GPU is mainly used for 3D gaming scenarios, while the DSP is primarily for sensor algorithm processing;
Specific function subsystems: Communication baseband processing supporting 4G LTE, positioning modules such as GPS/Beidou, etc.;
Connection modules such as WiFi, USB, Bluetooth;
Security processing modules;
Other peripheral modules.

Thus, the smartphone system forms the following layered structure:

Application Layer	Application Programs
System Layer	Operating Systems like Android/iOS
Driver Layer	Driver and HAL
Hardware Layer	General CPU; processing engines like GPU, DSP, ISP; subsystems like 4G, GPS; connection modules like WiFi, USB, Bluetooth; security modules; and others.

2.2.2 GPU Hardware Definition

The GPU consists of thousands of CUDA cores. Programming such a large number of CPU cores to leverage their computational power is a significant challenge.

A critical constraint in GPU programming is that the same program must run in the same parallel thread group, known as “lockstep execution.” Even so, the programming difficulty for GPUs remains substantial.

To enhance the usability and speed of GPU programming, the CUDA programming framework emerged.

In 2006, NVIDIA launched CUDA, a general-purpose parallel computing platform and programming model that utilizes the parallel computing engine in NVIDIA GPUs to solve many complex computational problems more efficiently than CPUs. CUDA provides developers with a software environment using C++ as a high-level programming language. It also supports other languages, application programming interfaces, or instruction-based methods, such as FORTRAN, DirectCompute, OpenACC.

CUDA is key to NVIDIA’s success, significantly lowering the barrier for users to engage in parallel programming based on GPUs. On this foundation, powerful development libraries and middleware have been built for different scenarios, gradually establishing a robust GPU + CUDA ecosystem.

2.3 Summary

2.3.1 Implementation Forms of System Business Logic Determine Software and Hardware Definitions

Three typical platforms:

CPU, because it has completely granular instructions, can flexibly organize desired programs.
ASIC, which solidifies the system’s business logic into hardware. Its advantage lies in completely custom ASICs that achieve the minimal transistor resource usage for the ultimate performance. When a system’s functionality is very fixed and does not require frequent updates in the future, ASIC is almost the optimal choice.
GPU, which is between CPU and ASIC, possesses certain performance while maintaining some flexibility.

According to the specific implementation of system business logic:

If a system runs on a CPU, it is said to be fully software-defined. This means that the system’s business logic can be completely defined through software programming.
If a system runs on ASIC, it is said to be fully hardware-defined. This indicates that the entire system logic of the ASIC is hardcoded in hardware, with software only performing simple initialization configurations and operational control.
GPU is positioned between software-defined and hardware-defined.

2.3.2 Software Depends on Hardware Platforms

ASIC implements a subsystem within the larger SOC system, where the business logic of the entire subsystem is primarily completed within the ASIC module hardware (sometimes requiring control software to manage the ASIC module’s operation). Thus, its performance is generally higher than that of CPUs and GPUs.

When designing an ASIC hardware module or accelerator, corresponding drivers need to be provided. If it is to be integrated with OS or other standard interfaces, an additional HAL layer is required to map different interfaces to the OS standard interfaces. Application programs utilize the hardware based on the functionality provided by the ASIC hardware.

In the GPU platform, the hardware GPGPU provides interfaces to CUDA, which in turn provides interfaces to applications. Each generation of specific GPUs offers different access interfaces to the upper layers, and the CUDA framework has specific drivers and HALs that shield the implementation details of different GPUs. Furthermore, to maintain ecosystem compatibility, CUDA ultimately maps to standard libraries. These standard libraries provide standard interfaces to upper-layer CUDA applications.

Even though CUDA maps to standard library API interfaces, CUDA programs still depend on the interfaces provided by CUDA to design application software.

To summarize, software exists based on hardware platforms in two aspects:

Software can only complete system implementation based on the functionality provided by hardware, and the system’s business logic is implemented in hardware;
Software relies on hardware-provided interfaces to access hardware. The nature of the interface provided by hardware dictates the interface used by software, placing the initiative in the hands of hardware.

3 Software Defines Hardware

3.1 Systems Transitioning from Hardware to Software

In the previous section, we mentioned that systems evolve from initial development to gradual stabilization, with the operating platform transitioning from CPUs to ASICs. So, is ASIC the ultimate operating platform for all systems?

The answer is no, primarily for the following reasons:

Software updates occur rapidly. New hot technologies emerge continuously, and the update speed of existing technologies is still increasing. The iteration cycle of hardware is too long to keep up with the pace of software iteration.
Custom ASIC designs implement all functions in hardware. In complex cloud computing, autonomous driving, and other scenarios, chip companies may lack deep understanding of user scenarios, leading to design discrepancies. Additionally, this restricts user initiative, making it challenging to implement desired functionalities on ASIC platforms.
Since ASIC functions are fixed, to adapt to more scenarios and ensure large shipments of chips, there is a necessity to implement a superset of functions. For instance, if an ASIC chip supports ten functions, but actual user scenarios only require 2-3 functions, the ASIC implementation becomes inefficient. Furthermore, these tightly coupled functions increase overall system complexity.

Thus, in many scenarios, based on actual needs, we begin to opt for DSA, FPGA, GPU, or even CPU as processing engines alongside ASIC.

3.2 Software Defined Networking

A typical case is the development of SDN (Software Defined Network). After decades of development, network chips have evolved into complete ASIC implementations, meaning that network devices based on ASIC chips have fixed functions, and users can only use network devices based on the functions implemented by manufacturers.

However, with the development of cloud computing, 4G/5G mobile communication, and others, new network protocols and functionalities continue to emerge, posing challenges to purely ASIC-based network systems.

As shown in the image, the number of IETF (Internet Engineering Task Force) RFCs (Request for Comments) has been explosively increasing, with various new protocols emerging for novel network scenarios. However, traditional network processing chips are closed and specifically designed for certain protocol processing. It is very difficult to add new protocols, and support for new protocols is constrained by different suppliers. Custom network processing chips struggle with insufficient support for new protocols and lack effective flexibility, making it challenging to introduce new functionalities into network systems and limiting customers’ innovative capabilities.

Customers hope to configure and manage networks quickly and easily; they also wish to innovate network protocols rapidly. Thus, the fixed functionality of ASIC increasingly constrains network innovation. Consequently, SDN initiated innovations in two areas:

First, separating the control plane from the data plane, making the control plane programmable. The network control plane is processed separately from the data plane, forming the OpenFlow protocol for a programmable control plane.
Second, further, the network data plane can also be programmed, allowing users to define their own protocols. This led to the emergence of the P4 language and P4 switches.

3.2.1 Software Virtual Switch Running on CPU

OVS (Open Virtual Switch) is an open-source software switch under the Apache 2 license. The goal of OVS is to implement a production-level switch platform, supporting standard management interfaces and providing open forwarding functions for program extensions and controls. OVS is very suitable for use as a virtual switch in VM environments, in addition to exposing standard control and visibility interfaces to the virtual network layer, it is also designed to support distribution across multiple physical servers. OVS supports various Linux-based virtualization platforms, including Xen, KVM, etc.

The latest version of OVS supports the following features:

Standard 802.1Q VLAN model with trunk and access ports;
NIC binding on uplink switches, supporting both LACP and non-LACP;
Enhanced visibility through NetFlow, sFlow(R), and mirroring;
QoS (Quality of Service) configuration and policies;
Geneve, GRE, VxLAN, STT, and LISP tunnels;
802.1ag connection fault management;
OpenFlow 1.0 and numerous extensions;
Transaction configuration database supporting C and Python;
High-performance forwarding module based on the Linux kernel.

As illustrated, OVS can be divided into three layers:

Management layer: ovs-dpctl, ovs-vsctl, ovs-ofctl, ovsdb-tool.
Business logic layer: vswitchd, ovsdb.
Data processing layer: datapath.

3.2.2 Data Plane Programmable Network Switch DSA

The image above shows the pipeline of the PISA (Protocol Independent Switch Architecture), which is a pipeline engine architecture supporting P4 data plane programmable packet processing. It consists of a programmable parser, multi-stage programmable matching actions, and a programmable reverse parser to achieve data plane programming. This allows for easy support of new protocol processing by writing P4 programs and downloading them to the processing pipeline.

When fully programmable pipelines are implemented, supported by the P4 toolchain, custom pipelines can be achieved through P4 programming to support custom protocols.

As shown, the P4-defined parser program will be mapped to the programmable parser, and data, header definitions, tables, and control flows will be mapped to various matching action stages for serial or parallel processing to achieve complete support for various protocol packet processing.

3.3 Software Defined Interface: Virtio

Virtio aims to provide a set of efficient, well-maintained, and standardized Linux drivers to achieve standardized interfaces between virtual machine applications and different Hypervisor-implemented simulated devices. As a quasi-virtualized I/O device interface, Virtio is widely used in cloud computing virtualization scenarios, and to some extent, it has become the de facto standard for I/O device interfaces.

Because software defines the standardized Virtio interface, as shown in the image, offloading virtualization and workload in SmartNIC and DPU is crucially about hardware-accelerating Virtio.

From a virtualization perspective, offloading Virtio can be viewed as transitioning from software to hardware. However, from the hardware interface perspective, transitioning from a completely hardware-defined interface (such as NV’s custom SR-IOV interface) to a software-defined interface (Virtio interface) can be seen as a transition from hardware to software.

3.4 Cross-Platform Software Definition: Intel oneAPI

Intel oneAPI is an open, accessible, and standards-based programming system that supports developers to participate and innovate across various hardware architectures, including CPU, GPU, FPGA, AI accelerators, etc. These processing engines have very different properties, thus used for various processing—oneAPI attempts to simplify these operations by unifying them under the same model.

Even today, developers face a persistent issue with the number of programming environments provided by our increasingly digital world. Different programming environments render strategies like code reuse ineffective, becoming a real barrier for software developers. As part of its software-first strategy, Intel launched oneAPI during the 2019 supercomputing event. This model marks Intel’s ambition to have a unified programming framework as a solution to proprietary programming platforms. oneAPI allows developers to work without the fatigue of using different languages, tools, libraries, and hardware.

Intel oneAPI enables the design of applications that can easily map programs to different processing platforms such as CPU, GPU, FPGA, or AI-DSA/other DSA as needed.

3.5 Extension: Software Defines “Everything”

Software definition is a vast and hot topic. Beyond software-defined networking, there are many other hot areas of software definition:

Software-defined storage is an architecture that separates storage software from hardware. Unlike traditional Network Attached Storage (NAS) or Storage Area Network (SAN) systems, SDS generally operates on industry-standard systems, eliminating software’s dependency on proprietary hardware.
Software-defined data centers abstract, pool, and automate data center infrastructure to achieve Infrastructure as a Service (IaaS). Software-defined infrastructure allows IT administrators to easily configure and manage physical infrastructure through software-defined templates and APIs, defining infrastructure configurations and lifecycle operations, and achieving automation.
Software-defined radio is a wireless communication technology that relies on software-defined wireless communication protocols rather than hardwired implementations. Frequency bands, air interface protocols, and functionalities can be upgraded via software downloads and updates without needing to replace hardware entirely.
Software-defined vehicles implement new in-car experiences and functionalities through software, providing updates and services wirelessly (OTA). This transforms vehicles from highly electromechanical mechanical terminals into intelligent, expandable, and sustainably iterative mobile electronic terminals.

Software-defined storage and software-defined radio primarily belong to the technical domain, while software-defined data centers and software-defined vehicles deepen and expand the concept of software definition, applying it to broader fields.

4 Software and Hardware Define Each Other

4.1 Software Definition Also Faces Challenges

4.1.1 Failure of Moore’s Law Based on CPU

Software defining XX essentially involves transforming the entire system from a hardware implementation to a more software-oriented implementation. This inevitably raises higher performance demands on CPUs.

However, as shown in the image, the performance improvement of CPUs has gradually stagnated, failing to meet the increasing demands for computing power in the digital economy era.

Thus, we must return to “hardware” acceleration.

4.1.2 DSA Only Solves Some Problems

The P4-supported network data plane programmable engine falls within the DSA category, specifically designed for accelerating network packet processing, with performance comparable to ASICs but possessing excellent software programmability.

Standard P4 programs have a P4 front-end compiler that compiles P4 programs into an intermediate state (similar to a Java compiler). The backend compiler, implemented on specific hardware, is responsible for mapping the intermediate state program to specific hardware implementations (similar to a Java Virtual Machine, but P4 is static).

Once the P4 DSA engine is pre-configured with a P4 program, it becomes a network packet processing engine executing specific protocol processing. It then needs to adapt to existing network programs to achieve data plane offload for network tasks.

The entire system stack of P4 differs from CPUs, GPUs, and ASICs in that it first defines a standard P4, and then various manufacturers implement their respective P4 processing engines based on this standard.

Although CPUs currently have three major architectures (x86, ARM, and RISC-V), the ISA defined for each architecture is very clear. Especially within the RISC-V ecosystem, everyone adheres to a consistent ISA, allowing existing programs to run conveniently on different vendors’ RISC-V CPUs.

However, in the DSA field, this consistency is currently lacking.

It can be said that all current DSAs, including P4 DSA, have not achieved software-defined interfaces; the interfaces remain hardware-defined, which can be described as incomplete software-defined XX.

4.2 A More Ideal Cross-Platform Framework

We can enhance the oneAPI model framework, as shown in the image. This way, cross-platform capabilities extend beyond just CPU, GPU, FPGA, and DSA, but also across different vendors’ processors.

4.3 Interfaces Define Each Other in Software and Hardware

Whether it is software defining hardware or hardware defining software, interfaces (here, interfaces are broadly defined; ISA is an interface, and the data access interfaces of I/O devices are also interfaces, as are the access interfaces presented by GPUs and other accelerators) play a crucial role, as they present their functionalities and how to access them to each other.

Let’s summarize the types of interfaces between software and hardware:

First type: Hardware is defined, and software is defined based on hardware. This is usually the case for CPUs, GPUs, ASICs, etc.
Second type: In software-defined scenarios, existing software must adapt to hardware designs. For example, the hardening of Virtio, or compatibility with OVS’s TC-Flower/RTE-flow interfaces.
Third type: Software-defined and hardware-defined interfaces, with an intermediate conversion layer for adaptation. For example, a HAL layer maps hardware drivers to OS; high-level language programs generate binary programs for specific processor architectures through compilers.
Fourth type: Software-defined, and the interfaces spanning software and hardware are also software-defined, but there is a conversion layer in the hardware. For example, the previously mentioned Virtio interface, where its specific implementation in many solutions resides in the embedded software on the hardware side, making the software emulated Virtio a performance bottleneck.
Fifth type: Software-defined, and the interfaces spanning software and hardware are also software-defined, with hardware directly implementing the software-defined interface. Hardware not only needs to implement its programming paradigm but also have the capability to map that programming paradigm to standard interfaces.

4.4 Summary

“Hardware defines software” is a traditional mindset. New technologies are evolving rapidly, and existing technologies are still iterating quickly; however, hardware systems are becoming increasingly complex, and their updates and iterations are slowing down. If a system is primarily hardware-based, with software relying on hardware, it will inevitably limit the rapid development of technology.

“Software defines hardware” represents a trend: to proactively define individual hardware from a system level. However, the description of “software defines hardware” emphasizes the system, which can easily overlook the characteristics of individuals. From a system perspective, one can “see far and wide,” representing macro and holistic thinking, enabling better resource coordination and defining system functionality more effectively. In contrast, from an individual perspective, individuals are the essence, representing the essential characteristics of things. Things are constrained by objective laws and have specific development rules. Moreover, individuals exhibit differentiation; how to better reflect these differentiated advantages is the source of continuous innovation for the system.

“Software and hardware define each other” can better coordinate the relationship between systems and individuals, achieving both the macro coordination of software definition and consideration of the characteristics and advantages of hardware individuals, thus organically combining the two for a more collaborative and superior system.

Content reproduced only represents the author’s views

Does not represent the position of the Institute of Semiconductor, Chinese Academy of Sciences

Editor: Chunxue