Three Trends in Embedded Software Development

In February 2024, IDC published data showing that in 2023, the number of Internet of Things (IoT) connections in China exceeded 6.6 billion, with a compound annual growth rate of approximately 16.4% over the next five years^[1]. With the increasing number of IoT devices and the expansion of application scenarios, embedded systems will play an increasingly important role in connecting and managing large-scale IoT devices. Embedded systems need to handle massive data and provide secure and reliable connectivity and communication functions to support various IoT applications, including smart cities, industrial automation, smart health, and more.

In the development of the IoT, edge computing is rapidly rising. With the increasing demands for real-time performance and privacy, edge computing, as a method of processing and analyzing data locally, is becoming more popular. Edge computing can achieve rapid responses and immediate decisions while reducing reliance on cloud resources. Embedded systems are playing a crucial role in edge devices, and the development trend of edge computing requires continuous improvement in the performance and energy efficiency of embedded systems to meet the demands of edge computing scenarios.

As artificial intelligence (AI) and machine learning (ML) continue to deepen their applications across various fields, and with the proliferation of IoT and edge computing, embedded systems will be increasingly capable of integrating AI technology. This capability includes implementing complex AI algorithms and model inferences on embedded devices to provide smarter and more adaptive functions and services, such as applications in autonomous vehicles, smart robots, and medical devices.

Research and discussions on trends in embedded software development have been actively progressing. Analysts from VDC Research have written a report on the 2024 IoT, Embedded, and Industrial Technology Predictions, outlining key themes and trends affecting the hardware and software markets and making a series of predictions for future technology developments^[2]. The report predicts that AI is steadily migrating from cloud data centers to the network edge, including embedded devices. The demand for optimized deployment of embedded and edge AI markets for specific industry applications is growing, driving edge AI hardware to become more application-oriented. The report also predicts that cloud-native development solutions will become mainstream, as cloud-native software development solutions achieved great success during the COVID-19 pandemic, and the embedded engineering community is rapidly adapting to this development model, gradually moving away from traditional local development methods.

Jacob Beningo, a well-known embedded software expert and founder of Beningo Embedded Group, pointed out in an article on five trends in embedded software for January 2024 that embedded software teams are rapidly adopting DevOps. It provides teams with methods for automating the build, testing, and deployment processes^[3]. Peter Winston, CEO of Integrated Computer Solutions (ICS), noted in a blog post in March 2024 that six major trends will have a profound impact on embedded development: cybersecurity requirements, coexistence and interoperability of devices, new chip architectures, connectivity technologies, AI/ML integration, and service-oriented architectures. These trends are sure to rapidly reshape the product development process in the short term and continue to have an impact in the future^[4].

In summary, this article will focus on discussing three major trends in embedded software development: AI/ML integration and edge intelligence, virtualization/containerization and hybrid deployment, as well as CI/CD and DevOps.

Microsoft defines edge computing as a distributed computing framework that allows IoT devices to quickly process data at the network edge and take action^[5]. Microsoft provides an example of edge computing: security cameras in remote warehouses use AI to identify suspicious activities and only send that specific data to the main data center for immediate processing. The cameras do not continuously transmit all recorded video clips but only send relevant segments, avoiding the burden of transmitting 24 hours a day to the network, thereby freeing up the company’s network bandwidth and computing resources for other uses.

Edge AI is moving intelligent computing away from a cloud-centric model and bringing it closer to the data source. The driving motivation for this shift is to reduce network traffic, allow time-critical applications (such as manufacturing and autonomous systems) to make real-time decisions, and enhance privacy by processing data locally.

Edge AI reduces reliance on hyperscale AI providers and promotes broader AI applications. It has the potential to transform operations in healthcare, automotive, and robotics, reshaping the operational paradigms of these industries. Looking ahead, AI will have different impacts on different types of edges. Different types of edge computing are illustrated in Figure 1.

Figure 1 Three Types of Edges and Common Associated Devices

Thick Edge refers to computing resources equipped with high-performance computing components (such as high-end central or graphics processing units), usually located in data centers, designed to handle compute-intensive tasks/workloads such as data storage and analysis. Thin Edge refers to smart controllers, network devices, and computers that aggregate data from sensors. Micro Edge refers to intelligent sensors and devices that generate data.

Edge AI involves deploying AI models on edge computing devices to achieve AI inference and decision-making without relying on cloud connectivity. Edge AI can also be segmented into thick edge AI, thin edge AI, and micro edge AI.

Thick edge AI is positioned to support multiple AI inference models on edge servers, including training or retraining AI models for locally sensitive data scenarios. Thin edge AI utilizes gateways, IPC (Industrial Personal Computer), and PLC (Programmable Logic Controller) for AI processing at the network edge, enhancing the intelligence capabilities of existing sensors and devices. Micro edge AI supports direct integration of AI into sensors, improving the scalability of intelligent systems and enabling everyday devices to make autonomous decisions.

At the micro edge, Tiny Machine Learning (TinyML) is an important technology. This technology enables machine learning models and algorithms to run on resource-constrained chips (such as MCUs) with limitations in memory, computation, and energy consumption. Unlike the currently popular large models, TinyML’s size is significantly smaller than that of typical machine learning models, and it is far less than that of large models. Many IoT devices are already using TinyML technology, such as smart homes (thermostats and smart lighting systems) and health monitoring (heart rate and sleep quality).^[7] In a case showcased at the Embedded World 2024 exhibition, TinyML was integrated into everyday items and tools, enabling them to make autonomous decision-making functions without cloud connectivity (as shown in Figure 2). This approach enhances privacy and data security.

Figure 2 Drill and TinyML Development Tools in IoT Development Kits for Edge Sensing Based on ESP32

Today, edge AI presents three major development trends: thick edge AI training, thin edge and micro edge NPU (Neural Processing Unit) acceleration, and micro edge TinyAI empowerment. The first trend is that AI model training is shifting from the cloud to thick edge locations such as data centers or micro servers. Technically, this is possible because edge micro servers integrate high-performance CPUs and GPUs, enabling powerful computing at the edge, including AI training and various AI inference functions. AI training can also be conducted at local data centers, which has four major benefits: reducing reliance on cloud infrastructure, lowering costs, enhancing privacy, and improving the responsiveness of AI applications on edge devices.

Integrating dedicated NPUs in edge devices can significantly enhance AI inference capabilities. NPUs can reduce power consumption, improve energy management, and facilitate efficient multitasking, enabling AI to be deployed in power-sensitive and latency-critical applications (such as wearable devices and sensor nodes). Data from Arm indicates that the configuration of ARM Cortex A55 for AI inference compared to Arm Cortex A55 + Arm Ethos U65 NPU shifts 70% of AI inference from the CPU to the NPU, improving inference performance by 11 times.

Micro edge and thin edge AI have achieved localization of autonomous decision-making. The trend of directly integrating AI chipsets into cellular IoT devices is emerging, marking a shift in IoT devices towards intelligent, autonomous IoT systems capable of localized decision-making. This trend is expected to have a significant impact on industries such as smart cities and factories, bringing significant advantages, including real-time data processing, reduced latency, and increased efficiency through smaller form factors.

Embedded developers place great importance on AI development. The 2023 Embedded.com Embedded Development Status Survey Report shows that among the advanced technology capabilities utilized in embedded development, embedded AI and machine learning are the most focused, followed closely by embedded vision and voice capabilities (as shown in Figure 3).^[8].

Figure 3 Utilizing Advanced Technology Capabilities in Embedded Development

Virtualization technology is an important technology for operating systems and is showing a thriving trend in embedded systems. By abstracting the underlying processor cores, memory, and peripherals, this technology allows multiple virtual machines to run on the same physical processor. Virtualization provides an environment for running multiple operating systems, such as high real-time operating systems (like FreeRTOS) and general-purpose operating systems (like Linux) simultaneously on the same device. Virtualization technology is a new solution adopted by today’s high-performance multi-core processor computing systems, balancing the needs for generality and reliability. A typical application of embedded virtualization can be found in intelligent automotive electronic systems, for example, running a high real-time operating system and a human-computer interaction operating system simultaneously in a smart cockpit system.^[9].

Hybrid critical system applications in smart industrial scenarios can be deployed using multi-core processor systems and virtualization technology. A domestically produced virtualization operating system solution is shown in Figure 4. This solution is based on the Intewell real-time operating system and is a configurable hybrid heterogeneous system solution aimed at CNC systems and low-cost hardware.^[10].

Figure 4 Intewell Configurable Hybrid Heterogeneous System Architecture

Containers are a lightweight, portable, and isolated software environment. Through container technology, developers can package applications and their dependencies together, ensuring consistent operation across different platforms and simplifying application development, deployment, and management processes. Containers and virtual machines have similar resource isolation and allocation functions; container virtualization is for software environments, while virtualization refers to hardware virtualization.

Orchestrators play an important role in container technology, with orchestrators like Kubernetes designed to automate the deployment, scaling, and management of containerized applications. Docker is the most common container platform in Linux environments. With the widespread application of cloud computing and big data, some virtualization is being replaced by lightweight container technology, and the role of orchestrators is becoming increasingly important. Research on orchestrator performance and energy consumption is gaining attention; for example, Zheng Zhongbin and others proposed using linear programming to improve resource utilization while reducing carbon footprint.^[11].

Real-time containers are container mechanisms implemented based on real-time operating systems (RTOS), closely aligned with Docker user habits and compliant with Open Container Initiative (OCI) standards, achieving lightweight, secure, and convenient container deployment.

Currently, there is limited public research in China on real-time containers, with a representative example being the real-time container ECS (Edge Container Stack) developed by Yihui based on SylixOS. ECS is a lightweight real-time container technology that maintains performance comparable to the standard version of SylixOS. ECS real-time containers provide isolated runtime environments for each container. The fully independent container runtime environment ensures that applications within the container are unaffected by the environment and other applications, ensuring the expected runtime environment for the container and enhancing the security of applications within the container.

Deng Guanghong and others pointed out that due to limitations in development time and software ecosystem support, ECS real-time containers are not yet mature and struggle to meet the demands of building large container clouds.^[13]Research on real-time containers abroad is relatively rich, mostly based on a combination of real-time Linux patches and Docker to achieve real-time containers. Experts from NXP^[14]have focused on container solutions in resource- and cost-constrained MCU/MPU fields, proposing a solution inspired by Android container technology—MICROEJ VEE, which can be used in scenarios such as smartwatches, smart washing machines, smart meters, and industrial printers, as illustrated in Figure 5.

Figure 5 A Small Real-time Container Solution: microeJ VEE

What are mixed critical systems (MCS)? Experts from the University of York in the UK provide the following definition: mixed critical systems refer to systems that contain two or more components with different levels of criticality, such as safety-critical, mission-critical, and non-critical components. These systems are often found in complex embedded systems in the automotive and avionics fields, evolving towards MCS to meet strict standards related to non-functional requirements such as cost, space, weight, and power consumption.^[15].

The research and engineering practice of mixed critical systems aim to achieve mixed deployments of multiple systems through deployment, isolation, and scheduling techniques, ensuring mutual isolation and protection between subsystems, and enhancing resource utilization through scheduling. Specifically, on a technical level, academia focuses on research into scheduling methods; in one study from Northwestern Polytechnical University, the authors proposed a mixed critical task scheduling algorithm based on heterogeneous multi-core systems.^[16]Industry is more concerned with mixed critical deployments under spatial and temporal isolation, ensuring mutual isolation between subsystems, with resource allocation being static, leading to low overall resource utilization. To address this issue, Huawei proposed a mixed critical system solution based on openEuler, as illustrated in Figure 6.

Figure 6 Mixed Critical System Architecture

The implementation of mixed critical systems can rely on a resilient integration base, with containers and virtualization technology at its core. This solution recommends using lightweight containers like isula, while embedded virtualization recommends using ZVM and Jailhouse. ZVM (Zephyr-based Virtual Machine) is an open-source project that implements virtualization functions based on the Zephyr real-time operating system, led by the Hunan University Key Laboratory of Embedded and Network Computing, currently capable of simultaneously starting a Linux and multiple Zephyr RTOS, thus achieving mixed kernel deployment on the same hardware platform.^[18]Mixed critical systems are currently mainly applied in strong real-time and high-security fields such as smart manufacturing, robotics, energy, military, and aerospace.

DevOps is a software development practice that promotes collaboration between development and operations, enabling faster and more reliable software delivery. DevOps is often understood as a culture that connects developers, processes, and methods to provide continuous value.^[19]DevOps is widely used in the IT industry, with specific practices being CI/CD (Continuous Integration/Continuous Delivery), which is a method of rapidly iterating and delivering applications to customers by introducing automation technologies during the application development phase. The steps associated with CI/CD are commonly referred to as the CI/CD pipeline, collaboratively completed by development and operations teams in an agile development manner.

With the rapid development of technologies such as cloud computing, containers, and microservices, the concept of cloud-native has been widely accepted in the market. Cloud-native technology is a core component of DevOps practices, enabling automated application development, testing, and deployment through the use of cloud-native tools and processes, accelerating software delivery speed and improving application quality and stability. The DevOps one-stop platform integrates cloud-native development models into products, providing developers with user-friendly cloud-native research and development management solutions, including agile project management, code management, automated testing management, and CI/CD pipeline functionalities, allowing seamless integration of the entire process of development, testing, and deployment with the cloud-native underlying platform, lowering the threshold for embedded developers to get started and accelerating the implementation of CI/CD and DevOps cloud-native technologies in embedded development.

Zurich University of Applied Sciences’ Institute of Embedded Systems (InES) adopted an automated orchestration method based on cloud-native CI/CD pipeline technology in industrial protocol certification testing. The specific steps include integrating test cases and testing environments with Azure DevOps CI/CD, integrating Pipelines (orchestration), proof of concept for PROFINET (Industrial Ethernet), and creating and validating unit testing environments. The conclusion is that this method has transformed the traditional V model (Verification Model) development process from waterfall to agile development, accelerating the development cycle and improving efficiency.^[20].

Siemens conducted a detailed analysis of the current software engineering development in the automotive electronics industry. Siemens researchers pointed out that OEMs and many first-tier suppliers are adopting software factory development methods, such as the Volkswagen Group’s software company CARIAD, which was separated from traditional automotive development, and the software development teams in Shannon, Ireland, and Manchester, UK.^[21]Automotive software development is embracing CI/CD, supporting continuous software updates throughout the lifecycle across multiple vehicle models and platforms. A typical CI/CD development process is illustrated in Figure 7.

Figure 7 Typical CI/CD Development Process

Traditional embedded software development tools are integrated development environments that combine editing, compiling, building, and debugging into a development environment centered around engineering packages, characterized by low entry barriers, high integration, and strong targeting. However, the drawbacks are also evident: cross-platform compatibility, upgrading, and third-party integration are inconvenient. Typical traditional development tools include IAR Embedded Workbench.

In the past decade, the complexity of software running on high-end microcontrollers has significantly increased, with real-time operating systems (RTOS) and communication stacks achieving wide application. With the proliferation of IoT, additional software layer communication protocols (MQTT) and advanced security management (such as cloud-based device management) are essential in IoT applications. Meanwhile, middleware such as GUI libraries, DSP libraries, and machine learning software frameworks have increased the complexity of manually integrating software components.

To improve the embedded development process, Arm developed the CMSIS-Toolbox utility, which is an open-source project and part of Open-CMSIS-Pack^[22]. Arm launched CMSIS-Pack in 2014, which is part of version 4 of CMSIS (Cortex Microcontroller Software Interface Standard). CMSIS-Toolbox supports the use of different tools and workflows, helping embedded software developers collaborate more easily, as illustrated in Figure 8.

Figure 8 CMSIS-Toolbox Open Workflow

There are now embedded development platforms that integrate cloud and CI/CD, such as Wind River Studio (hereinafter referred to as Studio). Studio is a cloud-native toolset for developing, deploying, operating, and servicing mission-critical intelligent systems.^[23]It is built for developing embedded device software platforms and applications, including cross-compilation systems and embedded device testing systems, allowing users to view the status of end-to-end workflows to diagnose complex problems in customized pipelines. Studio provides a highly integrated toolset on cloud-based infrastructure, automating embedded software development, and achieving custom CI/CD processes for platform developers, application developers, and testers.

In the era of intelligent systems, the significant challenge for embedded systems comes from software development. Over the past 40 years, integrated circuits have rapidly developed under Moore’s Law; unfortunately, software development has lagged far behind. Currently, embedded software development faces enormous challenges. Intelligent applications represented by autonomous driving must be based on high safety levels, while at the same time, due to the rapid development of high-performance multi-core chips, underlying software must be continuously updated to adapt to new heterogeneous multi-core architectures that include AI/GPU and various chip types. Additionally, the accelerated development of microkernel operating systems requires virtualization and container technologies to support hybrid deployment applications.^[25].

Looking ahead, the industry needs to study trends in embedded software development, grasp the latest demands of the electronic information industry, organize software engineering practice experiences, improve embedded software development methods, and be bold in adopting new technologies and platforms to embrace new opportunities for embedded software in the AI era.

(Author Affiliation: 1 Embedded Systems Association, Beijing; 2 Broadcom Corporation, Sunnyvale, USA)