In the third issue of 2024 of “New Industrialization”, an article titled “Research on Key Technologies and Development Trends of Industrial Real-Time Operating Systems” authored by Meng Yan, Li Mingshi, and Deng Changyi from the National Industrial Information Security Development Research Center is shared in full below:
Key Technologies and Development Trends of Industrial Real-Time Operating Systems
Research
Meng Yan, Li Mingshi, Deng Changyi
(National Industrial Information Security Development Research Center, Beijing 100040)
Industrial Real-Time Operating Systems (RTOS) play a crucial role as the central nervous system in various industrial scenarios and are an important foundation for digital industry. Currently, the global market for industrial real-time operating systems is largely occupied by foreign manufacturers, which provide a complete upstream and downstream software ecosystem around mainstream RTOS product kernels and have conducted extensive hardware bundling in embedded form, forming a stronghold. This article outlines the key technologies and development trends in the software system of industrial real-time operating systems, providing references for the accelerated development of domestic systems.
1. Overview of RTOS
RTOS is a real-time operating system aimed at industrial applications, widely used in critical scenarios in industrial automation, military, power, new energy, and other industrial fields. It typically does not have high requirements for processor performance and has relatively limited resource demands, but it has stringent requirements for real-time performance, parallelism, reliability, and power consumption. The update cycle for its processor chips usually ranges from several years to several decades. In industrial products or manufacturing processes, real-time operating systems usually serve as the “brain” of equipment or production systems in embedded form, significantly enhancing the intelligence of equipment and production processes, and meeting the demands for efficiency, reliability, real-time performance, and sustainability in industrial production. In practical applications, RTOS is not a singular, closed software tool but is widely connected and coordinated with hardware, protocols, and application software, forming a vertically and horizontally interconnected ecological network. Modern RTOS should focus on the new challenges brought by the development trends of human-machine-object integration in intelligent environments under the industrial internet, such as how to achieve optimized scheduling and arbitrary migration of tasks in cloud-edge-end collaborative environments with embedded software-hardware coupling, resource constraints, hardware heterogeneity, multiple categories or no operating systems, and various communication methods.
2. Current Development Status at Home and Abroad
With the advancement of national strategies such as “Industry 4.0” and intelligent manufacturing, in-depth research on real-time operating systems has increasingly attracted the attention of research teams both domestically and internationally, resulting in a wealth of research achievements. However, overall, domestic research in the RTOS field is still in the development stage, and there are still many challenging tasks that need further exploration. The following elaborates on the current research status related to RTOS overall framework and further outlines the mainstream real-time operating system products globally.
(1) Related Research Work Currently, a large number of studies have been conducted in China focusing on the overall framework design of industrial operating system software, with a focus on enhancing system security, reliability, and trustworthiness. To ensure the security and reliability of the operating system kernel, the team from East China Normal University has introduced formal methods into the verification of operating system kernels, proposing a general automated verification framework for operating system kernels to assist in solving the security and reliability issues of foundational components in software systems. In response to the inability of existing embedded operating systems based on the MILS architecture to effectively perform correct and safe migration after operational failures, the team from Xi’an Aviation Computing Technology Research Institute proposed a dynamic management architecture for multi-level security domains in embedded operating systems, ensuring dynamic migration of tasks and functional reconstruction. The team from Zhejiang University proposed a high-trust embedded operating system based on artificial intelligence, capable of achieving high trust rates, precision rates, and recall rates under varying numbers and categories of tasks. (2) In addition, research on RTOS software frameworks is gradually becoming more detailed and closely integrated with typical scenarios in key industries. For example, the team from Nanjing University of Science and Technology built an overall framework for a fire control system based on the SylixOS embedded real-time operating system, simulating the ballistic calculation and fire control hit process. Based on the analysis results of flight control software requirements for aircraft, the team from Beijing Aerospace Automatic Control Institute proposed a multi-core distributed flight control software architecture based on the Zhanxing embedded real-time operating system, improving the reliability and safety of flight control software. In response to the high-speed, high-real-time, and high-stability communication requirements of current phased array antenna control systems, the team from Xi’an Jiaotong University designed a real-time control system based on the Zynq-7000 and RT-Thread operating systems, ensuring the real-time performance and stability of system execution. In the fields of shipping and maritime operations, the team from China Shipbuilding Industry Corporation 713 Research Institute designed a system control software based on VxWorks, achieving good control effects in terms of stability and speed; the team from Nanchang University designed a general software architecture for ship operating systems based on embedded technology, achieving precise route control of ships in harsh marine environments. (3) Major Domestic and Foreign Products
1. Major Foreign Products Advanced technologies and mainstream products related to RTOS are dominated by countries such as the United States, the United Kingdom, and Germany, with high maturity and usability, widely applied in core fields such as aerospace, shipping, and energy, occupying a significant share of the international market. Major products from the United States include VxWorks from Wind River, μC/OS-II from Micrium, TI-RTOS from Texas Instruments, INTEGRITY RTOS and µ-velOSity RTOS from Green Hills Software, Azure RTOS from Microsoft, Lynx OS from Lynx Software, DEOS from DDC-I, Amazon FreeRTOS from Amazon, MIPS Embedded OS from MIPS, and RTLinux from FSMLabs; major products from the United Kingdom include FreeRTOS from RTOS, Mbed OS and RTX from ARM, and SAFERTOS from WITTENSTEIN; major products from Germany include embOS from Segger, PikeOS from SYSGO, and Nucleus RTOS from Siemens. Additionally, real-time operating system products such as ENEA OSE from Enea Data AB in Sweden, SCIOPTA from SCIOPTA Systems in Switzerland, QNX Neutrino RTOS from BlackBerry in Canada, MQX RTOS from NXP in the Netherlands, Zephyr from the Linux Foundation, µClinux from the GPL organization, as well as TizenRT, OpenWrt, DuinOS, and Apache NuttX also occupy a stable market share.
2. Major Domestic Products In recent years, the scale of China’s RTOS industry has continued to grow, with many excellent enterprises and products emerging, including RT-thread from Shanghai Ruide, Huawei Lite OS, AliOS Things from Alibaba, Sylix OS from Yihui Information, Intewell-C from Dongtu Technology, Delta OS from Kexin Jingcheng, Intewell OS from KEDONG Software, Acore OS from AVIC Computing Institute, Hopen OS from Kaisi Haopeng, JARI-Works from China Shipbuilding 716 Institute, ReWorks from China Electronics Technology Group Corporation 32 Institute, and Aworks OS from Zhiyuan Electronics, among other representative products. Among them, RT-thread and Huawei Lite OS were selected for the 2022 list of top real-time operating system companies published by SourceForge, a globally recognized open-source software development platform and repository. Major domestic and foreign RTOS products are shown in Figure 1.
Figure 1 Major Industrial Real-Time Operating System Products at Home and Abroad
Overall, the level of domestic industrial real-time operating system products still lags behind developed countries, with gaps in product reliability, three-dimensional graphics support capabilities, and other aspects compared to foreign software, while lacking a complete software and hardware ecosystem. Currently, the domestic market still primarily uses American products such as VxWorks and μC/OS-II, with a low localization rate in key areas. The market share of embedded real-time operating systems is shown in Figure 2.
Figure 2 Market Share of Embedded Real-Time Operating Systems
Data Source: VDC report / Zhiyan Consulting and other research reports
Figure 3 Industrial Real-Time Operating System Software Product System
3. Key Technologies of RTOS
The RTOS product system mainly consists of three parts: kernel layer, middleware layer, and auxiliary design tools, as shown in Figure 3.
(1) Real-Time Operating System Kernel The kernel provides the most basic functions of the real-time operating system and is the core part of the operating system’s operation. The kernel is the first layer of software expansion based on hardware, mainly implementing functions such as task management, memory management, file management, queue management, and clock management. Industrial real-time operating systems have strong real-time constraints on the kernel, requiring it to quickly process external requests and respond to requests within a specified time, involving key technologies such as task scheduling, interrupt management, clock management, synchronization, and communication mechanisms. Currently, most real-time operating system kernels can be divided into microkernel and monolithic kernel types. Microkernels generally only contain essential core functions such as task scheduling and memory management, while peripheral drivers and other services run in other processes and can communicate through bare kernels, with the main advantage being modular flexibility. Monolithic kernels share space with the kernel and operating processes, allowing messages to be directly passed between processes, featuring integration without the need for additional mediation modules, providing certain performance advantages.
Currently, operating systems widely used in critical industrial scenarios, such as VxWorks, have adopted monolithic kernel designs. As the limitations of hardware platform resources become increasingly prominent, more emerging industrial real-time operating system products are adopting microkernel architectures to achieve advantages such as ease of maintenance, convenient modification, and flexible operation.
1. Task Scheduling Algorithms Task scheduling algorithms are key to enhancing the real-time performance of multi-task management. Efficient scheduling algorithms can enable the system to quickly complete task scheduling and switching while meeting high performance and stability conditions.
Currently, domestic and foreign research teams have achieved certain research results and technological breakthroughs in the area of task scheduling for real-time operating system kernels. The team from Shanghai University of Science and Technology proposed a hybrid critical-level task semi-partition scheduling algorithm that can recycle idle resources of multi-core processors and allocate idle time slices to abandoned low-critical-level tasks when a high-critical-level mode is triggered. Additionally, this team proposed a real-time hybrid task scheduling algorithm for embedded multi-core systems. This algorithm improves the priority determination of tasks by introducing a relaxation parameter based on an improved boundary fair real-time hybrid task algorithm and further proposes a heuristic algorithm that combines relaxation and heuristic strategies to improve task allocation strategies, effectively meeting the real-time requirements of the system. The team from Beijing University of Posts and Telecommunications proposed an RTOS stream data task scheduling model based on multi-core operating systems, which adopts a structure of real-time streaming processing task packets to improve the efficiency of stream data processing. The team from Henan University proposed a heuristic independent task scheduling algorithm based on task execution time, which allocates tasks to different resources through preprocessing, decomposition, pre-scheduling, and adjustment of the task execution time matrix, saving task completion time while maintaining low time complexity.
2. Interrupt / Exception Management Interrupt / exception management directly affects the system’s response and processing speed to external events, ensuring that time-sensitive functional parts operate in a timely manner. It mainly includes two processing parts: application-layer and low-layer, while providing users with a unified interrupt handling interface.
In terms of system exception verification and interrupt management, the team from Beijing Control Engineering Research Institute proposed a verification framework based on Hoare-logic for operating systems aimed at SPARC processor architecture, which can be used to prove the correctness of exception management in SPARC processor architecture operating systems; the team from Xi’an Aviation Computing Technology Research Institute proposed an exception handling method for operating systems based on PowerPC processors, establishing a PowerPC exception handling mechanism; the team from Yunnan University proposed a new interrupt task type around the Forth real-time system, which can handle real-time burst events in embedded operating systems and successfully run system terminals, background, and interrupt tasks based on the further proposed task scheduling algorithm; the team from Beijing Zhongkong Institute proposed a real-time operating system exception management verification method based on SPARCv8 (Scalable Processor Architecture Version 8), establishing a Hoare-style program framework EMS, using the execution scenario of the SpaceOS real-time operating system on the BeiDou-3 satellite as an example to verify exception management.
3. Clock The clock is the pulse and heartbeat of the real-time operating system, and its granularity and accuracy affect both the timeliness of task responses and the overall efficiency of the system. Clock management mainly includes time management, timing management, process accounting management, load management, etc. The clock interrupt mechanism drives time and timers in the operating system, serving as the foundation for all time-related operations in the system.
In terms of clock management research, the team from National University of Defense Technology has made certain breakthroughs in clock management for real-time operating systems, proposing a lightweight precise clock synchronization protocol implementation technology aimed at MCUs, which reduces data processing copies through a shared storage mechanism and replaces some precise calculations with approximate calculations achieved through shift operations, reducing the occupation of computing and storage resources; the team from China Aerospace Propulsion Technology Research Institute proposed a high-precision clock synchronization method based on distributed simulation systems, which includes high-precision logical clock construction, network loop optimization, and online compensation of clock crystal oscillator frequencies, advancing the practical application of clock synchronization in distributed simulation systems; the team from Budapest University proposed a portable PTP clock synchronization implementation based on microcontroller systems and its performance evaluation method, abstracting PTP packet timestamps, PPS signal generation, and external HW event timestamps to ensure good portability.
Synchronization and Communication Mechanisms Synchronization and communication mechanisms allow real-time operating systems to coordinate tasks and interrupts to ensure the collaborative completion of tasks. The technologies involved include priority ceiling protocols, priority inheritance protocols, etc., which can be used to avoid delays caused by priority inversion and other issues.
In terms of research on synchronization and communication mechanisms, the team from the Institute of Microelectronics, Chinese Academy of Sciences has made certain breakthroughs in synchronization and communication mechanisms for operating systems, proposing a multi-core communication mechanism based on a multi-core communication interface framework, which is divided into logical layer, operating system porting layer, transport layer, and porting layer, meeting the needs of embedded asymmetric multi-core processors in upper-layer application porting; the team from East China Computing Technology Research Institute proposed a deterministic inter-core communication mechanism for embedded operating systems, designing an upper limit for end-to-end communication delays to meet the requirements of multi-core high-security embedded operating systems; the team from North China University of Science and Technology conducted a series of studies on synchronization and communication mechanisms for real-time operating systems, designing a concentration monitoring alarm device based on the synchronization and communication mechanism of the FreeRTOS operating system, as well as a human-computer interaction terminal based on the synchronization and communication mechanism of μC/OS-Ⅲ; the team from Eindhoven University of Technology proposed a modeling, implementation, and analysis method for the application of the XRCEDDS communication standard in distributed multiprocessor real-time embedded systems, establishing a scene-aware data flow model to capture the dynamics of system behavior based on different execution scenarios.
(2) Middleware The middleware layer is located between the real-time operating system kernel and application software, used to coordinate and manage communication and interaction between applications. The support for rich middleware is a key factor affecting the scalability of real-time operating systems. The middleware layer provides data management, application services, message passing, and other functions and services for application software, including file systems, network protocol stacks, graphical user interfaces, device driver frameworks, etc. Additionally, there are many industry-specific middleware and standard interfaces aimed at industries such as aviation, aerospace, automotive, and electronics. In terms of code volume, the development workload of middleware is far greater than that of real-time operating system kernels, and a weak middleware layer has become one of the main factors restricting the research and application of domestic real-time operating system products.
In terms of middleware-related research, the team from Shaanxi International Business College designed a real-time middleware based on a publish/subscribe mechanism, which features data synchronization functions and real-time assurance mechanisms based on distributed objects, as well as a publish-subscribe communication mechanism based on Ethernet, meeting the real-time requirements for data distribution in distributed applications; the team from Fudan University designed a data service middleware that introduces virtual entities and protocol adapters, modeling data from IoT environmental perception to eliminate data format heterogeneity; the team from Tianjin University of Technology designed a hybrid computing middleware that effectively integrates high-concurrency real-time processing business logic, batch processing business logic, and cross-layer dynamic calls, shortening development time; the team from TuSimple proposed a robot middleware that combines high performance and high reliability, which can reduce communication latency and CPU usage by more than 40% compared to traditional methods.
In terms of related products, after a long period of development, commercial companies like Micrium have equipped their kernels with rich middleware, while open-source projects like FreeRTOS have also accumulated relatively complete middleware in their commercial projects such as OpenRTOS, capable of forming a relatively complete real-time operating system software platform. Domestic products like RT-Thread, while providing POSIX interfaces, network protocol stacks, GUIs, device file systems, shells, libc libraries, etc., rely heavily on other open-source components and lack corresponding auxiliary design and debugging tools.
(3) Auxiliary Design Tools Mature and complete auxiliary design tools can significantly reduce the difficulty and cost of RTOS development, enhancing the overall technical level of design and development, and are an important foundation for building the RTOS ecosystem. Auxiliary design tools mainly include integrated development environments, simulators, testing frameworks, etc. Integrated development environments provide source code compilers, project management tools, and other tool modules; simulators can simulate software/hardware functions, enabling the co-verification of embedded industrial real-time operating systems; testing frameworks can effectively improve the efficiency of developing, executing, and reporting automated test scripts.
In terms of research related to auxiliary design tools, the team from Wuhan University of Technology proposed a knowledge graph-based embedded operating system test case recommendation model, which designs an ontology model based on historical case knowledge and establishes a reuse recommendation model through knowledge recommendation, saving testing costs and having certain engineering application value; the team from Shanghai Micro Satellite Engineering Center proposed an automated testing method for embedded operating systems, which utilizes version management tools and continuous integration tools to achieve automatic compilation of scripts, automatic execution, and recording of test results, enhancing the level of automation in testing; the team from Capital Normal University proposed a differential fuzz testing method for ROS, which can accurately identify vulnerabilities in functional packages of different versions of ROS; the team from Beijing Institute of Computer Technology and Applications proposed a compatibility testing method for domestic operating systems based on dependency element comparison, which classifies dependency elements and obtains compatibility testing results by comparing the consistency of dependency elements between two operating systems.
In terms of development languages, current domestic real-time operating systems mainly support C and C++, while compared to mainstream foreign products, domestic products have relatively poor support for languages such as Python, JAVA, and RUST. Additionally, in specialized fields such as robotics, there is a lack of complete upper-layer simulation and auxiliary design tools, leading to long design and debugging times and difficulties in fully utilizing existing results such as control algorithms and strategies. In light of the increasing complexity and scale of embedded systems, automated development configuration and testing verification, as well as offline simulation development, have become important development directions that can effectively improve development efficiency.
4. Development Trends of Industrial Real-Time Operating Systems
(1) Lightweight Trend As RTOS applications become more widespread and in-depth, the scale and complexity of systems have significantly increased, posing challenges to system flexibility, reliability, maintainability, and hardware requirements, requiring RTOS to achieve more functional sets with a more streamlined structure. At the same time, user demand for the customizability of RTOS has become more pronounced, achieving flexible customization and configuration based on hardware and application environments through “plug-and-play” components, thereby reducing hardware overhead and improving system operational efficiency and reliability. The realization of lightweight and flexible deployment of RTOS relies on strong openness support. Embedded real-time operating systems developed based on the Linux kernel, such as RTlinux and μClinux, have the advantage of open-source code, utilizing the advanced microkernel system of Linux to achieve flexible customization of hierarchical structures and convenient system debugging. Additionally, the Linux ecosystem has accumulated rich resources and provides strong support for application development, leading to the rapid development of Linux-based real-time operating systems, which are gradually forming strong competition against traditional embedded real-time operating systems. (2) Virtualization and Cloudization Trend With the continuous development of computer technology, mainstream processor architectures have transitioned from single-core processors to multi-core processors, and hardware platforms based on multi-core are rapidly developing and widely applied. In multi-core systems, multiple computing cores operate independently, with tasks on each core sharing system resources such as cache and memory. Compared to multi-processor systems that also use parallel task computing modes, shared components can improve the utilization efficiency of hardware modules and reduce the number of hardware components. This interconnection of on-chip resources enhances communication efficiency between computing cores and brings more challenges for resource usage and allocation in real-time operating systems. Along with this technological trend, the virtualization process of embedded industrial real-time operating systems is accelerating, allowing for the isolation of functions of different subsystems through virtualization technology, enabling multiple embedded systems to run on a single hardware system management program. Furthermore, the application of industrial real-time operating systems in scenarios such as the industrial internet has promoted their cloudization development, such as placing time-sensitive control processes in industrial control on the cloud or edge for centralized management, facilitating interaction and cooperation between control processes, reducing system costs, and providing high application value. (3) Cross-Platform Portability and Networked Applications One major difficulty in the development process of embedded RTOS is the poor reusability of code. The upgrade and replacement of real-time operating systems involve a large amount of existing software porting or even redevelopment, leading to serious resource waste as existing work results are not fully utilized. Therefore, related standardization work has increasingly attracted attention. The rapid development of applications related to operating systems has also raised higher requirements for the system’s own performance, as single-processor chip computer systems can no longer adequately meet the needs of complex real-time application systems. Different hardware platforms based on X86, ARM, MIPS, C-SKY, PPC, RISC-V architectures, as well as domestic hardware such as Loongson processors and Feiteng processors will see broader applications, while RTOS will pay more attention to cross-platform compatibility to improve the reusability of embedded software and enhance system portability and scalability. At the same time, with the rapid development of internet technology, industrial real-time operating systems will become easier to port and network, providing support for protocols such as TCP and UDP based on equipped standard network communication interfaces, as well as unified MAC access layer interfaces, facilitating connections with various mobile computing devices.
5. Conclusion
RTOS is widely used in various critical scenarios, and compared to traditional software products, it requires the construction of a rich software and hardware ecosystem to further meet the demands for extensive connectivity and diverse loads. In this context, unified and standardized interfaces and infrastructure are prerequisites for code portability and reuse, as well as the foundation for various software and hardware collaborations, allowing end users to accumulate and reuse software usage habits and related experiences on the same platform. Domestic RTOS should strengthen multi-party collaboration to address shortcomings, enhancing the usability, reliability, and controllability of the system. First, strengthen research on key common technologies to promote the technological development of RTOS kernels in terms of high trust, high availability, and strong real-time performance; second, co-build an open product ecosystem, enrich the general and industry middleware ecosystem, promote software and hardware adaptation, and improve integrated, debuggable design development environments and tool support; third, strengthen quality standard support and guarantee, develop data models, interfaces, and management evaluation standards, enhance third-party evaluation capabilities, strengthen open-source risk prevention, and promote the standardized and secure development of industrial real-time operating systems.
Author Information
Meng Yan, National Industrial Information Security Development Research Center, Master, Engineer, Research direction: industrial software, industrial real-time operating systems, robotics;
Li Mingshi, National Industrial Information Security Development Research Center, PhD, Engineer, Research direction: industrial IoT, industrial software, real-time operating systems;
Deng Changyi (corresponding author), Deputy Director of the National Industrial Information Security Development Research Center, PhD, Senior Engineer, Research direction: industrial software, cyber-physical systems, industrial control, artificial intelligence chips, email: [email protected].
Source: New Industrialization
