01
Reference Architecture of In-Vehicle Intelligent Computing Platform
The in-vehicle computing platform focuses on characteristics such as system reliability, real-time operation, distribution elasticity, and high computing power, achieving functions such as perception, planning, control, networking, and cloud control, ultimately completing safe, real-time, and scalable multi-level autonomous driving core functions. As shown in Figure 10, the overall architecture of the in-vehicle computing platform mainly includes two parts: the vehicle control operating system and heterogeneous distributed hardware architecture. It operates on top of the hardware of the in-vehicle intelligent computing platform and the automotive electronic control unit hardware, supporting a software collection that enables the realization of intelligent connected vehicle driving automation functions and safe and reliable operation, which includes system software and functional software.
Figure 10 In-Vehicle Intelligent Computing Platform Architecture Diagram
(1) System Software Layer
The system software is a complex large-scale embedded system operating environment tailored for automotive scenarios, as shown in Figure 11. The system software generally includes the operating system kernel, virtualization management (Hypervisor), POSIX, system middleware, and services.
Figure 11 System Software Architecture
1. Operating System Kernel
The vehicle control operating system kernel supports heterogeneous chips and must consider functional safety and real-time performance requirements. Currently, the kernel system functions loaded in various units of the heterogeneous distributed hardware architecture have different safety levels: AI unit kernel system QM~ASILB, computing unit kernel system QM~ASILD, control unit kernel system ASILD, resulting in a multi-kernel design with different safety levels or a single kernel supporting applications of different safety levels. Ensuring differentiated functional safety requirements while meeting performance requirements is key to the design of the vehicle control operating system software. Additionally, the complexity of the in-vehicle intelligent computing platform also requires the kernel to support libraries for functional software and application software and to be highly programmable.
2. Virtualization Management (Hypervisor)
Hypervisor technology is an important way to achieve cross-platform applications and improve hardware utilization. Hypervisor is a type of hardware virtualization technology that manages and virtualizes hardware resources (such as CPU, memory, and peripheral devices) and provides them to multiple kernel systems running on top of the Hypervisor. The vehicle control operating system achieves effective resource integration and isolation through Hypervisor.
3. Portable Operating System Interface (POSIX)
POSIX is a standard widely adopted and followed by mainstream operating systems. Applications based on POSIX can be easily ported between different operating systems. POSIX also adapts well to the high-performance computing and high-bandwidth programming required for autonomous driving. Adaptive AUTOSAR also adopts a kernel system based on the POSIX standard, allowing the use of the standard POSIX API subset PSE51, aimed at meeting the needs of future advanced autonomous driving. The vehicle control operating system software is based on a real-time embedded software unit architecture and can draw on the ideas of the Adaptive AUTOSAR platform, using POSIX API to interact with application software and functional software across different kernel systems.
4. System Middleware and Services
System middleware is located within the system software, primarily managing computing resources and network communication, and providing basic system services for upper-layer applications. The most important middleware refers to distributed communication services, which mainly provide data and information exchange services between SOA applications in a publish/subscribe manner. The vehicle control operating system can establish a universal, high-speed, and efficient communication and data-sharing mechanism across multiple kernels, multiple CPUs, and multiple boards. The distributed middleware using a publish/subscribe architecture emphasizes a data-centric approach, providing rich QoS policies that ensure data is distributed in real-time, efficiently, and flexibly, meeting the needs of various distributed real-time communication applications. Representative distributed communication middleware technology specifications include DDS, SOME/IP, etc.
5. Safety Domain Operating System and Functional Services
The safety domain operating system is a real-time safety vehicle control operating system running on the MCU above the system software layer. The safety domain operating system mainly includes modules such as hardware abstraction layer, basic software, real-time operating system kernel, and runtime environment. The most basic requirement of the safety domain operating system is high real-time performance. The system must have hard real-time characteristics, completing resource allocation, task concurrency, synchronization, and other specified actions within a specified time, which can refer to the CP software architecture.
(2) Functional Software Layer
Functional software is developed based on the service-oriented architecture design concept of the vehicle control operating system, extracting the core common needs of intelligent driving to form functional service modules for intelligent driving, efficiently realizing the development of driving automation functions. As shown in Figure 12, functional software consists of application software interfaces, general models for intelligent driving, a common framework for functional software, and data abstraction.
Figure 12 Functional Software Architecture
1. Application Software Interface
Vehicle applications are built on the foundation of functional software, which provides calls and services for application software through a unified application software interface. The development and operation of application software can be independent of specific sensors and vehicle models. Different market participants (including government authorities, OEMs, suppliers, traffic facility managers such as highways or parking lots, and individuals) can develop applications. Applications can be packaged, deployed, started, scheduled, and upgraded. The functionalities of the application can be defined by users, roadside units, and cloud services, triggered by application scenarios. With the support of the functional software layer, the development of applications will trend towards lightweight, increasingly focusing on rules set by business logic itself.
Applications are built on more abstract environmental models, vehicle models, task models, and resource models, providing better portability compared to functional software and able to be deployed across vehicle models and computing platforms. Compared to functional software, applications focus more on business rather than functionality, more on the user side than the system side, and more on objectives rather than methods. Applications can be built on the services provided by functional software or directly on environmental and vehicle models.
Application interfaces involve not only the operation of applications but also development and management interfaces for applications. System software vendors should provide a unified development environment and tools for application software development, offering users various forms of SDKs, such as environmental models, functional configurations, various algorithm SDKs, and including the necessary toolchain, software packages, development interfaces, development documentation, demonstration applications, and configurations for application development.
2. General Model for Intelligent Driving
The general model for intelligent driving is a modeling abstraction of processes such as intelligent cognition, intelligent decision-making, and intelligent control in intelligent driving. The general model for intelligent driving consists of an environmental model, a planning model, and a control model.
The environmental model serves as an intelligent cognition framework, providing a modeled general description of environmental information for intelligent decision-making and intelligent control. The environmental model schedules various perception, fusion, and localization algorithms, processing sensor detection information, vehicle-road, vehicle-vehicle coordination information, and high-precision map prior information, providing detection, characteristics, objects, situations, scenes, and various levels of semantic information about road traffic environments and vehicle status information.
The planning model provides the vehicle with a driving trajectory for the near future based on information from the environmental model, vehicle positioning, personalized settings, and vehicle status feedback, mainly divided into behavior prediction, behavior decision-making, and motion planning. Behavior prediction predicts the future driving trajectories of other traffic participants based on perception and map data, providing more comprehensive and reliable reference information for behavior decision-making; behavior decision-making provides behavioral strategies for the vehicle while providing corresponding planning constraints for motion planning, ensuring that the planning results not only meet the hard requirements of traffic laws but also align more closely with human driving strategies; motion planning, based on the information above, plans a safe, comfortable, and correct trajectory for the vehicle in the near future.
The control model mainly consists of normal operating conditions and degraded operating conditions, where normal operating conditions mainly address dynamic driving tasks within the ODD, while degraded operating conditions address systemic failures or dynamic driving tasks beyond the ODD, requiring input processing, state decision-making, control computation, and execution output. Both upstream and chassis information inputs and control outputs require an adaptation layer to match different functional algorithm framework platforms and vehicle platforms; for longitudinal and lateral control and emergency control algorithm modules, fault diagnosis, configuration, and calibration interface modules need to be unified and managed.
3. Common Framework for Functional Software
The common framework for functional software is the foundation that carries the general model for intelligent driving, divided into data flow framework and basic services.
The data flow framework encapsulates different intelligent driving system software and middleware services downwards, providing an algorithm framework that decouples algorithms from underlying system software within the intelligent driving general model. The main role of the data flow framework is to abstract, deploy, and drive algorithms within the intelligent driving general model, solving issues of cross-domain, cross-platform deployment, and computation.
The basic services are fundamental services shared by the functional software layer, primarily serving the intelligent driving general model or functional applications, but not limited to intelligent driving. The basic services platform includes reliable redundant components, information security services, and networking cloud control services, where reliable redundant components abstract all other software and hardware modules in the system as managed entities, completing monitoring and fault processing of the entire system through interaction with all managed entities; the data security service in information security basic services defines data types and security levels for vehicle-side data, formulating security policies and data processing rules for different types of data needed for vehicle functions and applications in various vehicle operating scenarios. Algorithm deployment and data flow orchestration modules on the data flow framework define control algorithm deployment and data exchange according to rules. Networking cloud control services can provide safety redundancy information of the operating system, beyond-visual-range information, and information from the general model, achieving communication with vehicle-to-vehicle, vehicle-to-cloud, vehicle-to-person, and vehicle-to-roadside infrastructure through LTE-V2X, 4G/5G communication methods.
4. Data Abstraction
Data abstraction standardizes the processing of data from sensors, actuators, vehicle status, maps, and cloud interface data, providing various data sources for the upper-layer intelligent driving general model, thereby establishing heterogeneous hardware data abstraction, achieving decoupling of functionality and application development from underlying hardware.
02
Core Architecture of the In-Vehicle Intelligent Computing Platform SOA
The design philosophy of SOA is to decompose applications into specific functional components or services, independent of hardware and operating systems, accessed through standardized protocols and application programming interfaces (APIs). These service designs should be shareable rather than restricted to specific hardware and vehicle models.
Some components or services related to the cloud should be designed to run on local computers (computing platforms) or distributed networked computer clusters (edge cloud or central cloud servers), with remote access and independent updates in the design of application and service components.
The underlying system and basic software design of the computing platform need to provide a friendly and stable SOA infrastructure for upper-layer services and applications. This mainly includes the following aspects:
-
Decoupling: The operating system decouples from the hardware platform, and the underlying software is independent of vehicle models, operating systems, and programming languages. Kernel/POSIX/middleware is independent of business logic, and data sources decouple from sensor hardware design.
-
Layering: The entire system should be designed with a layered architecture, clearly defining interfaces between different levels of the system and various basic service components, adopting industry-recognized interfaces and standards, compatible with traditional vehicle controllers, operating systems, and protocols.
-
Modularity: Decomposing the basic service software functions into different types of one or more independent functions, where functions are mutually independent, facilitating the construction of upper-layer applications, such as data collection, data backhaul, OTA, information security, and networking cloud control. The basic services for intelligent driving functions can also be decomposed, such as state machines, mode managers, algorithm modules, and environmental models.
-
Abstraction: Implementing common data abstraction for different perception hardware, isolating the last calculation module while enabling quick hardware matching.
-
Standardization: Standardization of interfaces and data.
(1) Software-Hardware Decoupling
Software-hardware decoupling is independent of hardware design in software system and application design, abstracting hardware interfaces through constructing a universal software architecture to accommodate different hardware devices.
Providing a sensor abstraction mechanism that supports mainstream types and models of sensors, with the capability to extend to new sensor types. Providing rich hardware adaptation service software, which includes two parts: rapid adaptation to hardware platforms and rapid adaptation to vehicle platforms, where rapid adaptation to hardware platforms includes kernel, middleware, AI, and safety domain aspects, and rapid adaptation to vehicle platforms includes sensor abstraction, actuator abstraction, HMI data interfaces. The main components include:
1) Platform decoupling and adaptation;
2) AI module porting and deployment;
3) Sensor abstraction;
4) Actuator abstraction;
5) Map data;
6) Middleware adaptation;
7) HMI data;
8) Core vehicle signals;
9) V2X data.
In the SOA architecture design, extracting common functions from complex applications and services, decomposing them into different basic service functions aims to maximize the use of existing modules and services to improve development efficiency. Function decomposition should follow:
1) High cohesion within basic services, low coupling between services;
2) Low-coupling services should use standardized service interfaces as much as possible;
3) If a certain functional module’s complexity is still very high, it needs to be further split through common extraction.
By decomposing complex autonomous driving functions and algorithms, forming basic modules, state machines/mode managers, algorithms, environmental models, providing a component-based solution for developing L0~L4 level autonomous driving function applications, supporting component-based rapid development and verification.
OEMs can choose different functional modules and algorithm components based on their strategies when designing and developing functional software, achieving plug-and-play functional combinations and flexibly constructing intelligent driving system-level solutions.
(3) Networking Cloud Control Services
Networking cloud control services provide standardized, abstract information services, such as traffic light information, traffic reminder information, safety warning information, roadside perception information, and information about surrounding vehicle movements, while also offering the capability to plug in and extend algorithms, allowing for the addition, conversion, and adaptation of different cloud control algorithms and applications. The networking cloud control module acts as a bridge for information communication inside and outside the vehicle, allowing the vehicle platform to broadcast its status and driving intentions to the surrounding environment or upload them to the cloud platform, while also receiving perception information (such as obstacle information), decision-making planning suggestions, and even operational trajectory information from the surrounding environment or edge cloud.
In the design of related service designs, the SOA design philosophy can be followed, ensuring that services are platform-independent. Perception algorithms running on the platform can integrate V2X road information from the cloud, achieving vehicle-road collaboration. Vehicles can subscribe to cloud-based perception and planning data, fully utilizing cloud computing power and multi-dimensional scenario information to realize operational control application scenarios. For example, fully automated parking in parking lots with perception devices.
The networking cloud control module can seamlessly connect to existing V2X scenarios through application designs based on SOA architecture design, supporting cloud control applications and cloud-vehicle collaborative applications. Through 5G low latency and high-speed communication technology, it supports digital twins, enabling in-vehicle computing and applications to float and extend to the cloud edge.
When designing information security services, the SOA approach should be considered. For example, information security monitoring may run on the platform or in the cloud. Information security services designed based on SOA do not rely on platforms and operating systems, allowing seamless integration with cloud security applications, or quickly introducing third-party information security services.
System software is a complex large-scale embedded system operating environment tailored for automotive scenarios. System software generally includes the operating system kernel, virtualization management (Hypervisor), POSIX, system middleware, and services. By integrating virtualization management, system kernel, middleware, and other components through the system software platform, a stable, efficient, and secure SOA service operating environment can be provided for upper-layer functional software, as well as hardware-independent application development interfaces.
Below is a description from the perspective of system layered design:
1) System Kernel: Isolates the platform hardware and is key to hardware platform migration and adaptation. The design of the computing platform should consider mainstream operating system kernels as much as possible, reducing the costs of platform migration and adaptation, meeting the needs of OEM vehicle designs to change platforms as required.
2) Virtualization Management: In the EE architecture, transitioning from distributed to centralized computing platforms, adopting Hypervisor technology that ensures various application systems have certain isolation will be key to achieving high-performance intelligent driving operating systems. For example, different operating systems can be used for vehicle computing and real-time control domains.
3) Middleware: Is a critical part that isolates system software and application services. Especially communication middleware is key to the computing platform SOA. The design of communication middleware must consider the needs of massive data transmission for autonomous driving (e.g., DDS), while also accommodating the requirements of traditional AUTOSAR and OSEK. Communication interfaces should include real-time APIs, non-real-time asynchronous C-S, Restful, etc.
The autonomous driving development SDK enables OEMs to freely choose different hardware and software algorithms through a series of software components and tools to assemble their autonomous driving systems. In particular, it allows OEMs to focus on building their specific applications, meeting the requirements for developing machine learning algorithms from L2 to L3+, isolating hardware integration, messaging, reliable real-time execution, and other issues.
Providing a common set of algorithms and models (including environmental models, planning models, control models) for different application classifications, supporting application developers in achieving efficient and low-cost application development through application software interfaces (SDK/API).
Through standardized algorithm frameworks, compatibility with multiple third-party algorithms can be achieved, providing OEM manufacturers with various choices through an increasingly rich algorithm ecosystem.
With a complete simulation testing process and a rich scenario library, it can support closed-loop simulation testing based on SIL, MIL, HIL. Users can use these SDKs, reference target vehicle platforms and hardware configurations, supported sensor and other hardware types, as well as the provided data abstraction, interface services, and development tools to realize complete, customized autonomous driving application functionality development (e.g., ACC, LKS, HWA, etc.).
