Real-Time Verification Monitoring of Automotive Embedded Systems: A Case Study of Transmission Control Systems

Complex embedded processing systems in safety-critical applications require assurance of good functional lifespan operation without substantial cost overhead. However, embedded program verification is often unsolvable because theorem-proving solutions do not scale to large embedded software systems in practice. A more viable approach is to formally observe some selected proof obligations during the actual operation phase of the system. This method does not replace comprehensive program and system verification, but it does provide a solution for confirming the correct behavior of certain critical aspects of the system at runtime. The concept of formally monitoring specific attributes during program execution is called runtime verification; where monitors are designed to detect or respond to attribute violations to provide a certain level of fault protection. In this paper, the ISO 26262 standard for functional safety of road vehicles is used to guide the definition of functional safety requirements; these specific requirements are mapped to logical formulas, and runtime monitors are developed to evaluate compliance with attributes at runtime. ISO 26262 is particularly relevant to this process as it imposes strict standards and emphasizes testing and verification throughout the product lifecycle (even in field deployment).

This paper focuses on the development of practical non-intrusive monitoring solutions without any additional instrumentation. Designers can leverage the understanding of such violations to modify the system to understand and eliminate these violations. However, it can be envisioned that such a monitor could be further used during the operational phase of the vehicle’s lifespan, but the system needs to respond in real-time to violations and take appropriate remedial actions, as mere detection of behavior does not guarantee safety.

The structure of this paper is as follows: Section 2 describes the ISO 26262 standard in the context of this paper; Section 3 summarizes some previous research work in this area; Section 4 describes the monitoring concept; Section 5 describes a case study example; Section 6 summarizes the conclusions.

Over the past two decades, the automotive industry has made significant advancements, including a surge in electrical and electronic (E/E) systems (including software) in vehicles and other road vehicles, as well as the relevant demand for improved safety standards. Given the role of E/E systems in modern automotive systems, these demands have prompted the International Organization for Standardization to develop the ISO 26262 standard to ensure that the design of automotive E/E systems meets stringent functional safety and development standards. The ISO 26262 standard adapts the earlier IEC 61508 safety standard for E/E systems used in road vehicles. A large number of companies involved in automotive electronic systems and software, including Freescale and Infineon, are developing technical solutions to achieve compliance with the standard, while organizations like SGS-TUV Saar are also strongly supporting the adoption of this standard.

This paper considers a well-known and previously researched automotive E/E system, namely the electronic transmission controller, in the context of the ISO 26262 standard. The ASIL (Automotive Safety Integrity Level) levels are derived, and the implications for the controller are introduced, particularly regarding the relevant software development processes.

2.1 Scope and Application of ISO 26262 Standard

The ISO 26262 standard does not specify any development processes or techniques; rather, it assumes that ISO 9001, Capability Maturity Model, or similar processes are already in place and imposes specific safety-related requirements and outcomes on them. However, the standard is very specific and requires the definition and refinement of project plans and safety plans with designated stages and required outcomes (referred to as work products in the standard). Figure 1 (based on Part 2 of the standard) illustrates the safety lifecycle envisioned by the standard.

ISO 26262 covers the entire lifecycle of automotive E/E products, focusing on product development. Part 2 covers the management phases, Part 3 is the “concept phase,” where safety requirements are established and the safety lifecycle begins. Parts 4-6 are the development phases: Part 4 addresses system-level development, Part 5 addresses hardware, and Part 6 addresses software. Part 7 of the standard covers production and operation, including decommissioning; Part 8 addresses supporting processes, including documentation and tool verification; Part 9 addresses safety analysis, particularly safety analysis related to ASIL-oriented requirements, while Part 10 covers application guidelines, including illustrative examples. ISO 26262 defines multiple safety integrity levels (ASIL) from A to D, with level D being the most stringent. These levels are based on hazard, risk, and controllability analysis and apply to projects under development.

ISO 26262 uses the V-model as a reference process model for product development, as shown in Figure 2.

The “Double V” indicates that the V model applies to both the engineering phases of the system and the hardware and software phases. Figure 3 shows the “V model” of the software development phase in more detail. Integration testing and verification are necessary parts of the standard-required product development cycle, where the software runs in target hardware and target vehicles in a testing environment.

As shown in Figure 3, software testing in the “V” model is conducted at multiple levels, namely “Software Safety Requirement Specification,” “Software Architecture Design,” and “Software Unit Design and Implementation,” along with the corresponding software testing and integration levels.

2.2 Application of ISO 26262 in the Development of Electronic Transmission Controller Software

This paper considers the development of an automotive subsystem (software-driven transmission controller). This programmable E/E system falls within the scope of the ISO 26262 process. The development of the transmission controller requires an overall project plan, safety plan, and functional safety concept, including the ASIL level that must be determined for the project.

As part of the product development activity, the transmission controller project will undergo functional safety assessments and functional safety audits, thereby defining a set of functional safety requirements for development based on ISO 26262 standard Part 3. Based on these analyses, a technical safety concept and a set of technical safety requirements can be derived.

The transmission control subsystem may cause significant harm, such as engaging the wrong gear or failing to engage. However, such violations do not lead to immediate death; therefore, for this example, a nominal ASIL C level is recommended. In practice, determining the appropriate ASIL level is an important task for the project team responsible for functional safety. Determining the ASIL level is the responsibility of the safety manager (as described in Part 2 of the standard). The safety requirements for ASIL C are very high, and Part 4 of the standard specifies safety and related non-safety aspects, including integration and testing elements of combined hardware and software systems. Part 6 of the standard specifies safety requirements for software development architecture design and software unit development. Figure 3 shows the relationships among them.

Of particular interest for our case study is the content of Part 6 of the ISO 26262 standard, which specifies the requirement to use at least semi-formal notation at the product software architecture design level and, correspondingly, the necessary means to detect errors at this level.

Like most embedded software development processes, ISO 26262 explicitly indicates a preference for testing “production-intended hardware and software” rather than using “instrumentation code” to support testing activities. This paper proposes that by instantiating hardware-resident monitors for non-intrusive software performance observation based on production code, the monitoring requirements for selected safety attributes specified by ISO 26262 can be satisfied.

The application of embedded systems in automobiles needs to ensure good functional operation of vehicles throughout their entire lifespan. Based on emerging runtime verification methods, selected proof obligations can be monitored during the design and testing phases of the development process, and even during the actual operation phase of the vehicle.

Why not design the right product the first time?

Research in software engineering has provided formal theories based on reliable semantic models that enable verifiable design or constructing correct designs. Program verification determines whether a program meets specifications. However, embedded program verification is generally unsolvable. Furthermore, the development of onboard systems is an extremely complex software development task. Application software must operate correctly in the target environment. Onboard electronic systems are exposed to harsh physical and electrical noise environments, and handling failures sometimes occurs only under extreme conditions. Other threats to correct program behavior in vehicles arise from combinatorial issues. Typically, multiple embedded system components will be integrated into a system-level solution, interacting through the vehicle control network, and this integrated system composition may trigger unforeseen behavior under certain specific circumstances (usually in the corners of specifications). Sometimes, such rogue software behavior can only be observed when the system executes in its full operational environment.

These issues are easy to understand, but the solutions are not complete. An important outcome of research on these issues has been the realization of the AUTOSAR (Automotive Open System Architecture) standard; an open, standardized software architecture for automotive E/E systems. AUTOSAR provides an infrastructure to assist in the development of in-vehicle software throughout the product lifecycle. However, although AUTOSAR suggests an open attitude towards software monitoring, they do not propose any formal methods.

How do we monitor system behavior?

The concept of monitoring system behavior at runtime has matured. However, to date, most proposed methods and schemes have focused on using monitors as tools for assessing runtime behavior. This paper proposes a monitoring solution to observe safety attributes that can be directly embedded into products. The monitor is programmable, can be implemented at a moderate cost overhead, and can do so without interfering with the hardware and software of the observed target.

What do we monitor?

Runtime verification monitors relate to monitoring program execution behavior to determine compliance with requirement specifications. The attributes to be observed can be determined by developers to instill confidence in the correct operation of the program. This paper is based on this concept, but the proposal here is specifically to observe safety attributes defined by the ISO 26262 guidelines to ensure that these attributes can be guaranteed during the actual operation phase of the vehicle.

Why monitor safety attributes?

The concept of mapping specific safety requirements from safety standards to runtime monitoring attributes has not been well explored in the research literature. Sections 7 and 8 of ISO 26262-6 address software architecture design and software unit design, both of which indicate that external monitoring techniques are one means of achieving compliance with ISO 26262, while Sections 9 (software unit testing) and 10 (software integration and testing) cover test case derivation suitable for monitoring methods. Currently, some less formal monitoring methods are used for certain ISO 26262-compliant microprocessors, such as Freescale, where coprocessors are often used to verify the functionality of the main processor rather than verifying the functionality of the software running on the processor.

Lifecycle and Design Phase Monitoring

Using non-intrusive monitoring solutions will greatly benefit safety managers, as it will provide evidence to ensure compliance with system safety requirements during actual vehicle operation. This again raises the question of whether such a monitor can be retained throughout the entire lifecycle of the vehicle to record any safety attribute violations. If this functionality is not further extended to enable real-time responses to violations and take appropriate remedial actions, the value of using such a monitoring method may be limited. The development of such a closed-loop solution raises further research questions; as the introduction of any new reactive control functionality adds additional safety concerns regarding the behavior of the reactive functionality itself. The authors view the proposed monitoring solution as a component of a future complete reactive behavior solution.

In this work, specific functional safety statements need to be mapped to properties that can be represented as logical formulas so that the runtime behavior of the system regarding such properties can be formally verified. To demonstrate this concept, a logic must be chosen; past-time LTL (ptLTL) was selected. LTL is a modal temporal logic that provides a powerful formalism for succinct specifications of complex systems, useful for verifying the behavior of software systems. A variant of LTL is “past-time LTL” (ptLTL), which uses temporal operators to refer to past states of execution trajectories relative to the current state. A key feature of ptLTL is that the satisfaction of a ptLTL formula can be determined by evaluating only the current state sn and its predecessor state sn−1.

ptLTL formulasψ use the following syntax

where AP is a set of atomic propositions representing predicates. Standard propositional binary operators such as ∧ (conjunction), ∨ (disjunction), etc., are used.

Past-time Operators

Basic past-time operators allow temporal reasoning, listed in Table 1a. Monitoring operators are derived from the basic operators to provide a more intuitive and compact presentation, as shown in Table 1b.

Section 5 will present a case study example based on the automotive transmission controller to demonstrate how to accurately express the natural language description of the desired behavior as ptLTL formulas. Figure 4 shows an example of the natural language description of the desired behavior condition that must apply when the clutch is opened.

Monitoring Timed Behavior

ptLTL logic does not support the concept of physical time. The monitoring circuit includes physical clock/timer circuits to calculate the passage of time with a resolution of 1 millisecond. Timer values are read in real-time and used as variables for logical evaluation.

4.1 Monitoring Architecture

A hardware-based runtime monitor is proposed to run the verification process in real-time throughout the product’s lifecycle. This solution monitors runtime events non-intrusively (modeling the AP) and views the trace of runtime events as finite words over AP and checks for inclusion in the language generated by the relevant ptLTL formula.

Figure 5 illustrates the abstract architecture of the on-chip monitoring system. The microcontroller executes the application program observed by the monitor. The application program is undetected, in the sense that the monitoring scheme is non-intrusive. The hardware monitor is connected to the system bus of the microcontroller. The microcontroller and monitor can coexist on a single SoC device, allowing the monitor to be instantiated within the integrated circuit structure. Some key components of the monitor in this scheme are as follows:

Filter – AP Evaluator

The filter component of the monitor is essentially a logical circuit used for instant evaluation of each AP. The filter circuit monitors the microcontroller system bus at the read/write cycle level, observes changes in the values of specified variables, and evaluates the appropriate AP accordingly. The filter has a logic unit to represent each AP. The filter outputs a set of APs, for example, the APs in Figure 4 include (FromGear ! = 0) and (GCTimer < 150).

Event Recognizer – State Change Update Event Detector

Evaluates the observed variables associated with events to determine the predicate judgments of these variables. The AP updates from the filter unit combined with the memory address bus reading values will define a unique event representing a state change. The output of the event recognizer will include the APs shown in Figure 4 and the identification of the latest state transition event.

Runtime Checker – Formula Truth Evaluation

The runtime formula checker operates such that for each event, the truth of the formula ψ can be determined to ascertain whether the trajectory t satisfies ψ(t⊨ψ) up to the current state. The ptLTL evaluator can be directly synthesized based on the algorithm proposed by Havelund and Rosu, described in hardware logic using a hardware description language. The “result” output can simply suggest which formula failed; for example, if the formula in Figure 4 fails, this would identify the failure of the clutch attribute. However, if the system is designed to respond to failures, this output will become input to further logic circuits to handle the required operation.

Independent Timer

Figure 5 shows that some hardware clocks/timers are included in the monitoring circuit.

4.2 Prototype Implementation

Figure 5 shows a monitoring architecture that meets both the mandatory and non-mandatory requirements of the ISO 26262 standard. In previous work, Heffernan et al. implemented a real-time runtime monitoring architecture based on non-formalized hardware logic, but in the current work, the formalized monitor is based on ptLTL logic. A structured approach is required to achieve non-intrusive access to internal program variables via the system bus. For this purpose, on-chip instrumentation (OCI) schemes can be used; for example, CoreSight is a debugging environment tailored for ARM processors. Fogarty et al. proposed an OCI architecture for real-time monitoring. Research has demonstrated a structured approach to instant access to internal program variables, where example applications are based on using the S12XDBG debugging module of the Freescale MC9S12XE microcontroller.

Reinbacher et al. proposed a complete hardware-based runtime verification scheme that supports the evaluation of ptLTL specifications. They reported that the size of the monitoring unit was approximately 367 logic units (Altera FPGA). However, in their scheme, each AP evaluation unit used about 290 additional logic units of space.

As a case study example, we explore an automotive subsystem to examine the feasibility of implementing a scheme that can demonstrate monitoring of some key attributes derived from ISO 26262 guidance on safety assessment. The prototype automotive gear controller design published by Lindahl et al. was selected as the example subsystem. Figure 6a shows the flowchart of the gear controller system. The GearControl component receives message requests through the interface component via the control network. GearControl initiates lower-level components by requesting services provided by the GearBox, Clutch, and Engine components. The system supports five forward gears, one reverse gear, and one neutral gear. The gear controller executes the gear shifting in five steps as follows: (i) achieve zero torque transmission, (ii) release the current gear, (iii) achieve synchronized transmission speed, (iv) set the new gear, (v) increase engine torque to achieve the previous wheel torque. Under special circumstances, when zero torque or synchronized transmission speed cannot be achieved for difficult driving conditions, the clutch may be allowed to open to facilitate gear shifting.

Figure 6. Gear Controller Diagram

a Flowchart

The transitions shown on the flowchart of Figure 6a represent the signaling or channels of the system. For a channel, two synchronous operations match at the enabled edge, where channel! represents output, and channel? represents input. In the implementation, each named signal represents a CAN message. For example, the interface module emits a CAN message named ReqNewGear with two parameters: FromGear and ToGear. Note that for the engine module, the normal operating mode is normal torque mode, and the engine module does not respond with an explicit signal when requesting this mode using the ReqTorque signal.

Table 2 shows a complete list of CAN messages derived from the transmission system design.

Figure 6b shows the implementation diagram of the system. The vehicle network is CAN. It can be seen that the monitoring circuit is located within the GearControl module, allowing direct access to the system bus of the microcontroller to observe the write updates of program variables, some of which represent CAN bus messages. It is not feasible to place the monitor directly on the CAN bus, as such a monitor would not have access to internal program variables. Additionally, failure of the CAN bus could lead to the failure of the monitor.

5.1 Program

Figure 6b shows the block diagram of the physical gear controller implementation. To simulate the real system, a module named “Other CAN traffic” was added to simulate other CAN message traffic on the bus. The CAN bus data rate is set to 125 kbps.

Note that to comply with the ISO 26262 standard, hardware design, development, and implementation work must be conducted according to Part 5 of the standard, which covers hardware development; for the purposes of this study, it is assumed that the hardware implementation meets the ISO 26262 standard.

The GearControl automaton in Figure 7 shows key timing features and signal interactions with other components. The model is a simple finite state machine that is extended by clock variables. The system is syntactically described by the timed automata model of Uppaal; a tool capable of modeling, simulating, and verifying real-time systems. The state machine model is automatically generated by the Uppaal tool, where the operation of the model is verified by the model checker based on text-based requirement specifications. The text-based specifications are represented in a simplified version of CTL, a well-defined machine-readable language.

Since the monitoring circuit can instantaneously evaluate each AP, the gear controller does not need to emit any special events or data information; therefore, the program design does not need to include any additional functionality for monitoring activities.

Monitoring Selected Attributes

As mentioned earlier, failures or faults in the gear controller may pose safety risks. While developing a detailed and comprehensive safety plan (which can be translated into a complete set of technical safety requirements) is beyond the scope of this case study, it is clear that certain specific failure cases (examples will be outlined below) may pose safety hazards for both system design (ISO 26262-4) and software design (ISO 26262-6). The functional safety plan will certainly include testing for such failure scenarios, which may be best handled through monitoring techniques.

Upon reviewing the specifications for the gear controller design, we can now translate some example safety statements or requirements into properties that can be represented as ptLTL formulas. The scope of the ISO 26262 standard fully allows for this approach.

In the experimental system, ten safety attributes were defined for the monitoring process. These attributes impact the entire system but are categorized in the documentation as follows:

· Two attributes related to forward gear shifting,

· Two attributes related to reverse gear shifting,

· Two attributes related to the interaction with the clutch,

· Two attributes related to the interaction with the engine,

· Two attributes related to the interaction with the interface (driver control).

Two safety attributes will be used as examples to illustrate how to write formal equations to represent the expected safety attributes. Consider the following safety requirement; note that the events in state Si are represented here as Esi.

Conditions for Opening the Clutch

As an example of a safety requirement, it stipulates that the system must absolutely guarantee that the clutch will not open normally but will only open under one of the following two conditions: (i) zero torque cannot be achieved within 250 milliseconds after a gear shift request, and the transmission is not in neutral, or (ii) the required synchronized speed is not reached within 150 milliseconds after the request event, and a move to a non-neutral gear is required.

Using information generated during program compilation/loading execution, symbols can be associated with memory addresses and used by the monitor. Thus, such high-level symbols can be used in specifications related to the conditions for opening the clutch as follows

Note that the example formula uses the program’s timer GCTimer, but this timer does not need to be trusted or can use the monitor’s own independent timer in the equation.

In this case study, state diagrams are used as part of the requirement specifications, as shown in the automaton in Figure 7; therefore, it may be more convenient to express the above specifications using formulas directly related to the state diagrams. The equivalent example formulas can now be represented in two parts, namely ψ1a and ψ1b, as follows

Another example of a safety requirement is that the system must absolutely guarantee that when a request is made to shift into reverse gear, the following conditions will be met: (i) the system will not attempt to request synchronized speed from the engine, and (ii) if the transmission is already in reverse, the system will not check torque or request or shift to neutral. The following formulas are used to satisfy the expression of this specification condition

The above formulas can be expressed using formulas directly related to the state diagrams. In this example, the formulas are divided into two parts, ψ2a and ψ2b, as follows

Test Plan

The entire set of tests is too detailed to describe here; therefore, only selected aspects of the testing will be introduced. The gear controller system (including the runtime monitor) is tested using a fault injection method. This testing method is recommended by the ISO 26262 standard documents. In the testing strategy, a set of tests is designed using the “programmatic violation for fault injection” method, while another set of tests is developed using the “simulating real faults for fault injection” method.

Fault Injection by Programming Violations

Fault injection by programming violations involves deliberately redesigning the observed program to operate in a way that violates specific requirements or certain specific requirements to confirm that the monitor can correctly identify error conditions. Table 3 lists two proven examples of violations that can be detected.

Fault Injection by Simulating Real Faults

A number of potential fault scenarios that could occur in practice are assumed, and some of these faults are injected into the system. For example, in real systems, the transmission may intermittently respond slowly; this is due to aging of its internal mechanical parts or lack of proper maintenance plans. This situation was simulated in the laboratory. A digital oscilloscope was used to bench test the transmission’s response to the OpenClutch! and CloseClutch control messages on the CAN bus. It was observed that timing violations due to any excessive delay in clutch operation can be correctly detected by the monitoring circuit. However, an interesting situation arose during testing; due to occasional excessive time delays in the “known-good” system setup, the monitor occasionally reported unexpected failure conditions.

Upon investigation, it was found that the unexplained marginal delays were due to the priority mechanism of message traffic on the CAN network. This was not anticipated in the system design analysis. This is a good example of how using such a runtime monitor can provide protection during design evaluation processes, helping to ensure correct system behavior. In this case, the designers of the individual components strictly implemented the design according to the requirement specifications, but in the actual system, when all components operate on a CAN-based network, marginal timing violations occurred due to combinatorial issues. The designers recalculated the CAN message priorities to resolve the violations. It can be said that in systems where control networks are based on a priority arbitration medium access control scheme, under noisy line conditions in automotive engine environments, messages can be resent after a failure occurs; such combinatorial issues may be difficult to foresee during design. Note that the hardware monitor itself has no impact on the CAN bus, as it does not directly monitor that bus.

Monitoring Overhead and Scalability

The experimental monitoring implementation is based on FPGA chips, as shown in Figure 5. The number of logic units required will be proportional to the number of APs the filter component needs to observe and proportional to the number of ptLTL formulas to be evaluated by the runtime checker, i.e., the number of safety attributes defined. For the number of safety attributes to be monitored, the overhead issue is based on the required size of on-chip space, rather than any required increase in the processing capabilities of the monitor. Furthermore, adding new safety attribute evaluators does not increase time consumption; because the evaluator circuits perform evaluations simultaneously. For the experimental monitoring circuit described in this paper, where the observed set of APs contains about thirty elements and defines ten safety attributes, the entire circuit is implemented in approximately 3500 gates of FPGA size. Significant reductions in gate counts can be achieved through more streamlined designs.

The addition of safety attributes involves inputting LTL formulas and generating synthesizable VHDL code that “checks” the input formulas. This code will then be processed in the synthesis tools of the target FPGA platform. Since the application and its main processor are not interfered with by the monitoring circuit, any expansion of monitoring functionality will not affect the resources of the application.

This study proposes a new concept of using the ISO 26262 standard for functional safety of road vehicles to guide the definition of product functional safety requirements, allowing for formal verification of the system’s actual runtime behavior throughout the product’s lifecycle. This paper provides a case study example of a transmission control system that demonstrates the feasibility of this approach. The observed system application software is not interfered with. The proposed monitoring scheme can be used to report safety attribute violations during the design phase and even throughout the entire lifecycle of the product.

Furthermore, it is demonstrated that the monitor can be constructed as a permanent function within the integrated circuit. In terms of circuit area, the overhead of such permanent monitoring circuitry is small. The monitoring circuit can be scaled to larger systems, where the size of the circuit will be directly related to the size of the AP set and the number of safety attributes to be monitored.

As microcontroller architectures (especially those employing multiple processing cores) become increasingly complex, the demand for reliable testing becomes more critical, while access to internal program operating functions becomes increasingly obscure. Widely used on-chip debugging architectures will facilitate more direct integration of monitoring circuits and allow for instant access to internal program variables.

The ISO 26262 standard aims to provide guidance for specifying appropriate requirements and processes to avoid systematic and/or random failures in automotive E/E devices. To help optimize the success of this important standard, novel design tools can be developed to translate the safety claims of the ISO 26262 safety teams into hardware safety monitors, potentially improving the reliability of automotive systems.

The monitoring solution proposed in this paper can be further developed for use throughout the entire lifespan of vehicles; designed to respond in real-time to safety violations and take appropriate remedial actions. However, vehicle design experience indicates that adding fault protection in systems often increases complexity, thereby increasing the likelihood of failures, and the fault protection logic itself may also lead to failures. Current work indicates that the use of runtime monitors is “safe” since no immediate reaction is taken upon failure. However, future challenges lie in developing closed-loop reactive schemes for real-time fault protection.