To validate the planning and construction methods of multi-level testing, the application of the proposed methods was studied in the actual functionality of gradually executing this series. The selected functionality is “Automatic Light Control,” which can demonstrate the entire example of our method, where there are interactions between systems and components. Any other functionality can be selected as long as the number of interactions between systems and components and the contribution of each component are known. The workflow of this sequence is as follows:

• Basic functional specification;

• System specification;

• Definition of system architecture and functional mapping within the system;

• Definition of functional parameters (Section 3.D of Series “One”);

• Definition of the required testing levels for the functionality;

• Definition of test sequences with suggested traceability.

A. Functional Specification

The description must be at a high level of abstraction and understandable without any technical knowledge of what the functionality is. In fact, this type of description is carried out by the sales and marketing teams that directly contact customers. The model requires information about the functional ID, name, and description, as shown in Table 10.

To prevent repeated switching, the automatic light control function has a 10% hysteresis. When external lighting is below the 60% threshold, darkness is detected, and the headlights turn on and remain on until lighting increases to over 70% (10% hysteresis), then the headlights turn off and remain off until lighting decreases to the 60% value. This is included in the functional test cases.

B. Functional Description

In this step, the systems involved in this functionality are identified. In the ALC function, two systems are required:

1) S1 represents the system that makes the driver interface, ignition signal, and reads the status of the light switch to detect when to select automatic light mode; 2) S2 identifies external luminance and controls the driver of external lights, as described in Table 11.

C. System Architecture and Functional Mapping

There are two main points in the system specification that need to be defined: the functional architecture within the vehicle’s electronic system and the contribution of each component to the functionality. Figure 18 shows the architecture of the functionality, and Table 12 shows the details of each system and component’s contribution. In this step, the information is still at an abstract level, but some basic information about vehicle systems and components is required, which is usually the activity of the technical team in the product development department.

The S1 system has two components involved in the functionality: C5 ignition control component and C9 control component of the light control knob. The S2 system has one component involved in the functionality, which is the C8 component responsible for controlling external lighting.

In Table 12, each part of the system is identified by a code that follows the interaction logic of the functionality, and each part of the system receives its participation description, referred to as functional contribution.

The functional coding definition of each contribution is unique and can be easily tracked and identified, allowing the summary of the code to have the “DNA” of functional contribution. Table 13 shows the mapping of functionalities within each system.

D. Definition of Testing Levels

Using all parameters defined in Table 13, considering that this functionality is related to functional safety, the workflow shown in Figure 12 can be executed to plan the tests, assessing the requirements of this functionality for each testing level. As shown in Figure 19, the result flows to the ALC function highlighted in red dashed lines. The first point evaluated in the flowchart is the complexity of the functionality, which is high in this case because the functionality is executed by multiple systems (S1 and S2). In the second point, the contribution at the system level is evaluated. For the case of ALC functionality, the combination between each system component results in the same outcome at the vehicle level. Therefore, the system-level assessment can be canceled, and only vehicle-level assessment is performed, which is mandatory since we are dealing with ASIL A classification. The last part of the flowchart assesses the necessity of customer testing. For ALC functionality, there is no complexity or interference from the customer that needs to be verified, so acceptance testing is not planned.

The results of the ALC workshop define the component-level and vehicle-level required to approve this functionality, as shown in Table 14.

E. Test Traceability

Test traceability is divided into two parts:

a) Identification of testing activities;

b) Alignment of test cases between testing levels in the V model.

1) Identification of ALC Function Testing Activities

In the proposed method, each testing activity must obtain a unique identifier based on the evaluation results performed in Section 6.D. The ALC function requires two testing levels: component and vehicle.

For the component testing activities, three components involved in the functionality must be considered. According to the method introduced in Section 5 F1 of Series “Two”, three main activities must be executed. Each activity at the component testing level receives a code starting with CT, added to the functional contribution code, generating the code CTFnSnCn. Using the information defined in Table 11, the testing activities for ALC functionality components include:

• In C5 – CTF3S2C5 component;

• In C9- CTF3S1C9 component;

• For C8 component, there is more than one contribution to the functionality, i.e., F3.1S2C8 and F3.2S2C8; in this case, the main activity is defined as the total participation of component CTF3S2C8.

At the vehicle level, two systems participate in the functionality S1 and S2. The identification defined in Section 6.D starts from adding the VT code to the functionality and the numbering of the involved systems, generating the code VTFnSnSn. Using the information defined in Table 11, the testing activities for ALC functionality components are VTF3S1S2.

Table 15 shows the codes for all activities of the ALC (F3) function. ALC function has four testing activities, three at the component level and one at the vehicle level.

2) Alignment of Test Cases for All Testing Levels

As mentioned in Section 5.F.2 of Series “Two”, the highest level test case serves as the basis for all other levels. In the case of ALC functionality, the highest level testing order is vehicle testing. In Table 16, one test case of the ALC functionality is used as the TCC at the vehicle level for this application.

At the vehicle level, testing is usually performed with all real sensors and actuators. Only external environments (in this case, lighting) can be simulated in test benches or real environment applications. Component-level testing can be divided into three parts, with the core part of the test coming from the C8 component, while the other parts are referred to as auxiliary components (AC) C9 and C5. The C8 component is chosen as the basis for TCC, adding ITA and OTA.

ITA and OTA adapt the TCC interface to the test interface by configuring the test cases for the test object, in this case, the C8 component. In the case of ALC functionality, Figure 20 illustrates the reuse of test cases at the component level, with descriptions of ITA and OTA.

OTA is needed to simulate the C5 auxiliary component by stimulating the CAN bus ignition and C9 component signals, and light switch signals. The adapter must follow the auxiliary component specifications to simulate the correct values, for example, when CAN messages are sent on the CAN bus. After defining ITA and OTA, component testing can be executed according to the test cases proposed in Table 15 without modifying their order.

Additionally, specific tests must be executed on C5 and C9 to validate the complete component-level testing of this functionality. Vehicle testing should only be performed after all components are approved at the component level to avoid identifying basic component failures at higher testing levels. In this application, the goal is to demonstrate the traceability method rather than define and discuss the content of test cases.

Table 17 lists the final codes for all ALC testing activities, considering the reuse of test cases in the C8 component, where only the letter RE is added in the C8 component testing to indicate that this test case is aligned with TCC.

The application literature’s recommendations on the traceability structure explain the identification of test case reuse in the code CTF3S2C8RE, defining the alignment with higher-level testing and establishing the procedural patterns and system connections for testing between levels, as defined in Section 5.F.2 of Series “Two”.

F. Analysis of Method Application

The application of the proposed method in the case of ALC functionality allows for the rapid initiation of testing planning and structured activities without technical deepening. All stages can be performed without detailed specifications for each component level, clearly showing how to describe the interactions between systems and components, which can be deployed in any other vehicle functionality, adopting the concepts of modularity and portability, considering that any functionality has two interaction levels, with clearly defined interfaces at the system and component levels, regardless of their complexity.

The method used to identify each part of the functionality is built systematically to map the functionalities within the embedded systems, facilitating the visualization of all parts of the vehicle system.

Automotive factors add a qualifier in the definition of testing levels for each functionality, including the programmatic, structured, and reasoning form that defines the testing strategy. This helps to discard redundant system-level testing in ALC functionality.

The traceability applied in the ALC functionality changes the complexity of structuring and mapping the interconnections between components and systems, providing control, management, testing execution, and result recording. In the first step, the ALC functionality is divided into three main contributions of the functionality, where each part has a unique code: F3S1C5; F3S1C9; F3S1C8. The composition of the code determines which functional, component, and system the role contribution belongs to. In the second step, the reuse of test cases using the method in the references and the method of encoding the activities proposed here systematically coordinates the testing activities.

This work presents a method for building and planning automotive embedded software testing. We define the relevant factors for testing and designing system methods. The method aims to define functional testing plans for the entire vehicle’s development and coordinate testing activities. We proposed this method based on the characteristics of the automotive field. However, it can be extended to other applications, following the same reasoning, detailing each application’s and environment’s requirements. In the proposed method, two points should be emphasized: 1) Structure and planning of testing; 2) Traceability of testing.

A. Structure and Planning of Software Testing

We developed a method to identify all electrical and electronic functional interfaces of vehicle systems and components. Starting from high-level functions, this method allows for mapping and building all vehicle functionalities. Each functionality is divided into its contributions, creating identifiers for each part of the functionality. It assigns appropriate roles for each component, displaying the correlation between components and systems. The proposed structure organizes embedded systems, facilitating visualization and understanding of the system, even when the system is complex and there are many interactions between components and systems.

One of the contributions of this work is to identify the factors that influence decisions to determine the testing levels required for the validation of each vehicle functionality. The identification of these factors allows for the creation of formal strategies to define all testing levels. This procedure enhances the efficiency of product development while evaluating all testing levels, eliminating gaps and redundancies, allowing for the determination of the exact testing plan for each functionality and customizing applications for each vehicle functionality.

One highlight is functional safety, which is standardized by ISO 26262 in the automotive industry, inserting a structured definition of the risk level of incidents for each functionality and applying the appropriate testing levels to mitigate or identify failures that may occur in each software functionality.

This method provides a structured advanced approach to identifying each part of the system, transforming complex systems into fully recognized and organized systems. Mapping functionalities by systems and components lays the foundation for planning testing strategies and systematically defining the required levels for each functionality.

The framework helps document the development activities required by ISO 26262, showcasing the robustness and transparency of all testing steps applied in product development.

The proposed method for determining the required testing levels for each functionality includes a programmatic, structured, and rational form to define testing strategies, aiding in identifying repetitions and gaps in testing planning.

In the testing levels of this method, acceptance testing is systematically included, creating a program to discuss customer involvement in testing software functionalities at the appropriate time.

All planning and structure proposed here can be defined at the start of the project. It only requires minimal information, without detailed specifications for components. This allows for early planning and quantification of testing costs. Thus, the project approval decision-making process becomes more robust and confident.

Since the costs and time required for testing each project are more precise, this helps decision-making when the costs and time required for software testing are critical factors in the development process.

B. Traceability

Traceability is a complex topic when discussing software development. Here, we bring it to a more practical level. We progressively organized the system view. We provide a systematic process to uniquely identify each system, component, and functionality. This helps allocate tasks for each functionality role. All high-level objects of embedded systems have received structured codes that become the DNA of each part of the system, making complex embedded systems traceable.

Two main contributions are proposed: a method for identifying all functionalities, i.e., functional contributions of systems and components; a method for linking components, systems, vehicles, and acceptance testing levels with test cases. In both cases, the proposal defines a unique code model containing the configuration identifiers of embedded complex systems in its structure. Additionally, the proposed coding method lays the foundation for identifying easily trackable documentation and registration of software testing activities, which can be used for future projects or field failure elimination.

The automotive field has become quite complex, but challenges are expected to increase, focusing on innovations such as autonomous vehicles, increasingly electrified cars, and the integration of connected solutions like the Internet of Things. In this context, external behaviors such as “hackers” in vehicle systems may pose challenges to future systems and may impact the planning of software testing. It is necessary to study and incorporate this vulnerability factor into the definition of the required testing levels for system approval planning.

The proposed method provides a structured basis for identifying and mapping embedded systems, with each part of the automotive system having a unique code. This can serve as a foundation for tracking and modeling future embedded system development and documentation tools, and this framework can also serve as a basis for creating an SW development database where all vehicle development and identification proposed here can be stored.