Advanced Software Architecture Design for Embedded Systems: Part 2

For firmware, because it is in the middle of the system, the problem is more complex. On the one hand, the firmware must be callable through means other than the GUI; on the other hand, it must simulate the functionality of the hardware. Regarding the first point, during the design phase, one should separate the interface from the implementation. The interface can be changed freely; for example, the interface with the GUI might be JSON, while also providing telnet’s TL1 interface, but the implementation remains identical. Thus, before integrating with the GUI, the firmware can be thoroughly tested via TL1. For the second point, the hardware abstraction layer should be extracted during design, isolating the main implementation of the firmware from factors like registers and memory addresses. A hardware simulation layer can be implemented when there is no hardware or the hardware design is undecided, ensuring the firmware can run and be tested completely.

6.5. Unit Testing

Unit testing is the most basic unit of software testing, executed by developers to ensure the correctness of the code they developed. Developers should submit tested code. Code that has not undergone unit testing, once integrated into the software, not only makes it difficult to locate issues but also makes it hard to achieve complete coverage of code branches through system testing. TDD is a development model based on this level.

The granularity of unit testing is generally at the function or class level; for example, the following commonly used function:

int atoi(const char *nptr);

This is a function with a very single function, so unit testing is very effective for it. Unit testing can verify the following situations:

1. General normal calls, such as “9”, “1000”, “-1”, etc.

2. Empty nptr pointer

3. Non-numeric strings, “abc”, “@#!123”, “123abc”

4. Strings with decimal points, “1.1”, “0.111”, “.123”

5. Excessively long strings

6. Excessively large numbers, “999999999999999999999999999”

7. Multiple negative signs and incorrectly positioned negative signs, “–1”, “-1-“, “-1-2”

If atoi passes these tests, we can confidently integrate it into the software. The probability of it causing further issues is very low (not completely absent, as we cannot traverse all possibilities; we only test representative exceptional situations).

The above example can be considered a model for unit testing, but in practice, this is often not the case. We often find that well-written functions are difficult to unit test, not only requiring significant effort but also yielding poor results. The fundamental reason is that the functions do not adhere to certain principles:

1. Single function

2. Low coupling

In contrast to the example of atoi, which has a clear single function and almost no coupling with other functions (I have not written the implementation code of atoi; you can implement it yourself, hoping it has 0 coupling).

Now I will provide a practical example.

This is a simple TL1 command sending and parsing software, with the functional requirements described as follows:

ü Communicate with TL1 server via telnet

ü Send TL1 commands to TL1 server

ü Parse responses from TL1 server

TL1 is a widely used protocol in the communications industry. To simplify matters for those unfamiliar with TL1, I define a simplified format:

CMD:CTAG:PAYLOAD;

CMD - The name of the command, which can be any string starting with a letter and composed of letters and underscores.

CTAG - A number used to mark the command's sequence.

PAYLOAD - Can be any format of content.

; - End delimiter

Correspondingly, the TL1 server's response also has a format:

DATE

CTAG COMPLD

PAYLOAD

;

DATE – Date and time

CTAG – A number, matching the CTAG carried by the TL1 command

COMPLD – Indicates that the command was executed successfully

PAYLOAD - The result returned, which can be any format of content.

; - End delimiter

For example:

Command: GET-IP-CONFIG:1:;

Result:

2008-7-19 11:00:00
1 COMPLD
ip address: 192.168.1.200
gate way: 192.168.1.1
dns: 192.168.1.3
;

Command: SET-IP-CONFIG:2:172.31.2.100,172.31.2.1,172.31.2.3;

Result:

2008-7-19 11:00:05
2 COMPLD
;

The software’s top layer might look like this:

Dict* ipconf = GET_IP_CONFIG();
ipconf->set("ipaddr","172.31.2.100")
ipconf->set("gateway","172.3.2.1")
ipconf->set("dns","172.31.2.1")
SET_IP_CONFIG(ipconf);

Taking GET_IP_CONFIG as an example, the functionalities this function should accomplish include:

ü Establishing a telnet connection if it has not yet been established

ü Constructing the TL1 command string

ü Sending

ü Receiving feedback

ü Parsing feedback and copying it into the IP_CONF structure

ü Returning

We certainly do not want to repeat these functions for each such function, so we define several modules:

1. Telnet connection management

2. TL1 command construction

3. TL1 result parsing

Here we analyze TL1 result parsing, assuming it is designed as a function with the following prototype:

Dict* TL1Parse(const char* tl1response)

This function’s role is to accept a string, and if it is a valid and known TL1 response, extract the results and place them into a dictionary object.

This would normally be a very unit-test-friendly example: input various strings and check whether the returned results are correct. However, there is a very specific problem in this software:

TL1Parse must know which command’s response it is processing when parsing a string. But note that the TL1 response does not include the command name. The only way is to use CTAG, which is a number corresponding to each command and response. TL1Parse first extracts CTAG and then looks up which command corresponds to this CTAG. This introduces an external call, which is coupling.

An object maintains a table mapping CTAGs to command names, allowing the lookup of command names based on CTAGs, thus knowing how to parse this TL1 response.

As a result, TL1Parse cannot be unit tested, or at least not easily. Typically, the stubbing method is ineffective.

What to do?

Redesign to eliminate coupling.

Split TL1Parse into two functions:

Tl1_header TL1_get_header(const char* tl1response)
Dict* TL1_parse_payload(const char* tl1name ,const char* tl1payload)

Both of these functions can be independently and thoroughly unit tested. The code of these two functions is essentially a split of TL1Parse, but their testability is greatly improved, significantly increasing the likelihood of obtaining a reliable parser.

This example demonstrates how design can enhance the testability of code—here referring to unit testing. Arbitrary design and implementation of software will make unit testing a nightmare; only by considering the need for unit testing during design can true unit testing be conducted.

6.5.1. Cyclomatic Complexity Measurement

The complexity of a module directly affects the coverage of unit testing. The most well-known method for measuring code complexity is cyclomatic complexity measurement.

The calculation formula is: V(F)=e-n+2, where e is the number of edges in the flowchart, and n is the number of nodes. A simple algorithm counts the number of if, while, do, and switch case statements plus 1. Code complexity that is suitable for unit testing is generally considered not to exceed 10.

6.5.2. Fan-In and Fan-Out Measurement

Fan-in refers to the number of other modules that reference a module. Fan-out refers to the number of other modules that a module references. We all know that good design should be high cohesion and low coupling, meaning high fan-in and low fan-out. A module with a fan-out exceeding 7 is generally considered poorly designed. Modules with excessive fan-out are challenging to unit test, whether in terms of stubbing or coverage. Combining the number of outgoing and incoming couplings of the system forms another metric: instability.

Instability = Fan-out / (Fan-in + Fan-out)

This value ranges from 0 to 1. The closer the value is to 1, the more unstable it is. When designing and implementing architecture, one should try to rely on stable packages, as these packages are less likely to change. In contrast, relying on unstable packages increases the likelihood of being affected when changes occur.

6.5.3. The Significance of Frameworks for Unit Testing

The application of frameworks can greatly assist in unit testing. This is because secondary developers are limited to implementing specific interfaces, which must be clear, simple, and low-coupling. The previously provided example code for a framework also illustrates this point. This again emphasizes that frameworks designed by high-level engineers can compel junior engineers to produce high-quality code.

7. Maintaining Architectural Consistency

In actual development, it is common for code to deviate from the carefully designed architecture. For example, the following diagram illustrates an MVC pattern designed within an embedded device:

View depends on Controller and Model, Controller depends on Model, while Model serves as the underlying service provider, not depending on View or Controller. This is a suitable architecture that can largely separate business, data, and interface. However, a programmer might implement a call from Model to View, thereby breaking the architecture.

This phenomenon often occurs during the maintenance phase of a product and sometimes during the implementation phase of the architecture. To add a feature or fix a bug, programmers, due to misunderstanding the original architecture or simply taking shortcuts, may take a “shortcut”. If such implementations are not discovered and corrected in time, a well-designed architecture can gradually be undermined, which we refer to as “architectural decay.” Typically, an aging software product has this problem. How to monitor and prevent such issues involves both technical and managerial measures.

Technically, tools can be used to analyze the dependencies of system components, as the external manifestation of architecture is mainly the coupling relationships between various parts. Some tools can count the fan-in and fan-out of software components. Testing code can be written to check the fan-out of components, and if a test fails, it indicates that the architecture has been compromised. This check can be integrated into some IDEs, performed synchronously during compilation, or conducted during check-ins. More advanced tools can perform reverse engineering to generate UML, providing further information. However, checking fan-in and fan-out is usually sufficient.

By establishing a code review process, the code checked in by programmers can also be prevented from such issues. Code review is a very important part of development, serving as a crucial measure in the later stages of development to prevent bad code from entering the system. Code reviews typically focus on the following issues:

1. Whether the requirements have been correctly and completely fulfilled

2. Whether the system architecture has been followed

3. The testability of the code

4. Whether error handling is complete

5. Code standards

Code reviews are usually conducted in a meeting format, set at project milestones when code needs to be checked in. For iterative development, this can be organized before the end of an iteration cycle. Participants may include architects, project managers, project members, and senior engineers from other projects. Generally, the meeting should not last too long, ideally not exceeding 2 hours. Notifications and relevant document codes should be sent out about two days before the meeting, and participants must familiarize themselves with the content and prepare. During the meeting, the code author first explains the functionality that the code needs to implement and their implementation ideas. Then, the code is showcased. Participants can raise various questions and improvement suggestions based on their experiences. The most taboo aspect of such meetings is making the author feel blamed or belittled; thus, the meeting organizer should first define the tone of the meeting: the success of the meeting is not based on the quality of the author’s code but on whether the participants provided beneficial suggestions. After the meeting, the author rates the participants, rather than the other way around.

8. The Evolution of an Actual Embedded System Architecture

In the 1990s, the rapid development of the internet greatly advanced communication testing equipment. During that time, the ability to achieve certain measurements in hardware was the core of competition, and the purpose of software was merely to drive the hardware and provide a simple interface. Thus, the initial software structure of products was very simple, resembling the aforementioned urban rail gate control system.

Advantages: The program straightforwardly fulfills user needs, and a single programmer can handle everything.

Disadvantages: There is no module division at all, leading to severe coupling between the lower and upper layers.

8.1. Data Processing

Users requested the ability to save measurement results and reopen them. The data storage module and interface were separated.

The main logic above is maintained, but the interface can now not only display real-time data but also read data from ResultManager for display.

Advantages: The embryonic form of separating data and interface begins to emerge.

Disadvantages: ResultManager only exists as a tool, responsible for saving and loading historical data. The interface and data source are still tightly coupled, with different interfaces requiring different data being hard-coded.

8.2. Window Management

As functionalities became increasingly complex, the original reliance on a single class to draw various interfaces could no longer sustain. Thus, the concept of windows was introduced. Each interface was viewed as a window, with the elements within the window being controls. The opening, closing, and hiding of windows were managed by a window manager.

Advantages: Interface functionality is separated by window units, no longer forming a massive collection.

Disadvantages: Although a window manager has been established, the interface is still directly coupled with the underlying layers, maintaining a large loop structure.

8.3. MVC Pattern

As the scale further expanded, the initial large loop structure could no longer meet the increasingly complex requirements. The standard MVC pattern was introduced, leading to a significant restructuring.

The data center served as the Model, storing the most current data. The View was placed in an independent task, periodically polling data from the DataCenter. User operations were sent from the View to the Controller, which further invoked hardware drivers for execution. The results of the hardware execution were updated from the driver to the Controller and then to the DataCenter. The interface, data, and commands were essentially decoupled. ResultManager became a component of the DataCenter, with the View no longer communicating directly with it.

The introduction of the MVC pattern marked the first time that this product had a truly clear division of responsibilities and independent functionality in its architecture.

8.4. Numerous Similar Modules, Inefficient Reuse

At this stage, as a standalone embedded device, the architecture could basically meet the requirements. However, as the market expanded, more and more devices were designed. Although these devices performed different measurement tasks, they all shared the same operational methods, similar interfaces, and, more importantly, faced the same problem domain. For a long time, copying and pasting was the only method of reuse, with class names and variable names often going unchanged. An error in one device could be fixed, but the same code error in other devices might not be corrected in time. As team sizes grew, even the basic MVC architecture was not adhered to in some new devices.

Ultimately, a framework was introduced for this series of products. The framework established the following:

1. The basic architecture of the MVC pattern

2. Window manager and component layout algorithms

3. Multi-language solutions (string manager)

4. Logging system

5. Memory allocator and memory leak detection

8.5. Remote Control

Clients wished to position the device fixedly at a certain location on the network, using it as a “probe” and accessing it via remote control from the office. This posed a challenge for a system originally designed to be purely handheld. Fortunately, the MVC architecture exhibited considerable flexibility, and the early investments paid off.

The TL1 Server provides a remote control interface based on Telnet. Within the system, it functions similarly to the View, only communicating with the existing Controller and DataCenter.

8.6. Automated TL1 Interpreter

As TL1 commands became numerous, and TL1 was often not the client’s primary requirement, many devices’ TL1 commands started to become incomplete. The underlying reason was that manually writing TL1 command interpreters was too laborious. Later, the introduction of Bison and Flex improved this issue somewhat, but it was still insufficient. Automated code generation was introduced at this stage. By defining TL1 in the following format, tools could automatically generate TL1 encoding and decoding code.

CMD_NAME
{
  cmd = "SET-TIME-CONFIG::<ctag>::<year>,<month>,<day>,<hour>,<minute>,[<second>]"
  year = 1970..2100
  month = 1..12
  day = 1..31
  hour = 0..23
  minute = 0..59
  second = 0..59
}

8.7. Testing Challenges

After decades of accumulation, the product has become a series with dozens of devices. Most devices have entered the maintenance phase, frequently prompting clients to request minor improvements or defect fixes. The heavy manual regression testing has become a nightmare.

Automated testing based on TL1 has greatly liberated testers. Through test scripts running on PCs, regression testing has become simple and reliable. The only downside is that the interface portion cannot be validated.

Automated tools based on Test Quest require a similar remote desktop software to be developed on the device running the pSOS system, which is not an easy task on pSOS. However, the good news is that since the framework has fixed the style and layout algorithms of the interface, the automated tools based on Test Quest will have high recognition efficiency.

8.8. Summary

This actual reconstruction journey of embedded products illustrates that the introduction of the MVC pattern in the third step and the framework in the fourth step are crucial. The mature MVC pattern ensures a series of extensibility, while the framework guarantees the accurate reuse of this architecture across all products.

9. Conclusion

This article discusses the architectural design thoughts and methods tailored to the characteristics of embedded software development. It attempts to provide a mindset and inspire thought among readers. Frameworks, automated code generation, and test-driven architecture are core content, with frameworks being a consistent element throughout.

Someone asked me, what is an architect, and how can one become an architect? I replied: Code, code, and code again; correct errors, correct errors, and correct errors again. When you feel bored, stop and think about how to complete this work faster and better. Architects emerge from practice, and they come from those who think diligently and are lazy about repetition.

Follow me【Learn Embedded Together】, learn and grow together.

If you find the article good, click “Share”, “Like”, or “View”!