Embedded systems are becoming smarter, more interconnected, and certainly more complex than ever before. To keep embedded systems robust and as error-free as possible, development teams need effective methods for testing and validating that the system works as intended. The most critical and often the most challenging aspect of testing is running software on microcontrollers.
This article will introduce developers to how to use these new technologies, as well as the equipment and tools required for implementing these new technologies.
Changing Requirements for Embedded Testing
Modern embedded system testing tools require developers to adopt the following four main components to fully test their systems:
-
Debuggers with trace capabilities
-
Communication adapters/sniffers
-
Logic analyzers
-
Analog-to-digital converters (ADC)
With these four components, developers can test embedded software at both the system level and the microcontroller level, and even drill down to the instructions executed by the microprocessor.
This is crucial in today’s development environment, aimed at ensuring that the built system not only meets requirements but also operates reliably.
At first glance, it may seem similar to very traditional embedded system testing tools, but the advancements and new features truly stem from a new way of analyzing what is happening inside the microcontroller, known as “deep insight analysis.”
Using Deep Insight Analysis to Test Software
Deep insight analysis allows developers to analyze the system at runtime. Deep insight analysis has three key components:
-
RTOS-aware debugging
-
Runtime analysis
-
Profiling and code coverage analysis
Developers typically try to understand how the system works using basic debugging techniques (such as breakpoint debugging) after designing the application, and then immediately start testing. Testing with breakpoints only scratches the surface and does not allow developers to truly understand what is happening inside the microcontroller.
With deep insight analysis, developers can delve deeper into RTOS, runtime behavior, execution analysis, and coverage beyond basic testing and debugging.
To add deep insight analysis capabilities to testing tools, developers must use specialized debugging tools such as Segger Microcontroller Systems, J-Trace, or J-Link Ultra+.
J-Link Ultra+ uses standard JTAG or SWD interfaces to extract trace data from the onboard debugging module. This information can be used to perform various analyses, such as RTOS-aware debugging.
Using RTOS-aware debugging, developers can monitor task execution while executing test cases. For example, developers can gain insights into:
-
Maximum stack usage
-
Task run counts
-
Task states
This view shows a test session example using Segger’s embOS RTOS and Embedded Studio. This view allows developers to gain insights into the operation of the RTOS but does not provide complete information about the application’s execution.
Developers can further enhance testing tools using Segger’s free SystemView utility or Percepio’s Tracealyzer tool. These tools provide developers with runtime analysis so they can visually view and analyze the application’s execution while running test suites.
To collect this trace data, developers need to configure their applications to enable tracing within the IDE or use configuration tools for tracing. If tools are not used, they can manually integrate low-level libraries.
Runtime analysis can provide developers with rich information about application execution. For example, developers can:
-
Track the timing and order of events
-
Obtain maximum, minimum, and average execution times
-
Visually see task execution and when tasks switch
-
Monitor CPU load
-
Analyze task statistics
-
Identify potential issues such as priority inversion, task jitter, and deadlocks
While adding RTOS-aware debugging and runtime analysis capabilities to testing tools is beneficial, it may still not be enough. In many instances, bugs may be hidden in code that is never executed during testing. For developers, understanding which lines of code have been executed can be very challenging. In this regard, tools like J-Trace provide significant convenience.
How to Track Executed Code
J-Trace uses the embedded trace macrocell (ETM) port in the microcontroller to perform instruction tracing. Through instruction tracing, J-Trace can “see” every CPU instruction executed on the processor and the exact path of the code.
Using this analysis in testing tools, developers can determine whether test cases achieve 80%, 90%, or 100% code coverage.
If test coverage is only 95%, but 100% coverage is required for product delivery, they can use free utilities like Ozone to see which lines of code have been executed and, more importantly, which lines have not been executed.
New test cases can then be added to ensure these missing lines of code are executed during testing.
With the powerful tools provided by deep insight analysis, developers can now freely focus on building other components necessary for effective testing tools. Another critical component of testing tools is the tools used to communicate with the microcontroller and control its behavior during testing.
Command and Control of Embedded Systems
Each embedded system has different requirements for how to interact with the outside world. Some devices may communicate via simple UART, while others may use CAN or TCP/IP.
To successfully build testing tools that can communicate with the system and execute commands, developers must incorporate communication hardware and software into the testing tools.
While embedded systems use a variety of different communication interfaces, one interface is more commonly used than any other: UART. It is important to include this interface in testing tools for embedded software developers for several reasons, including:
-
For debugging information, such as printing messages (although this information should be sent via the debugger)
-
To issue commands to devices
-
To monitor internal communication between multiple devices
-
Ease of use
One common UART tool that every developer should have in their lab is the BOB-12731 FT232R USB-to-UART evaluation board provided by SparkFun Electronics.
These low-cost evaluation boards can easily connect to any embedded system and serve as a simple communication port on a PC. They do not require any special drivers or software to communicate with the embedded system. Developers just need to open the COM port to start sending and receiving test messages.
Verifying Each Logic State
To fully test an embedded system, developers need to verify the internal workings of the microcontroller as well as the external logic it produces. This logic may include simple input and output states, as well as low-level communications such as I2C or SPI.
However, monitoring input/output states and low-level communications can be costly if developers use analog-to-digital converters (ADC) and digital-to-analog converters (DAC) boards. Developers can utilize some tricks to reduce costs while monitoring these signals and improving testing capabilities.
The first trick is to use a development board from the microcontroller on the system to monitor the microcontroller pins. For example, if developers are using STMicroelectronics STM32F767 or STM32L4 microcontrollers, they should first purchase the STM32F767 Nucleo board or STM32L476RGT6 Nucleo board.
Then, they should use the pins on the board to directly jumper each I/O pin to the corresponding pin on the system. Since they have already developed low-level drivers for the microcontroller, they can easily adjust these drivers to monitor the input and output states of the microcontroller.
Developers can also add a small amount of additional code, such as a USB driver, so that the development board can be directly plugged into the test host.
USB can be used to receive input/output sample status data, which can then be correlated with commands sent and received from the system to determine whether the operation is fully as expected.
In addition to using development boards, testing tools can also utilize logic probes, such as the Logic Pro 8 from SparkFun.
These logic analyzers are versatile and can be modified via software for each input to monitor the desired objects.
For example, developers can set the first two inputs to monitor switch inputs, while using the next two inputs to monitor I2C communication, and the remaining inputs to monitor SPI. Data can be easily collected and then synchronized with the remaining components of the testing tool, providing a comprehensive understanding of the embedded system’s operation.
Tips and Tricks for Building Testing Tools
Clearly, testing tools are essential for many modern embedded systems. Achieving certification can sometimes be very challenging, but the returns from certification and the value of increased system robustness far outweigh the investment costs.
When initially building testing tools, or even when upgrading testing tools, developers can leverage various tips to ensure they build the most effective testing tools. These include:
-
Using development kits with the same processor as the primary target to monitor the microcontroller’s digital inputs and outputs
-
Investing in debuggers with trace capabilities and utilizing free software packages for the deepest insights into system operation
-
When running software traces, be sure to use worst-case testing to ensure you can capture worst-case scenarios
-
If there are insufficient funds to build a complete testing tool, start with a smaller testing tool and improve it over time. Even conducting a portion of the tests is better than not testing at all.
-
Take the necessary time to master the different tools and components to be used in the testing tool
-
Do not hesitate to build your own interfaces and make full use of existing software to monitor system operation
-
Do not make any assumptions! If you are not monitoring outputs or triggering inputs, it is likely that errors will slip through.
Conclusion
Developing testing tools for embedded systems is a low-cost method to improve the reliability of embedded systems. Carefully selecting components for testing tools allows developers to easily monitor the external behavior of software.
The most critical yet often overlooked aspect of testing is checking trace data, which is now easily accessible from microcontrollers.
Using this trace data, developers can perform deep insight analysis while executing their test cases to ensure their software works as expected, even down to individual instructions.
1. Learning programming at 35, microcontroller programming solved my “immediate concerns”!
2. They invented a programming language called Rockstar for rock stars
3. How to implement variable-length data packet reception in MCU?
4. Give you a network cable to pry open embedded Linux!
5. Analog or digital circuits, how to choose for machine learning?
6. Common software for power circuit design, which one is suitable for your application?
Disclaimer: This article is a network reprint, and the copyright belongs to the original author. If there are any copyright issues, please contact us, and we will confirm the copyright based on the materials you provide and pay for the manuscript or delete the content.