Click the blue text to follow immediately
Testing the Internet of Things (IoT):
When discussing the definition of “basic necessities of life,” most people would list traditional elements such as “food, shelter, and clothing.”
However, looking back a century ago, human society underwent significant evolution, leading to the emergence of new necessities of life. This evolution has made our lives more convenient, comfortable, and efficient.
For example, we no longer rely on traditional switches to control lighting, nor do we pay tolls through information kiosks; instead, we monitor health conditions through intelligent means and track vehicle movements more efficiently. There are countless such examples.
In light of the current situation, how should we respond? What are the essential differences in these changes?
First, we must deeply understand these changes before we can master how to conduct scientific testing on them.
IoT Testing
What is the Internet of Things (IoT)?
The Internet of Things refers to the use of embedded electronic devices, microchips, etc., to connect vehicles, home appliances, and medical devices to collect and exchange different types of data.
This technology allows users to remotely control devices over the network.
Examples of IoT
Some examples of IoT in practical applications include:
1
In wearable technology:
Wearable devices such as Fitbit bands and Apple Watches can easily sync with mobile devices. For instance, the recently popular Huawei Watch D2 is an IoT device.
This helps capture necessary information such as health, heart rate monitoring, and sleep activity. It also helps display data and notifications from mobile devices.
2
Infrastructure and Development:
Using applications like CitySense, it is easier to obtain real-time outdoor lighting data and turn streetlights on or off based on this data.
Additionally, various applications can be used to control traffic signals and parking space availability in complex urban layouts.
3
Healthcare:
There are many software applications available to monitor patients’ health conditions.
Based on basic data, these services can control medication dosages at different times of the day.
For example, there is software called UroSense that can monitor the fluid levels in a patient’s body and initiate fluid transfer when needed. Moreover, this data can be wirelessly transmitted to all relevant parties.
Technologies Used in IoT
Here are some of the most commonly used technologies in IoT:
-
RFID [Radio Frequency Identification] tags and EPC [Electronic Product Code]
-
NFC [Near Field Communication] for bidirectional interaction between electronic devices. This is primarily applicable to smartphones, mainly used for contactless payment transactions.
-
Bluetooth: This is used for short-range communication where sufficient distance is required. It is mainly used in wearable technology.
-
Z-Wave: This is a low-power RF communication technology, mainly used for home automation, lighting control, etc.
-
Wireless Networks: This is the most commonly used option in IoT. When on a LAN, it helps in seamless transmission of files, data, and messages.
Testing IoT
Taking a healthcare monitoring system as an example, this system monitors key physiological parameters such as patients’ health status, heart rate, and fluid intake in real-time and transmits the monitoring results to healthcare professionals in report form. All data is recorded in the system database for easy retrieval of historical data for analysis.
Based on this data, doctors can formulate corresponding medication treatment plans and fluid replenishment strategies. This process can be remotely controlled and operated through any terminal device connected to the medical equipment, including computers or mobile devices.
To test such an architecture, we need to employ various testing methods, such as:
1
Usability:
-
We must ensure that every device used here has usability.
-
The healthcare tracking devices used should have sufficient portability to be transferable to different medical fields.
-
The devices should possess enough intelligence to not only push notifications but also error messages, warnings, etc.
-
The system should have an option to log all events to provide clear information to the end user. If it cannot achieve this function, the system should also push events to the database for storage.
-
Notifications should be displayed on the device [computer/mobile device] and ensure they are processed correctly.
-
Usability should be thoroughly tested in terms of displaying data, processing data, and pushing job tasks from the device.
2
IoT Security:
-
IoT security issues: IoT is data-centric, and all connected devices and systems operate based on data.
-
When it comes to data flow between devices, data may be visible or readable by others during transmission.
-
From a testing perspective, we need to check whether there are protective measures or encryption when data is transmitted from one device to another.
-
As long as there is an interface, we must ensure it has password protection.
3
Connectivity:
-
Given its nature as a healthcare solution, connectivity plays a crucial role.
-
The system must always remain available and needs to achieve seamless interconnectivity with relevant stakeholders.
-
When evaluating connectivity, two key aspects need to be tested:
-
First, under normal operating conditions, the process of data transmission and receiving device job tasks should remain uninterrupted.
Secondly, the situation of connectivity interruption must be tested. Even if the system and network performance are strong, the system may still go offline.
As testers, we also need to verify the offline state. Once the system cannot operate normally on the network, an alert should be triggered immediately to notify doctors so they can begin manual monitoring of patients’ health instead of relying entirely on the system until it comes back online.
Additionally, the system should have a built-in mechanism to ensure that all data collected during offline periods is properly stored. Once the system reconnects, this data should be transmitted promptly to ensure data integrity is maintained under all circumstances.
4
Performance:
-
When we talk about systems in the healthcare field, we need to ensure that the system has sufficient scalability for the entire hospital.
-
During testing, data should be propagated for 2-10 patients and spread across 10-20 devices.
-
When the entire hospital is connected, and there are 180-200 patients connected to the system, the amount of data propagated is much larger than the data tested.
-
As testers, we need to ensure the system performs the same tasks even as additional data is propagated.
-
We should also test monitoring utilities to display system usage, power usage, temperature, etc.
5
Compatibility Testing:
-
Given the complex architecture of IoT systems, compatibility testing is essential.
-
IoT compatibility testing requires testing items such as multiple operating system versions, browser types and corresponding versions, device generations, communication modes [e.g., Bluetooth 2.0, 3.0].
6
Pilot Testing:
-
For IoT, pilot testing is a must.
-
Only testing in a lab can ensure that the product/system operates well. However, when exposed to real-time conditions/steps/scenarios, this may backfire.
-
During pilot testing, the system will be exposed to a limited number of users in the real field. They use the application to provide feedback about the system.
-
This feedback is useful in making the application robust enough for production deployment.
7
Regulatory Testing:
-
This is a healthcare system that needs to pass multiple regulatory/compliance checkpoints.
-
Consider a scenario where the product has passed all testing steps but fails the final compliance checklist [tests conducted by regulatory agencies].
-
The best practice is to meet regulatory requirements at the very beginning of the development cycle. The same should be part of the testing checklist.
-
Doing so ensures that the product is also certified as part of the regulatory checklist.
8
Upgrade Testing:
-
IoT is a combination of various protocols, devices, operating systems, firmware, hardware, network layers, etc.
-
When performing upgrades, whether for the system or any of the aforementioned items, thorough regression testing should be conducted/a strategy should be adopted to overcome issues related to upgrades.
Challenges in IoT Testing
Challenges faced by testers in IoT include:
1
Hardware-Software Mesh:
IoT is an architecture that tightly couples various hardware and software components. Not only do software applications constitute the system, but hardware applications, sensors, communication gateways, etc., also play a crucial role.
Functional testing alone cannot help fully certify the system. There is always interdependence in aspects like environment, data transmission, etc. Therefore, it becomes a tedious task compared to testing generic systems [only software/hardware components].
2
Device Interaction Modules:
Since this is an architecture between different hardware and software sets, they must communicate with each other in real-time or near real-time.
When they integrate with each other, issues such as security, backward compatibility, and upgrade problems will become challenges for the testing team.
3
Real-Time Data Testing:
As discussed earlier, pilot testing/regulatory testing is mandatory, and obtaining such data becomes very difficult.
As a member of the testing team, obtaining regulatory checkpoints or deploying the system in a pilot is very challenging. If the system is healthcare-related, as in our example, the steps become even more difficult. This remains a significant challenge for the testing team.
4
User Interface:
IoT is distributed across devices belonging to every platform [iOS, Android, Windows, Linux]. Currently, testing can be done on devices, but testing on all possible devices is nearly impossible.
We cannot ignore the possibility of accessing the UI from devices we do not have or simulate. This is a challenging hurdle to overcome.
5
Network Availability:
Network connectivity plays a crucial role as IoT aims to transmit data at faster speeds. The IoT architecture must be tested against various network connections/speeds.
To test this, virtual network simulators are primarily used to change network load, connectivity, stability, etc. However, real-time data/network always presents a new scenario, and in the long run, the testing team does not know where the bottlenecks will develop.
IoT Testing Tools
Various tools will be used during IoT system testing:
1
Software:
-
Wireshark: This is an open-source application used to monitor traffic such as interfaces, source/target host addresses, etc.
-
tcpdump: This works similarly to Wireshark, except it does not have a GUI. It is a command-line utility that helps users display TCP/IP and other packets transmitted or received over the network.
2
Hardware:
-
JTAG Debugger: This is similar to a debugger in PC applications. It helps debug target platform code and step through variables.
-
Digital Storage Oscilloscope: This is used to check various events with timestamps, power glitches, and signal integrity checks.
-
Software Defined Radio: This is used to simulate receivers and transmitters of various wireless gateways.
In developing countries around us, the IoT market is in a continuous growth phase, containing numerous opportunities. It is expected that in the near future, IoT will become a key factor for testers to survive in these countries.
Gadgets, smart device applications, and communication modules supporting IoT play a crucial role in researching and evaluating the performance and behavior of various IoT services.
If there are design flaws in IoT devices and services, it may hinder the smooth operation of applications, adversely affecting the end-user experience.
Conclusion
The testing methods for the Internet of Things (IoT) may vary depending on the systems or architectures involved. Testers should place greater emphasis on user testing methods, namely Testing as a Service (TAAS), rather than testing solely based on requirements.
Another key aspect of IoT testing is integration testing. If the integration testing plan is designed accurately and has strong coverage, effectively identifying defects in the system, it can be considered that IoT testing has been successful.
IoT testing can be a highly challenging task; however, for testing teams, the process of certifying such a complex network of devices, protocols, hardware, operating systems, firmware, etc., is also filled with exciting challenges.
END
Disclaimer: This article is a user submission from 51Testing Software Testing Network M&T. The user has committed to independently assume all legal responsibilities related to intellectual property when submitting this content and has assured 51Testing that this article does not contain plagiarized content. The purpose of publishing this article is solely for learning and communication, not for any commercial use. Unauthorized reproduction is prohibited; otherwise, the author and 51Testing reserve the right to pursue responsibility. If you find any content in this public account that is suspected of plagiarism, please send an email to: [email protected] to report it, and relevant evidence will be provided. Once verified, the suspected infringing content will be deleted immediately.
Daily Prize Interaction
What IoT products have you tested?
What do you think is the most challenging issue?
