This article is from:
Source:
https://medium.com/@stephen.gardner.ag/integration-of-nvidia-jetson-systems-in-industrial-networks-abb8a17f1745
When industrial networks utilize NVIDIA®’s Jetson™ Xavier system, it opens up a world of machine learning applications. Xavier is based on the successful Tegra system, which is the most powerful system in the NVIDIA® Jetson™ series. The performance of Xavier’s 8-core ARM processor is comparable to many systems, and the exciting part lies in its 512-core Volta GPU with 64 Tensor cores. This turns this compact module into a monster of digital computation. It can be used to detect patterns in signals or objects in images, operations that require extensive matrix computations. PLCs typically cannot achieve this level of computational performance, making Xavier a perfect complementary system.
Our benchmark application combines image processing and machine learning techniques to solve the parts picking problem. The image processing component uses OpenCV to identify the outer edges of parts. The inference leverages a retrained faster R-CNN initial model. Complete HD images can be processed in under 0.8 seconds, which is more than twice the time required by a desktop computer equipped with a GTX1080Ti graphics card. Given that Jetson’s maximum power consumption is 30 watts, which is far lower than that of a desktop, this is even more impressive.
The correct path to integrate such systems largely depends on the application. Possibilities include using MQTT brokers, OPC UA, or simple socket-based communication. We tested MQTT communication using a Beckhoff system and the OPC UA interface using a Siemens S7–1200 PLC. The benchmark application described here uses the latter.
Output image of the benchmark application:
The benchmark application requires the following functionalities:
-
Update full HD images from a USB webcam at 10Hz – up to 60Hz maximum
-
Regularly update the robot’s position to interpret images in robot coordinates (only needed if the webcam is mounted on the robot arm)
-
Mask the input image to fuse the edges of the box, which sometimes reflect parts
-
Use inference to detect sides A and B of the nuts (parts on side B must be flipped)
-
Accurately detect the edges of isolated nuts using OpenCV
-
Determine using OpenCV whether a nut can be grasped without colliding with surrounding nuts
-
Pass a series of potential targets with coordinates to the PLC via OPC UA
The above functionalities are orchestrated by a lightweight application server. This server includes an HTTP server that allows access to all configuration data, logs, and images. These functionalities are implemented as plugins, each providing access via a standard API that includes input and output images along with a series of potential targets.
One could argue that a complete application could be implemented using OpenCV, and many similar applications can be made. It may even have been proven to be a faster solution; however, the advantages of using inference in this application are very clear:
-
These models replace a lot of dedicated code with data generated from a standardized process, where training consumes images automatically generated from the geometric data of the nuts.
-
It has been shown that models used for inference are robust to changes in lighting, exceeding the previous experience of coded solutions.
Comparison methods
This list will be updated as soon as possible:
Our evaluation of Jetson™ Nano has been completed
Our model has been converted to TensorRT, which will be the subject of further blogs
Conclusion
This benchmark application does not claim to represent a perfect parts picking solution. It has not yet undergone rigorous optimization. It allows us to compare the performance of the development system with the measured performance of the Jetson™ system using the same code, input images, and models.
We hope that after converting our model to TensorRT, the performance of Jetson™ will further improve.
More about Xavier:
How should I choose the NVIDIA Jetson development kit?
Japanese innovative company develops AI skeleton detection engine supporting NVIDIA AGX Xavier
How much worse are Jetson NANO / Jetson Xavier NX / Jetson Xavier in performance?
NVIDIA edge computing tool AGX Xavier memory doubles, more importantly…
AI captain sets sail, NVIDIA AGX Xavier aids the autonomous version of the Mayflower across the Atlantic!