Author of this article:
This article is reprinted from: https://towardsdatascience.com/nvidia-jetson-nano-vs-google-coral-vs-intel-ncs-a-comparison-9f950ee88f0d
As the maturity of artificial intelligence training frameworks like TensorFlow, PyTorch, Caffe, Keras, and OpenVINO reaches a certain level, Edge AI gains momentum. However, the complete toolchain from data collection to model deployment and inference remains unclear. Despite being in the research stage, developments are rapid. Exciting solutions are continuously emerging, such as object recognition from computer vision and speech recognition from a natural language processing perspective.
Traditional AI and Machine Learning Approaches
Many existing AI solutions consider cloud computing or storage as fundamental components of their architecture. This makes it difficult for certain sectors to adopt the technology in practical use cases due to issues related to privacy, latency, reliability, and bandwidth. While edge computing is limited by resources, it can alleviate these issues to some extent. Edge computing and cloud computing are not mutually exclusive but rather complementary.
Future Development of Edge AI
The emerging technology trends shown in Gartner’s “Hype Cycle for Emerging Technologies, 2019” report indicate that expectations for Edge AI and edge analytics have peaked. However, since this field is still in its infancy, software frameworks and hardware platforms will continue to evolve over time to create value in a cost-effective manner.
Who’s Competing: NVIDIA, Google, and Intel
The three major players in the AI field; Intel, Google, and Nvidia support edge AI by providing small-sized hardware platforms/accelerators. While all three have their pros and cons, it ultimately depends on the application, budget, and availability of SDKs.
In this blog, I will briefly compare three edge AI hardware accelerators: Intel Movidius NCS stick, Google Coral USB stick, and Nvidia Jetson Nano.
Testing Setup
To conduct the detection comparison, the same environmental setup was considered. The test subjects included people, buses, and cars. The same lighting conditions were ensured. In the experiment, the hardware components included the NVIDIA Jetson Nano Developer Kit, 32GB Scandisk Ultra SD card, Microsoft USB LifeCam 1080p HD (webcam), Raspberry Pi official power supply 2.5A, Raspberry Pi camera, Google Coral USB, and Intel NCS.
Image provided by the author
Performance and Resource Utilization
Resource utilization was measured using the top Linux command. Inference time refers to the time taken to detect an object in a single frame, while CPU usage indicates co-processor usage. The frame rate of Intel NCS is lower; however, the performance of the second-generation Intel NCS2 can be up to 8 times better. The detection results represent the confidence score of detections.
Figure 01 shows the detection results of Nvidia Jetson Nano, Figure 02 shows Google Coral, and Figure 03 shows the detection results from Intel NCS. It is evident that NVIDIA and Intel NCS provide better confidence, while Google Coral’s confidence is relatively lower. One reason is the infrared thermal camera used for Google Coral, which leads to insufficient detection performance due to the mismatch of the thermal camera sensor. However, this indicates how many objects are used to estimate processing load, memory usage, and inference time calculations.
Cost
The table below lists the costs of the hardware accelerators along with the required components. Notably, the Nvidia Jetson Nano is a development board that can be used as a standalone device. However, Google Coral USB and Intel NCS require a host to process the data stream. The host can be a single-board computer like Raspberry Pi or any other x86 computer with a Windows or Linux operating system. The costs in the table are calculated using Raspberry Pi 3 B+.
This cost is for the prototype system (translator’s note: i.e., development kit), and each accelerator provider has hardware available for production (translator’s note: i.e., module). The cost of ready-to-use accelerators will depend on the batch and third-party products, as these modules need to be integrated into the host.
Conclusion
From a latency perspective, it is clear that the Nvidia Jetson Nano’s ~25 fps outperforms Google Coral’s ~9 fps and Intel NCS’s ~4 fps. For certain applications, considering the cost difference, exceeding 4 fps may also be a good performance indicator. The Nvidia Jetson Nano is an evaluation board, while Intel NCS and Google Coral are more supplementary devices that can be attached to existing hardware.
Using the above hardware, a PoC can be quickly developed. It is also worth mentioning that all three products have hardware accelerators available for production environments, featuring better temperature ratings and performance modes. For example, Google Coral has dynamic frequency scaling and adjusts the load based on the temperature of the accelerator.
The costs of these prototype hardware accelerators are relatively low and within the same range, making them suitable for various low-cost applications.
More:
How should I choose the NVIDIA Jetson development kit?
You promised to use Raspberry Pi together, but you secretly played with Jetson NANO.
Jetson NANO and Raspberry Pi 4B are really different!