▲ Click on the above Leifeng Network to follow
The AI chip war among tech giants has spread from the cloud to the edge, which is a mixed blessing.
Written by | Bao Yonggang
The AI craze continues, and the battle for AI is naturally escalating. NVIDIA, as one of the most watched companies in this wave of AI, significantly influences the AI landscape. At GTC 2019 held in the United States last week, Jensen Huang extensively introduced NVIDIA’s advancements in AI software and computing power, but the Jetson Nano AI computer, priced at only $99 (approximately 664 RMB), became the focal point of attention. Earlier this month at the TensorFlow Developer Summit, Google also released the Edge TPU development board priced at $149.99 (approximately 1009 RMB).
This means that the AI chip war among tech giants has spread from the cloud to the edge, but why is this a mixed blessing?
The Cloud AI Chip War Descends
Although it is the most watched AI chip company, NVIDIA had a rough year in 2018, first suffering from high GPU inventory due to the mining crisis, and then being dragged down by lower-than-expected demand in the Chinese market and server market. Throughout 2018, NVIDIA’s market value shrank by nearly half. Therefore, against the backdrop of AMD’s early release of 7nm GPUs, there was heightened anticipation for NVIDIA to unveil its latest 7nm GPU at GTC 2019.
However, Jensen Huang did not release the latest 7nm GPU but instead spent a lot of time introducing RTX and CUDA-X AI.
CUDA-X AI integrates all of NVIDIA’s libraries. According to Huang, CUDA-X AI unlocks the flexibility of Tensor Core GPUs, enabling acceleration of machine learning and data science workloads by up to 50 times. Additionally, CUDA-X AI can accelerate every step of typical AI workflows, including training voice and image recognition systems with deep learning.
NVIDIA also announced that seven world-class manufacturers will launch servers based on the NVIDIA T4 GPU and NVIDIA CUDA-X AI acceleration libraries, all of which have been specially optimized for CUDA-X AI. Amazon AWS Vice President Matt Garman also announced that the latest EC2 G4 servers use NVIDIA T4 Tensor Core GPUs, which will be available in the coming weeks.
Although NVIDIA did not launch a more powerful GPU, it is enhancing the performance and appeal of its GPUs in the cloud through CUDA-X AI. Even so, Google’s important client, NVIDIA, has launched its own cloud AI chip, the TPU.
Since 2015, Google has been using TPU chips internally, publicly acknowledging the existence of TPU for the first time in 2016, and releasing the second generation TPU in 2017, with TPU 3.0 released in 2018. This means that Google’s relationship with NVIDIA in the cloud AI chip market has shifted from cooperative to competitive.
Leifeng Network learned that Huang expressed strong indifference to the threat posed by Google’s TPU. The competition between Google and NVIDIA in the cloud AI chip market is unlikely to reach a conclusion in the short term. However, it is clear that their chip competition has now extended to the edge.
The AI Chip War at the Edge
As an established chip giant, NVIDIA entered the edge computing market early, launching the Jetson series, including the Jetson AGX Xavier for fully autonomous machines and the Jetson TX2 for edge AI, but the prices of several hundred to over a thousand dollars have deterred many users. The high attention on the Jetson Nano, launched at GTC 2019, is primarily due to its price.
Looking at the development history of various industries, the explosion of the industry is not only due to technological maturity but also critically depends on product prices dropping to market-acceptable levels. The Jetson Nano computer launched at GTC 2019 has a surprisingly low price, compact appearance, but impressive performance. Reportedly, the Jetson Nano can achieve 472 GFLOPS (billion floating-point operations per second) with a power consumption of only 5 watts. At the same time, Jetson Nano supports high-resolution sensors, can process multiple sensors in parallel, and can run multiple modern neural networks on each sensor stream.
To meet different needs, NVIDIA has also launched two versions of Jetson Nano: one is a $99 developer kit aimed at developers, makers, and tech enthusiasts, while the other is a $129 production-ready module aimed at enterprises creating edge systems for the mass market.
NVIDIA Jetson Nano
Similar to NVIDIA’s Jetson Nano, Google released the Coral development board equipped with Edge TPU at the beginning of this month, priced at $150. The Coral development board features 1GB of LPDDR4 memory and 8GB of eMMC storage, running Mendel Linux or Android, capable of performing local offline computations with a peak performance of 40 trillion operations.
In addition to the Coral development board, Google also launched a $75 Coral USB Accelerator, which also contains an Edge TPU and can run on any 64-bit ARM or x86 platform with Debian Linux.
Google Edge TPU Development Board
Jensen Huang does not see Google’s TPU as a threat, but both giants seem to have a tacit understanding regarding the progress of low-cost products at the edge. First, Google launched the $75 and $150 development boards and accelerators equipped with Edge TPU. Shortly after, NVIDIA launched the $99 and $129 Jetson Nano.
Not only are the prices competing, but the target edge computing markets will also overlap. NVIDIA states that Jetson Nano can create millions of intelligent systems, targeting embedded applications such as network video recorders, home robots, and smart gateways with full analytical capabilities. NVIDIA aims to save time in hardware design, testing, and validation for complex, robust, and energy-efficient AI systems, shortening overall development time to bring products to market faster.
The Coral development board also emphasizes privacy, low latency, efficiency, and offline deployment for embedded devices. In terms of specific applications, Google showcased an interesting image classification application based on Coral. Google stated that it provides a simple API for executing image classification and object detection on Edge TPU devices. This means that Edge TPU is promising for image-related edge applications.
Therefore, from positioning, performance, applications, and pricing, Google and NVIDIA are once again in direct competition at the edge.
Popularizing AI or Revolutionizing AI Chip Startups?
The competition between Google and NVIDIA can promote AI development to a certain extent, especially at the edge. The Jetson Nano and Coral development boards significantly lower the difficulty of AI product development and accelerate time-to-market, providing existing AI application companies with more options, and certainly offering more portable choices for companies and individuals looking to innovate with AI, which has a positive significance for the popularization of AI at the edge.
However, this may be bad news for many AI chip startups. Leifeng Network previously reported that among 13 AI chip startups founded domestically, 11 have focused on the fields of autonomous driving and security, all targeting edge AI chips. The reason most AI chip startups have chosen the edge computing market is that in the cloud, Intel and NVIDIA hold an absolute advantage, making it very difficult for startups to succeed in this field.
While edge AI offers startups a larger market and opportunities, NVIDIA is also performing well in the autonomous driving sector. Now, with NVIDIA and Google both launching simpler, more user-friendly, and more affordable development boards, AI chip startups face two new competitors, and they are formidable ones.
What will make AI chip startups even more uneasy is that both giants have cloud AI chips, which can be combined with edge AI chips to achieve stronger competitiveness. Moreover, the important role of software in AI chips is increasingly recognized, and unfortunately, both NVIDIA and Google have strong software capabilities.
As mentioned at the beginning of the article, NVIDIA’s release of CUDA-X AI will enhance GPU AI performance, but at the same time, Jetson Nano is also an NVIDIA CUDA-X AI computer capable of running all AI models.
On Google’s side, along with the Coral development board, a cross-platform solution for mobile and embedded devices, TensorFlow Lite, was also released. This lightweight (Lite) framework helps deploy machine learning models on mobile and IoT devices. Google stated that after optimization with TensorFlow Lite, CPU performance reaches 1.9 times the original, with performance on Edge TPU improving by up to 62 times.
Leifeng Network believes that the giants possess AI chips from cloud to edge, along with powerful software to enhance hardware performance, as well as long-established advantages in branding, channels, and markets, which will not only promote the popularization of AI at the edge but also create competition with many AI chip startups.
However, there are still many uncertainties about how much NVIDIA and Google will impact AI chip startups in the future.
– END –
◆ ◆ ◆
Recommended Reading