Click the blue text to follow us
AI is being widely applied in industry, accelerating the use of methods such as machine learning (ML), deep learning, and LLM. These advancements have led us to anticipate a large amount of data to be used at the edge. While the current focus is on how to accelerate the operation of neural networks, Micron’s drive is to manufacture sophisticated memory and storage for AI at the edge. This article explores how innovative storage and memory technologies will enable future innovations.
What is Synthetic Data?
IDC predicts that by 2025, the world will generate 175 zettabytes (1 zettabyte = 10 billion terabytes) of new data. These numbers are hard to comprehend, but AI advancements will continue to drive the development of data-scarce systems.
In fact, the growing AI models have been stifled by the vast amounts of real physical data obtained from direct measurements or physical images. If you have 10,000 ready-made images of oranges, it is easy to identify an orange. However, if you need specific scenes for comparison, it is difficult to confirm accurate results unless there are all different samples to create a baseline model.
The industry is increasingly using synthetic data. Synthetic data is artificially generated based on simulation models, for example, providing statistical realities of the same image. This approach is particularly applicable in industrial vision systems, where the baseline of physical images is unique, and there are not enough “widgets” available online to provide effective model representation.
The challenge is where these new forms of data will reside. Of course, any newly created dataset must be stored in the cloud or closer to where the data needs to be analyzed—at the edge.
Model Complexity and Memory Wall
Finding the optimal balance between algorithm efficiency and AI model performance is a complex task, as it depends on factors such as data characteristics and quantity, resource availability, power consumption, workload requirements, and more.
AI models are complex algorithms characterized by the number of parameters: the more parameters, the more accurate the results. The industry starts with a general baseline model, such as ResNet50, as it is easy to implement and has become the baseline for network performance. However, this model focuses on limited datasets and applications. With the development of these transformers, we see that the development of transformers increases the parameters of memory bandwidth. The result is a clear pressure: no matter how much data the model can handle, we are limited by the memory and storage bandwidth available for the model and parameters.
For a quick comparison, we can look at the performance of embedded AI systems measured in operations per second (TOPS). Here we see that AI edge devices below 100 TOPS may require about 225 GB /s of memory bandwidth, while AI edge devices above 100 TOPS may require 451 GB /s of memory bandwidth (see Table 1).
Therefore, one way to optimize the model is to consider providing high-performance memory with the lowest power consumption.
Memory keeps pace with the acceleration of AI solutions by continuously developing new standards. For example, LPDDR4/4X (Low Power DDR4 DRAM) and LPDDR5/5X (Low Power DDR5 DRAM) solutions have significant performance improvements over previous technologies.
Micron is providing industry-leading and persistent
Industry-leading LPDRAM suppliers
-
LP5 industry-leading – first to obtain FUSA certification
-
LP5X provides the highest performance with the lowest power consumption
-
LP4 industry-first to 1A node—reducing costs and extending lifespan
-
Dedicated foundries providing continuous product support
Extensive industrial and multi-market portfolio
-
Up to 128Gb LPDDR4 and LPDDR5
-
Extensive participation and compatibility with all major CSV partners
-
LP4 / LP5 is the preferred solution for AI chipset ecosystems
-
Compared to older technologies, LP4 / LP5 offers a higher temperature range: (e.g., industrial Ti = 100℃)
LPDDR4 can operate at up to 4.2 GT/s per pin and supports up to x64 bus width. Compared to LPDDR4, LPDDR5X has improved performance by 50%, with a doubled performance per pin of 8.5GT/s. Additionally, LPDDR5 has a power efficiency that is 20% higher than LPDDR4X (Source: Micron). These are significant developments to meet the growing demands of AI edge use cases.
What are the considerations for storage?
Thinking that computing resources are limited by the raw TOP or memory architecture bandwidth is insufficient. As machine learning models become increasingly complex, the number of parameters in the models is also growing exponentially.
Machine learning models and datasets scale for better model efficiency, thus requiring higher performance embedded storage. Typical managed NAND solutions, such as 3.2 Gb/s e.MMC 5.1, are not only ideal for code booting but also for remote data storage. Additionally, solutions like UFS 3.1 can run at speeds increased by 7 times, reaching 23.2 Gb/s, thus supporting more complex models.
New architectures will also typically push the functionalities of cloud computing or IT infrastructure to the edge. For example, edge solutions implement a security layer that provides a gap between restricted operational data and the IT/cloud domain. Edge AI also supports intelligent automation, such as classifying, tagging, and retrieving stored data.
Memory storage developments supporting 3D TLC NAND and NVMeTM SSDs provide high performance for various edge workloads. For example, Micron’s 7450 NVMe SSD utilizes 176 layers of NAND technology, making it very suitable for most edge and data center workloads. It has a 2ms QoS latency, making it ideal for the performance requirements of SQL server platforms. It also provides FIPS 140-3 Level 2 and TAA compliance with U.S. federal procurement requirements.
The growing AI edge processor ecosystem
Joint market research firms estimate that by 2030, the AI edge processor market will grow to 9.6 billion dollars. Interestingly, this batch of new AI processor startups is developing dedicated integrated circuits and proprietary ASICs to fit more space and power-constrained edge applications. When it comes to memory and storage solutions, these new chipsets also need to balance performance and power consumption.
Additionally, we see that AI chipset suppliers have developed enterprise and data center standard form factor (EDSFF) accelerator cards that can be installed in 1U solutions and installed alongside storage servers, capable of accelerating any workload using the same module—from AI/ML inference to video processing.
AI is no longer a hype but a reality being implemented across all verticals. In a study, 89% of industries have developed strategies or will develop strategies around AI in the next two years.
Original link:
https://www.arrow.com/en/research-and-events/articles/ai-at-the-edge-future-memory-and-storage-in-accelerating-intelligence
|
High-end WeChat Group Introduction |
|
|
Venture Capital Group |
AI, IoT, chip founders, investors, analysts, brokers |
|
Flash Memory Group |
Covering over 5,000 global Chinese flash and storage chip elites |
|
Cloud Computing Group |
All-flash, software-defined storage SDS, hyper-converged, and public/private cloud discussions |
|
AI Chip Group |
Discussions on AI chips and GPU, FPGA, CPU heterogeneous computing |
|
5G Group |
IoT, 5G chip discussions |
|
Third Generation Semiconductor Group |
Discussions on compound semiconductors such as GaN, SiC, etc. |
|
Storage Chip Group |
Discussions on various storage media and controllers such as DRAM, NAND, 3D XPoint, etc. |
|
Automotive Electronics Group |
MCU, power supplies, sensors, and automotive electronics discussions |
|
Optoelectronic Devices Group |
Discussions on optoelectronic devices such as optical communication, lasers, ToF, AR, VCSEL, etc. |
|
Channel Group |
Storage and chip product quotes, market trends, channels, supply chains |
< Long press to recognize the QR code to add friends >
Join the above group chats
Long press and follow
Take you into the information revolution era of storage, intelligence, and connectivity
of all things
WeChat ID: SSDFans