As the demand for AI processing grows across various applications, including widespread demand for low-power chip applications in consumer and enterprise devices, edge AI inference is receiving increasing attention. Most attention is focused on optimizing the neural network processing engines of these smaller components and the models needed for their operation, but optimization has broader implications in many cases. In image recognition use cases, images must come from somewhere, typically from sensors with MIPI interfaces. Therefore, it makes sense for Perceive to integrate Mixel’s low-power MIPI D-PHY IP into its latest Ergo 2 Edge AI processor, bringing on-chip images for AI inference.
Resolution and frame rates are on the rise
AI processors have been enhanced and can now process larger images at higher frame rates through high-resolution sensors. The ability to quickly infer and make decisions before the scene changes in real-time is crucial. Given this, the image processing module in Ergo 2 has received significant attention.
Diagram of the Ergo 2 Edge AI processor system, provided by Perceive
Large images with a lot of pixels present a huge challenge for device developers. In a sense, the term image recognition is somewhat misleading. Most use cases where AI inference adds value require looking at a region of interest, or several such regions, where relatively few pixels are wrapped in a much larger image filled with mostly uninteresting pixels. The ability to discover those areas of interest faster and more accurately will determine the application’s performance.
The Ergo 2 image processing unit features dual simultaneous pipelines that can isolate pixels of interest, making it easier for AI models to process perceptions. The first pipeline supports four regions of interest, with a maximum image size of 4672 x 3506 pixels, at 24 frames per second (fps). The second pipeline can target a single region in 2048 x 1536 pixel images at 60 fps. The IPU can also handle tasks across the image range, such as scaling, range compression, rotation, distortion, and lens shading correction.
Lost frames can affect perception
In these fast, high-resolution images, excessive noise or jitter can lead to frame loss due to data errors. Missing frames in the image stream may affect the accuracy of inference operations, leading to missed or incorrect perceptions. Reliable image transmission in challenging environments is a prerequisite for accurate perception at the edge.
A typical feature of the Mixel MIPI D-PHY IP is its clock forwarding synchronization link, providing high resistance to interference and high jitter tolerance. In Ergo 2, there are three different MIPI IP solutions in operation: four-channel CSI-2 TX, two-channel CSI-2 RX, and four-channel CSI-2 RX. Each IP block integrates a transmitter or receiver and a 32-bit CSI-2 controller core. Link speeds of up to 2.5 Gbps are supported, as shown in the typical eye diagram below.
The success of the first step is critical
Defects in large SoCs are serious, and redesign can be very costly. However, larger SoC projects often have larger design teams, longer timelines, and higher budgets. On a smaller chip, a single failure can kill a project, and the costs of debugging and restarting can quickly exceed the initial development costs.
While first-time success is not guaranteed in the semiconductor industry, Perceive has been able to achieve this through Mixel IP. Mixel supports Perceive’s compliance testing, allowing the complete integrated design to withstand rigorous MIPI interface characterization before the SoC transitions to mass production. Mixel MIPI D-PHY IP includes driver front and driver back loopback and built-in self-test functions for testing the transmit and receive interfaces.
The result of Perceive integrating Mixel’s MIPI D-PHY IP is that the Ergo 2 meets its power, performance, and cost targets. In turn, Perceive’s customers can implement Ergo 2 on smaller, power-constrained devices where battery life is a critical metric, but AI inference performance must not be compromised. This is a great example of how well-designed integration brings images on-chip for AI inference, helping to save costs at the small system level.
Original link:https://semiwiki.com/ip/mixel/325762-mipi-d-phy-ip-brings-images-on-chip-for-ai-inference/
|
High-end WeChat Group Introduction |
|
|
Startup Investment Group |
AI, IOT, chip founders, investors, analysts, brokers |
|
Flash Memory Group |
Covering over 5,000 global Chinese flash memory and storage chip elites |
|
Cloud Computing Group |
All-flash, software-defined storage SDS, hyper-converged, and other public and private cloud discussions |
|
AI Chip Group |
Discussion of AI chips and GPU, FPGA, CPU heterogeneous computing |
|
5G Group |
IoT, 5G chip discussions |
|
Third Generation Semiconductor Group |
Discussions on compound semiconductors such as GaN and SiC |
|
Storage Chip Group |
Discussions on various storage media and controllers such as DRAM, NAND, 3D XPoint |
|
Automotive Electronics Group |
Discussion of MCU, power supplies, sensors, and other automotive electronics |
|
Optoelectronic Devices Group |
Discussion of optical communication, lasers, ToF, AR, VCSEL, and other optoelectronic devices |
|
Channel Group |
Storage and chip product quotes, market trends, channels, and supply chains |
< Long press to identify the QR code to add friends >
Join the above group chat
Bringing you into the new era of information revolution of storage, intelligence, and connectivity
of all things
WeChat ID: SSDFans