A Brief Discussion on AIoT Chip Architecture 2025

01 Heavy Computing Group

The “Transistor Hoarder” and “Master of Restraint” in the Linux Domain

This group’s battlefield is Edge Linux. The core focus is: should we choose to crazily stack transistors for community celebration, or deliberately cut performance for industrial reliability?

🦖 Rockchip RK3588

A performance monster raised by “2D” and “geeks”

🛠️ Hardware Specifications

• Process Technology: 8nm LP (Samsung) 111.

• CPU Architecture: Octa-core heterogeneous.4× Cortex-A76 @ 2.4GHz (high-performance cores) + 4× Cortex-A55 @ 1.8GHz (energy-efficient cores) 2.

• GPU Core: ARM Mali-G610 MP4, “Odin” architecture, supports OpenGLES 3.2 / Vulkan 1.2 / OpenCL 2.2 3.

• NPU Engine: 6 TOPS (INT8), tri-core architecture (RKNN NPU 3.0). Supports mixed precision (INT4/INT8/INT16/FP16), supports multi-task parallelism or single-task splitting 444.

• Memory Subsystem: 32GB maximum capacity, supports LPDDR4/4x/5, 64-bit width 555.

• Multimedia Pipeline: 8K@60fps H.265/VP9 decoding, 8K@30fps H.265/H.264 encoding 6.

• Super I/O: PCIe 3.0 x4 (supports NVMe SSD), 3× PCIe 2.1, 3× SATA 3.0, dual Gigabit Ethernet (RGMII) 7.

🗣️ Community Feedback (The Street Verdict)

• Polarized Reputation: In the SBC (Single Board Computer) community (like Orange Pi 5, Radxa Rock 5), it is the “God of Cost Performance”, as it provides the interface capabilities of a $300 competitor for just $100. However, in the industrial sector, it is“version hell”.

• Armbian Forum Feedback: Users generally report that the GPU driver (Panthor/Panfrost) for RK3588 is progressing slowly in the mainline Linux kernel. Current 3D acceleration heavily relies on the Legacy Kernel (5.10/6.1 BSP) provided by Rockchip, and once users want to upgrade to the latest Linux 6.x for server environments, the VPU (video hardware decoding) and NPU often fail.

• Github Pain Points: Issues regarding <span><span>rknn-toolkit2</span></span> are filled with complaints about“operator alignment”. The same YOLOv8 model runs on v1.4 SDK, but after upgrading to v1.6, the accuracy suddenly drops, or after quantization, there are inexplicable bounding box drifts 8888.

🏗️ Real Application Scenarios and Barriers

• Scenario A: Large Model Inference on the Edge (LLM)

• Feedback: The only cheap board that can run LLaMA-2-13B, relying on its high bandwidth 64-bit LPDDR5.

• 🚧 Barrier: Bandwidth Bottleneck. Although it supports LPDDR5, when the CPU/NPU/GPU simultaneously compete for the bus, the memory bandwidth saturates instantly. In tests, an 8B model only achieves 4 tokens/s, which is not only a computing power issue but also amemory latency problem 9.

• Scenario B: Multi-channel Video Analysis Box

• Feedback: Can decode 32 channels of 1080P video simultaneously, a “universal tool” in the security field.

• 🚧 Barrier: Cooling Design. Full-load power consumption is 12W+, requiring active cooling fans, which limits its application in closed industrial enclosures.

🛡️ NXP i.MX 93/95

The “Expensive Safe” in the Industrial Sector

🛠️ Hardware Specifications

• Process Technology: 16nm FinFET (speculated, based on energy efficiency performance) 11.

• CPU Architecture: 2× Cortex-A55 @ 1.7GHz (application core) + 1× Cortex-M33 @ 250MHz (real-time core) 121212.

• NPU Engine: Arm Ethos-U65 microNPU, computing power 0.5 TOPS. This is Arm’s official micro NPU IP, not developed by NXP 13131313.(Note: The high-end i.MX 95 uses a self-developed Neutron NPU).

• Memory Subsystem: Supports LPDDR4X-3733 / LPDDR4-3200, 16-bit width (with inline ECC) 14.

• Security Island: EdgeLock® secure area, compliant with ASIL-B automotive functional safety standards 15.

🗣️ Community Feedback

• NXP Community Voices: Developers complain the most not about the chip itself, but about the steep learning curve of Yocto Project. Compared to Rockchip, which directly provides a Debian image, NXP strongly promotes the Yocto build system, making it extremely painful for beginners 16.

• Stability Reputation: “Stable as a rock”. On EEVblog, engineers working in medical and automotive fields complain about its cost and speed, but dare not replace it because its errata is publicly transparent, and the official commitment is 15 years of supply cycle 17.

🏗️ Real Application Scenarios and Barriers

• Scenario: Industrial HMI / Medical Gateway

• Feedback: No packet loss at -40℃, and the full-load AI running power consumption is only ~780mW 18, which is an energy efficiency ratio that RK3588 cannot reach.

• 🚧 Barrier: AI Capability is Lame. The 0.5 TOPS Ethos-U65 can only perform simple vibration anomaly detection or keyword wake-up. If customers want to add facial recognition, the FPS drops to single digits.

02 Cross-Border Breakthrough Group

MCU’s “Memory Anxiety” and “Thermal Paradox”

This group attempts to replace Linux with MCU but is thwarted by the laws of physics.

🗡️ ST STM32N6

The “Assassin” Dancing with Shackles

🛠️ Hardware Specifications

• CPU Architecture: Arm Cortex-M55 @ 800MHz. This is an MCU core optimized for AI by Arm, supporting Helium vector extension technology 19191919.

• NPU Engine: ST self-developed Neural-ART Accelerator, computing power 0.6 TOPS (600 GOPS) 20202020.

• Memory Subsystem: This is key—integrated with large-capacity SRAM (specific size varies by model, usually in the 1MB-4MB range), aimed at reducing reliance on external DRAM.

• Peripheral Interfaces: Supports camera interfaces, even with simple ISP functions, allowing it to directly process visual data.

🗣️ Community Feedback

• ST Community Expectations: It is currently the hottest MCU. Everyone is asking one question:“How big is the SRAM?” This is a life-or-death parameter.

• Potential Barrier (The Wall): Memory Wall. The 0.6 TOPS NPU can theoretically run YOLO-Nano, but where to place the weight files? If placed in on-chip Flash, the read speed is slow; if placed in off-chip DDR, BOM costs and wiring difficulties increase. If relying solely on on-chip SRAM, the model size can easily exceed (OOM).

• Toolchain Concerns: Neural-ART is a self-developed architecture by ST, meaning it does not support the standard operator library of TFLite Micro, and developers worry they must use STM32Cube.AI for forced conversion, and if the conversion fails, they cannot manually optimize the underlying 21212121.

🏗️ Real Application Scenarios

• Scenario: Handheld Thermal Imager / Industrial Barcode Scanner

• Feedback: Startup speed is a killer (ms level), power consumption is extremely low, no need to wait for dozens of seconds like Linux Bootloader.

• 🚧 Barrier: Models must be extremely trimmed. Want to run MobileNet V2? Sorry, it might not fit.

🔥 NXP i.MX RT1170

The “Thermal Paradox” of Frequency Maniacs

🛠️ Hardware Specifications

• Process: 28nm FD-SOI (speculated).

• CPU Architecture: Dual-core MCU. High-performance core Cortex-M7 @ 1GHz + real-time core Cortex-M4 @ 400MHz 22.

• Memory: 2MB on-chip SRAM. This is a huge on-chip cache, ensuring data feeding at 1GHz frequency.

• Graphics: 2D GPU (Vector Graphics engine), supports 720p display.

🗣️ Community Feedback

• Heat Complaints: On the NXP official forum, the biggest discussion thread about RT1170 is usually “Thermal Design”. A MCU running at 1GHz, if no heat sink is added, will throttle when the ambient temperature rises. This contradicts the traditional impression of MCU being “cool and calm”.

• BGA Nightmare: High pin density requires PCB to be 6-layer or even 8-layer, directly deterring makers who want to create low-cost boards.

🏗️ Application Scenarios

• Scenario: High-end Audio Effects Processor / Multi-axis Motor Control

• Feedback: Interrupt latency is extremely low, capable of running 4 motor FOC algorithms simultaneously with ease.

• 🚧 Barrier: The main issue is not performance, but cost. The chip is expensive, the board is also expensive, and the development difficulty is close to that of MPU.

03 Emerging Mixed Battle Group

The Survival Philosophy of “Special Forces” and “Ecological Fragmentation”

The battlefield of domestic chips, where there are no standard answers, only extreme trade-offs.

🎥 Espressif ESP32-P4

The Misunderstood “Multimedia Engine”

🛠️ Hardware Specifications

• CPU Architecture: Dual-core RISC-V @ 400MHz, supports AI instruction extensions.

• Wireless Connectivity: No (No Wi-Fi / No Bluetooth). This is the biggest feature.

• Multimedia: H.264 hardware encoder (supports 1080p), MIPI-CSI (camera) / MIPI-DSI (screen).

• Memory: Large-capacity on-chip SRAM (768KB), supports external high-bandwidth PSRAM.

🗣️ Community Feedback

• “Why is there no Wi-Fi?”: This is the biggest confusion in the ESP32 enthusiast community. But experienced hardware engineers rate it highly—“Finally, there is a clean high-performance MCU”. By removing RF interference, the P4 can safely run at 400MHz and handle high-speed MIPI signals.

• IDF Ecosystem Concerns: Espressif’s IDF framework is unbeatable on the network stack, but has always been weak on the multimedia pipeline. Developers worry that the H.264 codec library for P4 will have many initial bugs, and the driver compatibility list for MIPI screens may be very short.

• PSRAM Bandwidth Doubts: The P4 heavily relies on external PSRAM for video stream processing. Under 1080P@30fps recording, can the PSRAM bandwidth hold up without tearing? This is a hot topic of discussion on GitHub.

🏗️ Application Scenarios

• Scenario: Smart Appliances with Screens (Washing Machines/Coffee Machines with Color Screens)

• Feedback

  : LVGL (graphics library) scores extremely high, capable of creating extremely smooth UI animations. By connecting an ESP32-C3 via SPI/SDIO for networking, it is a perfect “compute + connect” separation architecture.

🤖 Bouffalo Lab BL808

The “Nightmare” and “Paradise” for Geeks

🛠️ Hardware Specifications

• CPU Architecture: Tri-core heterogeneous RISC-V. Big core C906 (64-bit @ 480MHz, Linux) + Medium core E907 (32-bit @ 320MHz, RTOS) + Small core E902 (low-power standby).

• NPU Engine: BLAI-100, computing power 0.1 TOPS (100 GOPS), supports face detection.

• Memory: Embedded 64MB PSRAM. This is a stroke of genius, but also the biggest limitation.

• Wireless Connectivity: Wi-Fi 4 + BT 5.x + Zigbee.

🗣️ Community Feedback (Pine64 / Sipeed Community)

• “Documentation is a Puzzle”: Bouffalo Lab’s SDK is referred to by developers as a “puzzle game”. Most of the low-level register documentation is missing, forcing developers to read the source code to guess functionality.

• Wireless Blob Controversy: Although it is RISC-V, its Wi-Fi/BT protocol stack is a closed-source binary blob. This has greatly displeased open-source purists (like core members of the Linux community), leading to delays in mainline Linux kernel support.

• 64MB Trap: Users of M1s Dock (BL808 development board) found that after running the minimal Linux system, only about 20MB of memory is left for users. Running OpenCV slightly crashes due to OOM (out of memory).

🏗️ Application Scenarios

• Scenario: Ultra-low-cost USB Camera / Geek Toys

• Feedback

  : Running Linux + NPU for under $5, what more could you want?

🚪 Broadcom BK7258

The “Invisible Champion” of the Tuya Ecosystem

🛠️ Hardware Specifications

• CPU Architecture: Multi-core Arm Cortex-M33.

• Wireless Connectivity: Wi-Fi 6 (802.11ax) + Bluetooth 5.4. Supports TWT (Target Wake Time).

• Multimedia: 720p/VGA H.264 encoding/decoding, built-in audio ADC/DAC and AEC (Acoustic Echo Cancellation) hardware acceleration.

• AI Unit: Dedicated lightweight AI unit for audio noise reduction and simple image classification.

🗣️ Community Feedback

• “Can’t Find Information”: Ordinary developers can hardly find its datasheet on Google. This is because Broadcom mainly targets large customers (KA) and is deeply bound to the Tuya ecosystem.

• SDK Opacity: The Armino SDK is even more closed than NXP, with many functional modules being black boxes.

🏗️ Application Scenarios

• Scenario: Visual Doorbell / Cat Eye

• Feedback

  : The TWT technology of Wi-Fi 6 really saves power. In congested apartment buildings, its connectivity and keep-alive time are far superior to ESP32-S3.

04 Selection 🧭

After reviewing these real community evaluations and technical barriers, the selection logic becomes exceptionally clear:

• 💡If you want to tinker with large models and create edge servers: Only RK3588. Endure its driver bugs, embrace community patches, it is currently the only choice.

• 💡 For creating high-end UI panels: ESP32-P4. As long as you can accept external Wi-Fi, its graphical performance is the king among MCUs.

• 💡 For making industrial-grade high-reliability devices: NXP i.MX 93. Even if its computing power is weak and Yocto is difficult to use, it can give you peace of mind at night.

• 💡 For achieving extreme low-power video transmission: BK7258. Provided you accept the Tuya ecosystem or have original factory support.

• 💡 If you want to be a RISC-V martyr: BL808. It will teach you what “heterogeneous communication pain” and “memory overflow pain” are.

⚠️ In one sentence

Datasheets can only tell you what it*”can do”, while community forums and GitHub Issues can tell you what it“cannot do”*. In the AIoT era,the number of “pits” in the SDK often correlates with the sales volume of the chip.

🔮

Endgame Outlook: From “Instruction Set Wars” to “Agent Explosion”

1. Architectural Evolution: From “CNN Accelerators” to “Transformer Native Engines”

Current Situation: Current NPUs (like RK3588’s NPU 3.0 or Ethos-U65) are essentially convolution calculators. They excel at processing pixels (images), but when faced with sequential data (language/inference), it’s like asking a weightlifter to do embroidery—powerful but ineffective. Because LLMs require a lot of Softmax, LayerNorm, and Attention operators, which currently run on CPUs, leading to massive bus congestion.

🚀 2027 Outlook: Standardization of Transformer-Native NPU (T-NPU)

• Architectural Revolution: The next flagship (assuming model RK3688 or i.MX 10) must integrate nonlinear computing units. NPUs will no longer just perform multiply-accumulate (MAC) operations but will be able to directly hardware-decode Attention mechanisms.

• New Metrics: Core metrics on datasheets will shift from meaningless TOPS to Tokens/s (at INT4).

• My Expectation: I hope to see a chip priced at $5 that can run at 15 tokens/s with a 3B parameter SLM (small language model). This would mark the baseline capability for edge devices to have “real-time conversations” with humans.

2. Memory Revolution: Breaking Down the “Von Neumann Wall”

Current Situation: We are drinking water with a “straw”. Whether it’s STM32N6 or BL808, the biggest bottleneck is always Memory Bandwidth. Moving model weights between Flash, DRAM, and SRAM consumes 80% of power, leaving only 20% for actual computation.

🚀 2027 Outlook: Popularization of Processing-in-Memory (PIM) and Heterogeneous Storage

• Architectural Revolution:

• Low-end: MCUs will generally integrate 10MB+ of MRAM or high-speed PSRAM, using 3D stacked packaging (SiP), completely eliminating off-chip RAM.

• High-end: A cheap version of unified memory architecture (UMA) will emerge. CPUs, GPUs, and NPUs will truly share the same physical memory pool, achieving “zero-copy” data flow, just like Apple’s M series chips, but at IoT-level costs.

• My Expectation: Chip manufacturers should stop selling “bare dies”. Please sell SiP (System in Package) directly. Stack LPDDR5x directly on top of the SoC to solve the signal integrity nightmare of 64-bit high-speed wiring for engineers.

3. Ecological Endgame: RISC-V’s “Special Operations” and Arm’s “Counterattack”

Current Situation: RISC-V (like Espressif P4, Bouffalo BL808) is doing interesting experiments, but the software ecosystem remains “fragmented”. Arm (NXP, ST) is building barriers with a large old ecosystem but lacks flexibility.

🚀 2027 Outlook: “Transparency” of Instruction Sets

• Trend: Developers will no longer care whether the underlying architecture is Arm or RISC-V.

• Middleware Domination: A new generation of compilers and deployment tools (like TVM, IREE) will completely shield the differences in underlying ISAs. You train a model in PyTorch and deploy it to a custom NPU on RISC-V with one click, with optimizations handled automatically by the toolchain.

• Opportunities for RISC-V: In the field of multimodal I/O (like chips that process radar, vision, and audio simultaneously), RISC-V will completely defeat Arm with its low-latency advantages of custom instruction sets.

• My Expectation: I hope to see Espressif or Bouffalo launch a **“RISC-V Heterogeneous Complete Body”**—big cores running Linux for business, small cores running RTOS for connectivity, and NPUs for inference, with seamless communication through hardware message queues, rather than the current clumsy software-simulated shared memory.

4. Application Forms: From “Remote Control” to “Autonomous Agents”

Current Situation: Current smart homes are “command-based”—”turn on the living room light”. This is not intelligence; it is voice-controlled switching. Broadcom BK7258 and ESP32-P4 are still mainly acting as data movers.

🚀 2027 Outlook: Agent-on-Chip (On-Chip Agents)

• Scenario Revolution:

  The air conditioner will no longer listen to your commands but will see you wearing short sleeves through visual sensors (STM32N6), combine it with temperature sensors (i.MX 93), infer “you might feel hot”, and decide “to lower the temperature by 2 degrees, directing the airflow away from you”.

  All of this happens in its local SLM (small model), without needing to upload to the cloud, protecting privacy and ensuring no latency.

• Chip Support: This requires the chip to have RAG (Retrieval-Augmented Generation) capability, meaning the chip can establish a local “vector database” through external Flash to remember user habits over the past 30 days.

🧠 Conclusion

The future of AIoT is not about everything being interconnected, but about everything having a soul.

And the one that gives everything a soul is this next-generation silicon brain, only the size of a fingernail, yet integrated with a Transformer engine, unified memory, and heterogeneous computing.