Comparison of Common Inter-Processor Communication (IPC) Solutions in Embedded Systems

This article clarifies the mainstream solutions for “Inter-Processor Communication (IPC)” in embedded systems, covering aspects from technology, ecosystem, performance, practical cases to selection strategies, and aims to illustrate through comparative tables. The content includes shared memory / mailbox / hardware semaphore, Linux remoteproc/rpmsg/mailbox, OpenAMP/RPMsg, NXP RPMsg-Lite, TI IPC, Zephyr, ESP-IDF, as well as cross-chip SPI/I2C/UART/CAN, Ethernet TSN, USB, PCIe, etc.

Abstract

·Open source and mainstream ecosystem: On heterogeneous/multi-core systems, OpenAMP/RPMsg (in conjunction with Linux remoteproc/rpmsg) and Linux mailbox have become de facto standards: the former is responsible for lifecycle management and message channels, while the latter handles the underlying mailbox/interruption notifications; widely implemented in STM32MP1, Zynq, i.MX, TI AM series, etc.

·Vendor lightweight and proprietary: NXP’s RPMsg-Lite is optimized for small MCU memory usage; TI provides supporting documentation, examples, and resource tables; Zephyr’s built-in rpmsg_service further lowers the access threshold; ESP-IDF offers symmetric multi-core “function cross-core call” IPC.

·Low-level primitives and protocols: Hardware semaphores/mailboxes (e.g., STM32 HSEM, ARM MHU) provide the lightest synchronization/wakeup; SCMI specifies firmware service calls through the “mailbox + shared memory” protocol.

·Cross-chip interconnect alternatives: When not on the same chip (or requiring higher bandwidth/distance), SPI/I2C/UART offer better usability and cost but limited bandwidth; USB CDC, Ethernet (including TSN), PCIe (MSI/doorbell/NTB) provide more controlled bandwidth/latency/determinism but with higher complexity.

Market and technology panorama (classification and positioning)

·On-chip inter-processor communication (same SoC/MCU):

·Shared memory + hardware semaphore/mailbox: STM32 HSEM, ARM MHU, in conjunction with Linux mailbox client/controller drivers.

·OpenAMP/RPMsg + remoteproc: virtio+rings+vrings+RPMsg channel, with Linux side remoteproc responsible for powering on/loading/restarting the remote core.

·Vendor lightweight/proprietary libraries: NXP RPMsg-Lite reduces heap memory and dependencies; TI provides resource tables and tutorials for AM/OMAP/DRA.

·RTOS encapsulation: Zephyr rpmsg_service; ESP-IDF symmetric dual-core IPC (functions executed on another core).

·Cross-chip interconnect (different devices/boards):

·Low to medium speed: I2C (100kbps~ several Mbps but commonly ≤400kbps), UART (commonly 9.6kbps~460kbps), SPI (several Mbps~10+ Mbps depending on the platform).

·High speed: USB (HS theoretical 480Mbps, CDC class actual several MB/s limited by stack/host), Ethernet (100M/1G/TSN deterministic enhancement), PCIe (using MSI/doorbell/NTB for sharing or signaling).

Comparison Table 1: Mainstream On-Chip Inter-Processor Communication Technology Matrix (Mechanism/Ecosystem/Adaptation)

Technology/Framework	Core Mechanism	Linux/RTOS Ecosystem	Typical Chips/Platforms	Key Implementation Points/Limitations	Applicable Scenarios
Linux mailbox + hardware mailbox/semaphore (MHU/HSEM, etc.)	Mailbox controller + client driver, delivering messages/ringing through shared registers/interrupts	Kernel general framework, stable driver interface	ARM MHU, STM32 HSEM, NXP MAILBOX	Requires DT/driver matching; commonly used for notifications/short messages; often combined with upper-layer RPMsg/SCMI	Lightweight notifications, low-latency wake-up, SCMI firmware channel
OpenAMP/RPMsg + remoteproc	virtio+vring, RPMsg message channel; remoteproc manages lifecycle	OpenAMP across OS; Linux upstream includes remoteproc/rpmsg	Zynq/Versal, STM32MP1, i.MX, TI AM/OMAP	Common RPMsg buffer of 512B in Linux 4.19 (customizable), clear endpoint model AMD UG1186	Heterogeneous AMP, Linux+RTOS/BM collaboration
NXP RPMsg-Lite	Simplified RPMsg, reduced dependencies, low RAM/ROM	Integrated with MCUXpresso SDK, rich examples	i.MX RT/i.MX 8M(M4), etc.	Compatible with OpenAMP concept but lighter, suitable for resource-constrained MCUs	Resource-constrained MCU and A/MPU heterogeneous
TI IPC (remoteproc+RPMsg)	Follows Linux remoteproc/rpmsg, provides resource table	TI SDK/Academy tutorials are comprehensive	AM62/64, DRA7x/AM57x, etc.	Resource table drives firmware loading and RPMsg channel configuration TI Academy	Sitara/KeyStone and other industrial control
Zephyr rpmsg_service	Easy-to-use encapsulation based on OpenAMP	Native support in Zephyr	STM32MP1, i.MX, Renesas, etc.	Simplifies endpoint creation and initialization in Zephyr	Fast integration on Zephyr side
ESP-IDF IPC (symmetric multi-core)	Inter-Processor Call: executes callbacks on another core	ESP32 series	ESP32 dual-core	Suitable for symmetric SMP, multi-core task scheduling ESP-IDF	Pure MCU symmetric multi-core

Comparison Table 2: Cross-Chip Interconnect (When “Not on the Same Chip”)

Interface	Typical Bandwidth Levels (Common in Engineering)	Determinism/Latency	Implementation Complexity	Adaptation Scenarios
I2C	100kbps/400kbps, higher in extended mode but less common	Medium/High latency, easily affected by noise/arbitration	Low	Low-speed control, configuration, small data
UART	Commonly 9.6kbps–460kbps	Medium/High latency, asynchronous, requires baud rate matching	Low	Debugging, logging, simple point-to-point
SPI	Several Mbps–dozens of Mbps depending on the master	Low latency, master-slave structure	Medium	High-speed short-distance point-to-point/board-to-board
USB (CDC)	HS 480Mbps theoretical; actual CDC several MB/s affected by end-to-end implementation	Medium; host stack determinism is weaker	Medium/High	Bulk data from device to host
Ethernet (including TSN)	100M/1G/higher; TSN provides time-aware scheduling and shaping	Can achieve determinism (Qbv, CBS, etc.)	High	Industrial real-time, distributed control
PCIe (MSI/doorbell/NTB)	Several Gb/s per lane; flexible sharing and interrupt	Low latency, high bandwidth	High	Multi-processor/multi-board interconnect

Key Technology Deep Dive (Mechanism, Ecosystem, Considerations)

1.Linux mailbox + hardware mailbox/semaphore (including ARM MHU, STM32 HSEM)

·Mechanism: The kernel’s Common Mailbox Framework abstracts “controller/client”, sending messages/ringing through register windows and interrupts, commonly used as a notification path for upper-layer RPMsg/SCMI.

·Protocol: SCMI explicitly specifies waking through mailbox + shared memory as a channel, defining firmware services such as power/clock/performance.

·MCU-level semaphore: STM32 HSEM provides atomic locks and cross-core synchronization, implementing lightweight IPC modes in conjunction with shared memory, with examples provided by Zephyr.

·When to choose: When a minimal “notification/preemption/resource mutual exclusion” is needed, or as a base for OpenAMP/SCMI; bandwidth is not a primary concern.

2.OpenAMP/RPMsg + remoteproc (de facto standard heterogeneous AMP combination)

·Mechanism: RPMsg is based on virtio vring message bus; remoteproc manages firmware loading, startup/stop, and crash recovery for the remote core; the Linux side kernel includes related frameworks. OpenAMP RPMsg

·Features: endpoints, service discovery, shared buffers; common single packet of 512B in Linux 4.19 (customizable), projects often do “fragmentation + reassembly” to handle large packets. AMD UG1186 buffer size..

·Ecosystem: Widely adopted by STM32MP1, Zynq, i.MX, TI AM, etc.; Zephyr/FreeRTOS/NuttX have ports; examples and system reference projects are complete.

·When to choose: For heterogeneous systems with Linux+RTOS/Bare-metal, requiring both a “standardized framework” and “tunable reliability”.

3.NXP RPMsg-Lite (embedded-friendly)

·Mechanism: Retains the core idea of RPMsg, creating a “lightweight implementation” to reduce RAM/ROM and porting costs; MCUXpresso provides examples (ping-pong, echo).

·When to choose: When MCU resources are constrained, and the other side is Linux/RTOS AMP.

4.Texas Instruments (TI) IPC (remoteproc/RPMsg and resource table)

·Mechanism/process: Through the resource table, Linux remoteproc understands the memory layout, VRING configuration, and RPMsg channel of the remote core; TI documentation and Academy tutorials cover PRU/DSP, etc.

When to choose: Sitara/KeyStone platforms, commonly used options for industrial real-time and multi-core collaboration.

·Zephyr rpmsg_service and ESP-IDF’s symmetric multi-core IPC

·Zephyr: abstracts OpenAMP initialization and endpoints, shortening bring-up time (including examples of interoperability with Linux).

·ESP-IDF: IPC calling model allows one core to directly request execution of callbacks on another core, suitable for symmetric SMP MCUs.

Cross-chip IPC alternatives and deterministic reinforcement

·Low/medium-speed buses: I2C/UART/SPI are easy to use but have limited bandwidth, with common engineering speed levels as shown in the table above (note wiring constraints and signal quality).

·USB CDC: HS theoretical 480Mbps, but CDC/serial class actual throughput is affected by end-to-end implementation, with community feedback indicating “achievable several MB/s levels” (not hard metrics, strongly dependent on stack/host/buffer).

·Ethernet TSN: Achieves low jitter and predictable latency through time-aware scheduling (Qbv)/credit shaping (CBS), suitable for distributed real-time control.

·PCIe: Utilizes EP/RC MSI/doorbell triggers, or shares/transfers through NTB; low latency and high bandwidth, but system design is complex.

Comparison Table 3: Overview of Advantages and Disadvantages (On-Chip Solutions)

Solution	Advantages	Limitations
Mailbox/HSEM (primitive level)	Extremely simple, good determinism, low latency, low power consumption	Only suitable for notifications/mutual exclusion, data plane needs to be managed independently
OpenAMP/RPMsg + remoteproc	Wide ecosystem, cross OS/SoC, complete lifecycle management, abundant debugging materials	Requires proper memory area/resource table/interrupt routing for the platform; message size is limited by default
RPMsg-Lite	Small footprint, quick to port	Features are more streamlined, needs to match with the counterpart stack
TI IPC	High compatibility with TI platforms, comprehensive documentation	Cross-platform generality is not as strong as upstream OpenAMP descriptiveness
Zephyr rpmsg_service	Simplified API, quick integration	Mainly targeted at the Zephyr side, system-level details still need to be understood
ESP-IDF IPC	Convenient symmetric multi-core calls	Not heterogeneous AMP, applicable range limited to ESP symmetric multi-core

Implementation Cases and Reference Paths

·STM32MP1 (A7+M4): Linux (A7) remoteproc loads M4 firmware, rpmsg establishes a channel, with the underlying notification through mailbox; ST provides an end-to-end process.

·AMD/Xilinx Zynq/Versal: Official UG1186 full process and examples, buffer size description, Kria SOM guide, rich OpenAMP examples (matrix multiplication demo, etc.).

·NXP i.MX/RT: RPMsg-Lite + Linux/RTOS combination, SDK provides ping-pong, echo, and other examples for quick verification.

·TI AM6x/AM57x: Resource table configuration + remoteproc loads the remote core (PRU/DSP/M4), RPMsg channel communication, documentation and academy tutorials cover debugging details.

Selection Recommendations (Practical Guide for AI Integration/Edge Applications)

·Heterogeneous collaboration of Linux+RTOS/BM: Prioritize OpenAMP/RPMsg + remoteproc, following vendor BSP’s memory area/interrupt routing recommendations; large message packets should be fragmented/shared buffer extended; notifications should go through mailbox.

·When MCU resources are tight: Use RPMsg-Lite on the NXP/i.MX RT side, the other end can connect to Linux RPMsg; pay attention to buffer and cache consistency.

·Symmetric multi-core MCU (no heterogeneous): IPC like ESP-IDF is simple and efficient for “cross-core calls”, less is more.

·Only need locks/wake-ups: Hardware semaphore/mailbox (HSEM/MHU) + shared memory circular queue, self-built lightweight protocol is sufficient; if Linux is present, attach the mailbox client.

·Board-to-board/device-to-host:

·Control class low speed: I2C/UART are simple and reliable;

·Medium-speed short distance: SPI is easy to connect directly;

·High throughput: USB (fast implementation) or PCIe (complex but extreme performance);

·Distributed and deterministic: Ethernet TSN. When selecting, weigh “bandwidth/latency/determinism/wiring/power supply/stack maturity” as axes.

Implementation and Pitfall Checklist

·Memory and cache: RPMsg/shared memory require correct non-cache/cache consistency settings and alignment; Linux side DMA/CMA and remote core perspectives must be consistent.

·Interrupt and mailbox routing: DT/hardware routing errors often lead to handshake failures or deadlocks, prioritize reusing vendor reference designs.

·Message size and fragmentation: The default 512B buffer in Linux is a common “surprise”, large packet strategies and zero-copy buffer pools need to be planned at the protocol level.

·Lifecycle management: remoteproc’s crash recovery/hot restart and resource table must be coordinated, otherwise “can start but cannot reset”.

·Debugging and observation: Prefer to use official demos (echo/ping-pong/mat-mul) to verify the link before integrating business data; combine with trace/statistics and watchdog.

·Security and functional safety: SCMI channel/MPU protection/memory domain isolation, avoid “wild pointers across domains”; industrial real-time suggests TSN/time synchronization to ensure system-level determinism. Attached: More detailed ecosystem and documentation guidance (to facilitate your team’s implementation)

·Need to checkOpenAMP project page and white papers/design documents/examples

·Linux kernel framework documentation: remoteproc/rpmsg/mailbox

·Vendor paths: ST (STM32MP1 wiki), AMD/Xilinx (UG1186), NXP (RPMsg-Lite guide), TI (Processor SDK/Academy).

Reference Information:

Summary of Inter-Process Communication (IPC) Methods (Part 1)

https://www.cnblogs.com/joker-wz/p/11000489.html

Complete Guide to Inter-Process Communication IPC: Principles and Practices of Various Mechanisms

https://cloud.tencent.com/developer/article/2439262