A Powerful Inter-Process Communication Tool for Embedded Systems!

Follow our official account to keep the embedded knowledge flowing!

1. Communication in Embedded Systems

In embedded Linux projects, there are various options for inter-process communication:

Message Queues (POSIX MQ): Complicated API, message size is limited (usually 8KB), and cross-machine communication requires rewriting.
Shared Memory + Semaphores: Best performance, but requires manual synchronization, which can lead to deadlocks if not handled carefully.
Socket Programming: Flexible but requires a lot of code; error handling and reconnection logic can take hundreds of lines.
D-Bus: Powerful but too heavy; a single daemon can consume several MB of memory.

All these options require: attention to too many low-level details.

In projects like smart gateways and edge computing devices, the system is often divided into multiple processes: a collection process reads sensor data, a processing process performs algorithms, and a reporting process handles cloud communication.

Looking for a unified interface that can handle both inter-process and cross-network communication? Want automatic reconnection and load balancing? The cost of implementing it yourself is too high; consider using nanomsg.

https://github.com/nanomsg/nanomsg

A Powerful Inter-Process Communication Tool for Embedded Systems!

nanomsg is a high-performance communication library that implements several scalable protocols; the task of scalable protocols is to define how multiple application systems communicate, forming a large distributed system.

Positioning of nanomsg

nanomsg is not a simplified version of ZeroMQ; it is a redesign by Martin Sustrik, the author of ZeroMQ. The core idea is simple: to provide high-level messaging patterns while hiding low-level transport details.

In one sentence: it elevates socket programming to an abstract layer of “message communication patterns”.

Key Data:

Binary size: approximately 300KB when statically linked
Runtime overhead: about 4KB of memory per socket

2. Usage Example

Assuming you have a temperature and humidity collector that needs to send data to a local display process, a logging process, and a network reporting process simultaneously.

Publisher (Collection Process): publisher.c

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <nn.h>
#include <pubsub.h>

// Simulate sensor reading
float read_temperature() {
    static float temp = 25.0;
    temp += (rand() % 20 - 10) / 10.0; 
    return temp;
}

int main() {
    int pub = nn_socket(AF_SP, NN_PUB);
    if (pub < 0) {
        perror("nn_socket");
        return 1;
    }
    
    if (nn_bind(pub, "ipc:///tmp/sensor.ipc") < 0) {
        perror("nn_bind");
        return 1;
    }
    
    printf("Temperature collector started, publishing to ipc:///tmp/sensor.ipc\n");
    
    while (1) {
        char msg[32];
        float temp = read_temperature();
        int len = snprintf(msg, sizeof(msg), "TEMP:%.2f", temp);
        
        if (nn_send(pub, msg, len, 0) < 0) {
            perror("nn_send");
        } else {
            printf("Published: %s\n", msg);
        }
        sleep(1);
    }
    
    nn_close(pub);
    return 0;
}

Subscriber (Any Subscription Process): subscriber.c

#include <stdio.h>
#include <string.h>
#include <nn.h>
#include <pubsub.h>

int main() {
    int sub = nn_socket(AF_SP, NN_SUB);
    if (sub < 0) {
        perror("nn_socket");
        return 1;
    }
    
    // Subscribe to all messages (empty string means no filter)
    if (nn_setsockopt(sub, NN_SUB, NN_SUB_SUBSCRIBE, "", 0) < 0) {
        perror("nn_setsockopt");
        return 1;
    }
    
    if (nn_connect(sub, "ipc:///tmp/sensor.ipc") < 0) {
        perror("nn_connect");
        return 1;
    }
    
    printf("Subscriber process started (PID: %d)\n", getpid());
    
    char buf[64];
    int bytes;
    while ((bytes = nn_recv(sub, buf, sizeof(buf), 0)) >= 0) {
        buf[bytes] = '\0'; 
        printf("[PID %d] Received: %s\n", getpid(), buf);
    }
    
    perror("nn_recv");
    nn_close(sub);
    return 1;
}

Compilation:

gcc publisher.c -o publisher -lnanomsg
gcc subscriber.c -o subscriber -lnanomsg

Execution:

# Start the publisher
./publisher &

# Start multiple subscribers
./subscriber &
./subscriber &
./subscriber &

Notes:

The publisher does not need to know how many subscribers there are: The sender and receiver are completely decoupled.
Transport layer can be switched: Changing ipc:// to tcp://192.168.1.100:5555 allows cross-machine communication.
Automatic connection management: Subscriber reconnection is automatic, no code required.

Comparison with traditional Socket code:

If implemented with raw sockets, the same functionality might require:

Listening on a port → socket() + bind() + listen()
Managing multiple client connections → accept() + connection list
Looping to send to all clients → Iterating through connections, handling EPIPE errors
Handling client disconnections → Cleaning up the connection list

3. Introduction to nanomsg

3.1 Directory Structure

Core Directory Structure:

Module Dependency Relationships:

3.2 Layered Architecture

nanomsg adopts a “protocol stack” design, but unlike TCP/IP, the “protocols” here refer to message communication patterns.

What does the protocol layer do?

Taking the REQ/REP (request-reply) pattern as an example, the protocol layer ensures:

Strict message pairing: A REQ must wait for a REP before sending the next one.
Automatic retries: If the peer crashes, requests will automatically route to other available servers.
Load balancing: When a REQ connects to multiple REPs, it automatically distributes requests in a round-robin manner.

3.3 Zero-Copy

For large messages (like image data), nanomsg provides nn_allocmsg to reduce intermediate copies:

Method 1: Using nn_allocmsg (recommended):

Method 2: Regular send

Two internal paths of nn_sendmsg:

Comparison of copy counts between the two methods (using inproc transport as an example):

Stage	Regular send	nn_allocmsg
Filling data	1 copy	1 copy
Sending to queue	1 copy	0 copies (passing pointer)
Receiving from queue	1 copy	0 copies (passing pointer)

4. Choosing Protocol Patterns

nanomsg provides six basic patterns; how to choose?

Requirement Scenario	Recommended Pattern	Reason
Multiple processes need the same data	PUB/SUB	Automatic broadcasting, subscribers are independent
Remote calls need to return results	REQ/REP	Strict request-response pairing
Tasks need to be processed in parallel	PUSH/PULL	Automatic load balancing
Need a bidirectional dedicated channel	PAIR	Simple and efficient
Multi-node mutual notification	BUS	Fully interconnected broadcasting
Need to collect responses from multiple nodes	SURVEY	Time-limited response collection

90% of actual projects only need:

PUB/SUB – Sensor data, event notifications
REQ/REP – Configuration management, RPC calls
PUSH/PULL – Task queues, pipeline processing

5. Important Experiences

5.1 Three Tips for Performance Optimization

1. Prefer inproc

For processes on the same machine, prefer using inproc:// instead of ipc://:

2. Set buffer sizes appropriately

The default is 128KB; for high-throughput scenarios, it can be adjusted to 1MB, but be mindful of memory usage.

3. Use nn_poll for multiplexing

When a process needs to handle multiple sockets:

5.2 Comparison with Other Projects

Project	Binary Size	Dependencies	Maturity	Applicable Scenarios
nanomsg	~300KB	None	Stable (1.x)	Embedded, inter-process communication
nng	~400KB	None	Active development	nanomsg upgrade
ZeroMQ	~2MB	libsodium	Very mature	General distributed systems
Mosquitto	~500KB	OpenSSL	Mature	MQTT specific

Resource-constrained (<16MB memory): choose nanomsg
Need MQTT protocol: choose Mosquitto
New project with moderate stability requirements: choose nng (new work by nanomsg author, API compatible)

Conclusion

The core problem solved by nanomsg is standardizing communication patterns. No longer needing to repeatedly implement “publish-subscribe” or “request-reply” for each project, just like you don’t need to implement TCP yourself. 90% of scenarios use PUB/SUB and REQ/REP, with a preference for inproc transport.

You might also like:

A lightweight ring buffer management library suitable for embedded systems!

Git interactive rebase to modify commit descriptions

Singleton pattern: The guardian of global state consistency in embedded systems

Embedded field: The ultimate showdown between Linux and RTOS!

Embedded software advancement guide, let’s level up together!

Clickto read the original text and get popular embedded books