Understanding the Commonly Used MQTT Protocol in IoT

Scan to FollowLearn Embedded Together, learn and grow together

Overview

1.1 What is MQTT

MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol,

developed by Andy Stanford-Clark of IBM and Arlen Nipper of Arcom in 1999, specifically designed for low bandwidth, high latency, or unstable network environments.

1.2 Features

• Lightweight: The protocol header is as small as 2 bytes, making it very suitable for resource-constrained devices
• Publish/Subscribe Model: Decouples message producers and consumers
• Supports QoS Levels: Provides three levels of message delivery quality assurance
• Low Power Consumption: Suitable for battery-powered IoT devices
• Supports Persistent Sessions: Reduces overhead during reconnections
• Supports Last Will Messages: Notifies other clients when a client disconnects unexpectedly

1.3 Protocol Versions

• MQTT 3.1: The original version
• MQTT 3.1.1: OASIS standardized version (widely used)
• MQTT 5.0: The latest version, adding more features (gradually becoming popular)

Protocol Architecture

2.1 Basic Components

1. Broker:

• A message relay station responsible for receiving, filtering, and forwarding messages
• Maintains client subscription relationships
• Common open-source Brokers: Mosquitto, EMQX, HiveMQ, etc.

Publisher:

• A client that publishes messages to specific topics
• Does not need to know the existence of subscribers

Subscriber:

• A client that subscribes to topics of interest and receives messages
• Can subscribe to multiple topics simultaneously

2.2 Communication Process

1. The client establishes a TCP connection with the Broker
2. The client sends a CONNECT message for authentication
3. The Broker returns a CONNACK to confirm the connection
4. The subscriber sends a SUBSCRIBE to subscribe to a topic
5. The publisher sends a PUBLISH to publish a message to the topic
6. The Broker forwards the message to all clients subscribed to that topic

Detailed Explanation of Core MQTT Concepts

3.1 Topic

The topic is a classification identifier for messages, using a hierarchical structure separated by “/”:

home/livingroom/temperature
home/bedroom/humidity

Wildcards:

• Single-level wildcard<span>+</span>: Matches one level

  home/+/temperature
  

• Multi-level wildcard<span>#</span>: Matches multiple levels

  home/#
  

3.2 QoS (Quality of Service) Levels

QoS Level	Description	Message Guarantee	Network Traffic
0	At most once	No guarantee	Lowest
1	At least once	Guaranteed delivery, may be duplicated	Medium
2	Exactly once	Guaranteed and not duplicated	Highest

3.3 Retained Messages

The Broker stores the latest retained message for each topic, and new subscribers will immediately receive that message.

3.4 Last Will and Testament

Clients specify this upon connection; if they disconnect unexpectedly, the Broker will publish a preset message on their behalf.

3.5 Clean Session

• True: The Broker does not save the client’s subscription information and unacknowledged messages
• False: The Broker saves the session state, allowing the client to resume after reconnecting

MQTT Message Format

The MQTT protocol consists of 14 types of control messages, each composed of a fixed header, variable header, and payload.

4.1 Fixed Header Format

| Bit   | 7-4 | 3-0 |
|-------|-----|-----|
| Byte1 | Message Type | Flags |
| Byte2 | Remaining Length |

4.2 Main Message Types

1. CONNECT: Client requests to connect to the Broker
2. CONNACK: Broker confirms the connection
3. PUBLISH: Publish a message
4. PUBACK: QoS 1 message acknowledgment
5. PUBREC: QoS 2 message received
6. PUBREL: QoS 2 message released
7. PUBCOMP: QoS 2 message completed
8. SUBSCRIBE: Subscribe to a topic
9. SUBACK: Subscription acknowledgment
10. UNSUBSCRIBE: Unsubscribe
11. UNSUBACK: Unsubscription acknowledgment
12. PINGREQ: Heartbeat request
13. PINGRESP: Heartbeat response
14. DISCONNECT: Disconnect

Practical Applications

5.1 Setting Up the Development Environment

Installing Mosquitto Broker

# Ubuntu
sudo apt-get install mosquitto mosquitto-clients

# Start the service
sudo systemctl start mosquitto

# Test publish and subscribe
mosquitto_sub -h localhost -t "test/topic"
mosquitto_pub -h localhost -t "test/topic" -m "Hello MQTT"

Common Client Libraries

Language	Common Libraries
C/C++	Paho MQTT C, Eclipse Mosquitto
Python	Paho MQTT Python
Java	Eclipse Paho Java Client
JavaScript	MQTT.js
Go	Eclipse Paho Go Client

5.2 Python Example Code

Publisher

import paho.mqtt.client as mqtt
import time

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

client = mqtt.Client()
client.on_connect = on_connect

client.connect("localhost", 1883, 60)

for i in range(10):
    client.publish("test/topic", payload=f"Message {i}", qos=1)
    time.sleep(1)

client.disconnect()

Subscriber

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe("test/topic", qos=1)

def on_message(client, userdata, msg):
    print(f"Received message: {msg.payload.decode()} on topic {msg.topic}")

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect("localhost", 1883, 60)
client.loop_forever()

5.3 C Language Example (Using Paho Library)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <MQTTClient.h>

#define ADDRESS     "tcp://localhost:1883"
#define CLIENTID    "ExampleClient"
#define TOPIC       "test/topic"
#define QOS         1
#define TIMEOUT     10000L

volatile MQTTClient_deliveryToken deliveredtoken;

void delivered(void *context, MQTTClient_deliveryToken dt) 
{
    printf("Message with token value %d delivery confirmed\n", dt);
    deliveredtoken = dt;
}

int msgarrvd(void *context, char *topicName, int topicLen, MQTTClient_message *message) 
{
    printf("Message arrived\n");
    printf("     topic: %s\n", topicName);
    printf("   message: %.*s\n", message->payloadlen, (char*)message->payload);
    MQTTClient_freeMessage(&message);
    MQTTClient_free(topicName);
    return 1;
}

void connlost(void *context, char *cause) 
{
    printf("\nConnection lost\n");
    printf("     cause: %s\n", cause);
}

int main(int argc, char* argv[]) 
{
    MQTTClient client;
    MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
    MQTTClient_message pubmsg = MQTTClient_message_initializer;
    MQTTClient_deliveryToken token;
    int rc;
    
    MQTTClient_create(&client, ADDRESS, CLIENTID,
        MQTTCLIENT_PERSISTENCE_NONE, NULL);
    conn_opts.keepAliveInterval = 20;
    conn_opts.cleansession = 1;
    
    MQTTClient_setCallbacks(client, NULL, connlost, msgarrvd, delivered);
    
    if ((rc = MQTTClient_connect(client, &conn_opts)) != MQTTCLIENT_SUCCESS) 
    {
        printf("Failed to connect, return code %d\n", rc);
        exit(EXIT_FAILURE);
    }
    
    pubmsg.payload = "Hello World";
    pubmsg.payloadlen = strlen("Hello World");
    pubmsg.qos = QOS;
    pubmsg.retained = 0;
    MQTTClient_publishMessage(client, TOPIC, &pubmsg, &token);
    printf("Waiting for up to %d seconds for publication\n", (int)(TIMEOUT/1000));
    rc = MQTTClient_waitForCompletion(client, token, TIMEOUT);
    printf("Message with delivery token %d delivered\n", token);
    
    MQTTClient_disconnect(client, 10000);
    MQTTClient_destroy(&client);
    return rc;
}

5.4 Implementation on Embedded Devices (Based on ESP32)

#include <WiFi.h>
#include <PubSubClient.h>

const char* ssid = "your_SSID";
const char* password = "your_PASSWORD";
const char* mqtt_server = "broker.example.com";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() 
{
    delay(10);
    Serial.println();
    Serial.print("Connecting to ");
    Serial.println(ssid);
    
    WiFi.begin(ssid, password);
    
    while (WiFi.status() != WL_CONNECTED) 
    {
        delay(500);
        Serial.print(".");
    }
    
    Serial.println("");
    Serial.println("WiFi connected");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* message, unsigned int length) 
{
    Serial.print("Message arrived on topic: ");
    Serial.print(topic);
    Serial.print(". Message: ");
    String messageTemp;
    
    for (int i = 0; i < length; i++) 
    {
        messageTemp += (char)message[i];
    }
    Serial.println(messageTemp);
}

void reconnect() 
{
    while (!client.connected()) 
    {
        Serial.print("Attempting MQTT connection...");
        if (client.connect("ESP32Client")) 
        {
            Serial.println("connected");
            client.subscribe("esp32/output");
        } 
        else
        {
            Serial.print("failed, rc=");
            Serial.print(client.state());
            Serial.println(" try again in 5 seconds");
            delay(5000);
        }
    }
}

void setup()
{
    Serial.begin(115200);
    setup_wifi();
    client.setServer(mqtt_server, 1883);
    client.setCallback(callback);
}

void loop() 
{
    if (!client.connected()) 
    {
        reconnect();
    }
    client.loop();
    
    static unsigned long lastMsg = 0;
    if (millis() - lastMsg > 5000) 
    {
        lastMsg = millis();
        
        float temperature = readTemperature(); // Hypothetical temperature reading function
        char tempString[8];
        dtostrf(temperature, 1, 2, tempString);
        client.publish("esp32/temperature", tempString);
    }
}

MQTT Security Practices

6.1 Authentication Mechanisms

1. Username/Password Authentication:

   client.username_pw_set("username", "password")
   

2. Client Certificate Authentication:

   client.tls_set(ca_certs="ca.crt", 
                 certfile="client.crt", 
                 keyfile="client.key")
   

6.2 Transport Encryption

• Use MQTT over TLS/SSL (usually port 8883)
• Disable old SSL protocols, use TLS 1.2+

6.3 Access Control

Configure ACL (Access Control List) on the Broker side, for example, in Mosquitto:

# Control topic access permissions
user username
topic readwrite home/#
topic read$SYS/#

MQTT Performance Optimization

1. Reasonable Keep Alive Settings: Balance heartbeat overhead and connection detection sensitivity
2. Batch Message Processing: Combine small messages to reduce protocol overhead
3. QoS Selection: Choose the appropriate QoS level based on the scenario
4. Topic Design Optimization:

• Avoid excessive levels
• Reduce wildcard subscriptions

Persistent Session Trade-offs: Clean Session=True can reduce Broker resource usage

New Features in MQTT 5.0

1. Reason Codes: More detailed error feedback

2. Shared Subscriptions: Achieve load balancing

   $share/group1/topic
   

3. Message Expiration: Set message TTL

4. Topic Aliases: Reduce transmission overhead of long topic names

5. User Properties: Custom metadata

6. Flow Control: Control receive window

Common Problem Solutions

9.1 Connection Issues

• Error Code 3: Server Unavailable
• Check if the Broker is running
• Verify network connection
• Check firewall settings
• Error Code 5: Unauthorized
• Check username/password
• Verify if the client ID is conflicting

9.2 Message Loss

• Confirm QoS level settings
• Check Clean Session settings
• Verify network stability

9.3 High Latency

• Reduce message size
• Lower QoS level
• Optimize Broker deployment location

MQTT Usage Recommendations

1. Topic Naming Conventions:

• Use lowercase letters and numbers
• Avoid special characters
• Adopt a consistent hierarchical structure

Client ID Design:

• Ensure uniqueness
• Include device type and identification information
• Avoid using random IDs (affects persistent sessions)

Error Handling:

• Implement reconnection logic
• Monitor connection status
• Log key events

Resource Management:

• Embedded devices should pay attention to memory usage
• Limit the number of subscribed topics
• Release MQTT objects in a timely manner

Application Scenarios

1. IoT Device Communication
2. Mobile Push Notifications
3. Real-time Monitoring Systems
4. Smart Home Control
5. Vehicle Networking Communication
6. Industry 4.0 Device Interconnection

Conclusion

With its lightweight and efficient characteristics, the MQTT protocol has become the de facto standard protocol for IoT communication.

By reasonably designing topic structures, selecting appropriate QoS levels, and implementing necessary security measures, developers can build stable and reliable MQTT application systems.

With the popularity of MQTT 5.0, the protocol offers more powerful features to meet the needs of more complex IoT application scenarios.

In actual development, it is essential to choose the appropriate MQTT implementation scheme based on specific application scenarios and device capabilities, and to follow best practices to ensure system performance and security.

Follow 【Learn Embedded Together】 to become better together.

If you find this article helpful, click “Share”, “Like”, or “Recommend”!