Understanding the MQTT Protocol: A Comprehensive Guide

Hello everyone, I am Xiaomai. Recently, I worked on an IoT project and summarized the MQTT protocol. As we all know, the MQTT protocol is widely used in the Internet of Things. If you are not very familiar with it, I believe this article can help you get started.

• MQTT Protocol

• 1 Features of MQTT Protocol

• Publish and Subscribe

• QoS (Quality of Service levels)

• 2 MQTT Packet Structure

• 2.1 Fixed Header of MQTT

• 2.2 Variable Header of MQTT

• 2.3 Payload Message Body

• 3 Environment Setup

• 3.1 Setting up MQTT Server

• 3.2 MQTT Client

• 4 Summary

MQTT Protocol

MQTT (Message Queuing Telemetry Transport) is a “lightweight” communication protocol based on the publish/subscribe model, built on the TCP/IP protocol and released by IBM in 1999.

The greatest advantage of MQTT is that it provides real-time reliable messaging services for connecting remote devices with minimal code and limited bandwidth.

As a low-overhead, low-bandwidth messaging protocol, it has a wide range of applications in IoT, small devices, and mobile applications.

1 Features of MQTT Protocol

MQTT is a client-server based message publish/subscribe transport protocol.

The MQTT protocol is lightweight, simple, open, and easy to implement, making it suitable for a wide range of applications. It is widely used in many scenarios, including constrained environments such as machine-to-machine (M2M) communication and the Internet of Things (IoT).

It has been widely used in applications such as satellite link communication sensors, occasionally dial-up medical devices, smart homes, and some miniaturized devices.

The current version of MQTT is MQTT v3.1.1, released in 2014. In addition to the standard version, there is a simplified version called MQTT-SN, which is mainly aimed at embedded devices that generally operate on TCP/IP networks, such as ZigBee.

Like HTTP, MQTT runs on top of the Transmission Control Protocol/Internet Protocol (TCP/IP) stack.

Publish and Subscribe

The MQTT uses a publish/subscribe messaging pattern, providing a one-to-many message distribution mechanism that decouples it from the applications.

This is a messaging pattern where messages are not sent directly from the sender to the receiver (i.e., point-to-point), but are distributed by the MQTT server (also known as the MQTT Broker).

The MQTT server is the core of the publish-subscribe architecture.

It can be very simply implemented on single-board computers such as Raspberry Pi or NAS, and of course also on mainframes or Internet servers.

The server distributes messages, so it must be a publisher, but never a subscriber!

Clients can publish messages (senders), subscribe to messages (receivers), or do both.

Clients (also called nodes) are smart devices, such as microcontrollers or computers with a TCP/IP stack and software that implements the MQTT protocol.

Messages are published under topics that allow filtering. Topics are hierarchically divided UTF-8 strings, with different topic levels separated by slashes /.

Let’s take a look at the following setup.

• The photovoltaic power station is the publisher (Publisher).

• The main topic (Topic) level is "PV", and this plant publishes two sub-levels "sunshine" and "data";

• "PV/sunshine" is a boolean value (true/fault, which can also be 1/0), and the charging station needs it to know whether it should load the electric vehicle (only when it is sunny 🙂 ).

• The charging station (EVSE) is the subscriber, subscribing to "PV/sunshine" to get information from the server.

• "PV/data" on the other hand, transmits the instantaneous power generated by the plant in kW, and this topic can be subscribed to by a computer or tablet, for example, to generate a chart of the transmitted power over a day.

This is a simple application scenario of MQTT, as shown in the figure below;

QoS (Quality of Service levels)

Quality of Service is an important feature of MQTT. When we use TCP/IP, the connection is protected to a certain extent. However, in wireless networks, interruptions and interference are frequent, and MQTT helps avoid information loss and its service quality levels. These levels are used during publishing. If a client publishes to the MQTT server, the client will be the sender, and the MQTT server will be the receiver. When the MQTT server publishes messages to the client, the server is the sender, and the client is the receiver.

QoS 0

This level can lead to message loss or duplication, and message publishing relies on the underlying TCP/IP network. That is: <=1

QoS 1

QoS 1 promises that messages will be delivered to subscribers at least once.

QoS 2

Using QoS 2, we guarantee that messages are delivered to the destination only once. To do this, messages with a unique message ID are stored twice, first by the sender and then by the receiver. QoS level 2 has the highest overhead in the network because two flows are needed between the sender and receiver.

2 MQTT Packet Structure

• Fixed Header, exists in all MQTT packets, indicating the packet type and the grouping class identifier;
• Variable Header, exists in some MQTT packets, the packet type determines whether the variable header exists and its specific content;
• Payload, exists in some MQTT packets, indicating the specific content received by the client;

The overall MQTT message format is shown in the figure below;

2.1 MQTT Fixed Header

The Fixed Header exists in all MQTT packets, and its structure is as follows:

Let’s briefly analyze the message format of the fixed header;

MQTT Message Type

**Location:** byte 1, bits 7-4.

A 4-bit unsigned value, types are as follows:

Flags / DUP

**Location:** byte 1, bits 3-0.

In message types that do not use flags, the flags are reserved. If an invalid flag is received, the receiver must close the network connection:

• DUP: A duplicate of the published message. Used to ensure reliable message transmission; if set to 1, the MessageId is added in the variable length below and requires a confirmation response to ensure message transmission is completed, but cannot be used to detect duplicate message sending.

• QoS Quality of service for publishing messages (already introduced), that is: guarantees the number of message deliveries

• 00: At most once, that is: <=1
• 01: At least once, that is: >=1
• 10: Exactly once, that is: =1
• 11: Reserved

• RETAIN: Publish retain flag indicating that the server should retain this pushed message; if a new subscriber appears, it will push this message to it, if set, then release after pushing to the current subscriber.

Remaining Length

Location: byte 1.

The second byte of the fixed header is used to store the total size of the variable header and message body, but not directly stored. This byte can be extended, with the first 7 bits used to store the length, and the last part used for identification. When the last bit is 1, it indicates that the length is insufficient, requiring two bytes to continue storage. For example, if the calculated size for the following is 0

2.2 MQTT Variable Header

The MQTT packet includes a variable header, which resides between the fixed header and the payload. The content of the variable header varies depending on the packet type, with common applications serving as the packet identifier:

Many types of packets include a 2-byte packet identifier field, including:

PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP,

SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK

2.3 Payload Message Body

The Payload message body is the third part of the MQTT packet, with message bodies for the following types: CONNECT, SUBSCRIBE, SUBACK, UNSUBSCRIBE:

• CONNECT, the message body mainly includes: Client’s ClientID, subscribed Topic, Message, and username and password

• SUBSCRIBE, the message body includes a series of topics to subscribe to and QoS.

• SUBACK, the message body includes the server’s confirmation and response to the requested SUBSCRIBE topics and QoS.

• UNSUBSCRIBE, the message body includes the topics to unsubscribe from.

3 Environment Setup

Having introduced the basic theoretical part, let’s now set up a simple MQTT application on Windows platform for basic applications, with the overall architecture shown in the figure below;

[Image link failed, the source may have anti-leech mechanism, it is recommended to save the image and upload it directly (Architecture Diagram.png)]

3.1 Setting Up MQTT Server

Currently, the mainstream platforms for MQTT brokers include:

• Mosquitto: https://mosquitto.org/
• VerneMQ: https://vernemq.com/
• EMQTT: http://emqtt.io/

This article will use Mosquitto for testing. Go to the installation page and download the program that suits your computer’s operating system;

After successful installation, go to the installation path and find mosquitto.exe;

[Image link failed, the source may have anti-leech mechanism, it is recommended to save the image and upload it directly (image-20210705171401654.png)]

Hold Shift, right-click on the empty space, and then open Powershell, or simply open a terminal software;

• Input ./mosquitto.exe -h to view the corresponding help;

• Input ./mosquitto.exe -p 10086, which will start the MQTT service, listening at 127.0.0.1 on port 10086;

As shown in the figure below;

3.2 MQTT Client

With the server set up, the next step is to start the client to publish and subscribe, allowing for message transmission.

Here I am using a self-compiled QT mqtt client program, compiled based on the official Qt library. Below, I will briefly introduce how to complete this client and set the corresponding parameters:

• Address: 127.0.0.1

• Port: 10086

Then subscribe to the topic, and you can send data to each other, as shown in the figure below;

Combining with the previous images, the overall architecture is shown below;

4 Summary

This article briefly introduces the working principle of the MQTT protocol, the corresponding protocol format, and some details of the protocol, providing specific application scenarios. The author has limited skills and capabilities, and there may be errors and omissions in the text. Please forgive me for any shortcomings.

This concludes this issue. I am Xiaomai, and see you next time.

RingBuff’s Wonderful Use in Multi-Core Communication

STM32 Using DMA to Send Serial Data

Introduction to BLDC Drive: The Simplest Tutorial

Sourcetail: A Code Editing Tool that Makes Reading Source Code Smooth

Original content is not easy, welcometo share, comment, like, and share with your friends. Thank you for your support!

Long press to identify the QR code to follow me

You take every good-looking thing seriously as a like