Old Jun is currently engaged in the Internet of Things industry and has a certain understanding of MQTT. This article briefly describes what MQTT is and how to use it.

What Is MQTT

The Internet of Things (IoT) and machine-to-machine (M2M) communication have had a lasting impact on how we interact with technology. By the end of 2022, the number of globally connected IoT devices grew by 18% to 14.4 billion, and it is estimated that the number of connected IoT devices will reach approximately 27 billion by 2025. With this growth, messaging protocols capable of handling real-time, reliable, and efficient communication between devices and processing systems (often located in the cloud) are essential, hence the emergence of the MQTT protocol.

• MQTT (MQ Telemetry Transport) is a lightweight, open messaging protocol that provides a simple way for resource-constrained network clients to distribute telemetry information in low-bandwidth environments. The protocol uses a publish/subscribe (PUSH/SUBSCRIBE) communication model for machine-to-machine (M2M) communication.
• MQTT is a low-overhead protocol created to adapt to bandwidth and CPU limitations, designed to run in embedded environments where it can provide reliable and efficient communication paths. MQTT is suitable for connecting devices with small code footprints and is a good choice for wireless networks experiencing different levels of latency due to occasional bandwidth limitations or unreliable connections. The protocol is applied in IoT industries such as automotive, energy, and telecommunications.
• Although MQTT was originally a proprietary protocol used for communication with monitoring and data acquisition (SCADA) systems in the oil and gas industry, it has become popular in the smart device field, and today it is the leading open-source protocol for connecting IoT devices.
• Initially, the “MQ” in “MQTT” came from IBM’s MQ (then known as MQSeries) product line, representing “Message Queue.” However, despite its name, the protocol does not use message queues; instead, it provides publish and subscribe messaging: devices publish messages about specific topics, and all devices subscribed to that topic receive the message. Its main applications include sending messages to control outputs and reading and publishing data from sensor nodes.

How MQTT Works

The working principle of MQTT can be summarized as follows: MQTT implements the publish/subscribe model through its two main components: clients and brokers.

• Client is a device or application that publishes or subscribes to messages, making it a publisher or subscriber. Clients can publish messages to topics, which are logical channels to which messages are sent. Subscribers can be one or more clients that receive messages published to the topic. Clients can also act as both publishers and subscribers simultaneously.
• On the other hand, the broker is an intermediary server that routes messages between clients. The broker manages message distribution between clients by tracking subscriptions and publishing messages to subscribed clients. MQTT clients can be implemented in various programming languages and can run on various hardware platforms, from low-power microcontrollers to powerful servers. Brokers can be deployed in the cloud or on-premises and can scale horizontally to support millions of devices and messages per second. Large-scale applications such as Alibaba Cloud, Huawei Cloud, AWS, or Azure provide MQTT wrappers around their services, offering varying levels of support for the MQTT protocol.

What Is MQTT? A Detailed Explanation of MQTT Protocol in IoT — image.png

Messages are shared with other devices or software using the broker of MQTT. Each message has a topic, and the broker can further process the message based on that topic. Additionally, each message contains a message content known as the payload. MQTT payloads are not bound to a specific structure and can be freely designed. However, specifying a specific structure for message content is helpful so that it can be read by other devices or software. Potential message structures can be JSON, XML, or OPC UA. As long as all devices and software communicate using the same structure, the defined structure can achieve smooth internal communication.When a subscribing client loses connection to the broker, the broker can, depending on the use case and implementation – retain any messages intended for the subscriber and immediately deliver them when the subscriber reconnects.This ensures that subscribers do not miss any messages and receive them in the correct order, commonly referred to as the will mechanism.

MQTT Quality of Service Levels (QoS)

QoS refers to the protocol between the message sender and the message receiver. It is a key feature in MQTT that allows clients to choose among three service levels. The three different QoS levels determine how the MQTT protocol manages content. Although higher QoS levels are more reliable, they have more latency and bandwidth requirements, so subscribing clients can specify the highest QoS level they wish to receive.

• The simplest QoS level is unacknowledged service. This QoS level uses the PUBLISH packet sequence; the publisher sends a message to the broker once, and the broker delivers the message to the subscriber once. There is no proper mechanism to ensure that the message has been correctly received, and the broker does not store the message, this QoS level can also be referred to as at most once, QoS0 or fire and forget.
• The second QoS level is acknowledged service. This QoS level uses the PUBLISH/PUBACK packet sequence between the publisher and its broker and between the broker and the subscriber. The acknowledgment packet verifies whether the content has been received, and if acknowledgment is not received in a timely manner, a retry mechanism resends the original content. This may result in subscribers receiving multiple copies of the same message. This QoS level can also be referred to as at least once or QoS1.
• The third QoS level is guaranteed service. This QoS level uses two pairs of packets to deliver messages. The first pair is called PUBLISH/PUBREC, and the second pair is called PUBREL/PUBCOMP. These two pairs ensure that, regardless of how many retries are made, the message will be delivered only once. This QoS level can also be referred to as exactly once or QoS2.

MQTT Protocol Packet Structure

MQTT is a binary-based protocol where control elements are binary bytes rather than text strings.

The command and command acknowledgment formats are used by MQTT, meaning each command has an accompanying acknowledgment. Connection commands have connection acknowledgments, subscription commands have subscription acknowledgments, and publish commands have publish acknowledgments, as shown in the diagram above.This mechanism is similar to the handshake mechanism of the TCP protocol.

MQTT message format consists of 3 fields.

• Fixed Header must exist for all MQTT packets, occupying a fixed size of 2 bytes.
• Variable Header may not always exist; sometimes this part is absent.
• Payload is where the data being sent is stored, however, the payload part may not always exist as some commands, such as disconnect messages, do not use the payload field.

Fixed Header Message Parsing

The fixed header contains 2 bytes,the first byte’s first 4 bits (Bit7Bit4) represent the message type, and the last 4 bits (Bit3Bit0) indicate the transport protocol and QoS level,the second byte is used to store the packet length,this length = variable header length + payload length.

MQTT Control Packet Types:

Name	Value	Direction of Flow	Description
Reserved	0	Forbidden	Reserved
CONNECT	1	Client To Server	Connection Request
CONNACK	2	Server To Client	Connection Acknowledgement
PUBLISH	3	Client To Server Or Server To Client	Publish Message
PUBACK	4	Client To Server Or Server To Client	Publish Acknowledgement
PUBREC	5	Client To Server Or Server To Client	Publish Received
PUBREL	6	Client To Server Or Server To Client	Publish Release
PUBCOMP	7	Client To Server Or Server To Client	Publish Complete
SUBSCRIBE	8	Client To Server	Subscribe Request
SUBACK	9	Server To Client	Subscribe Acknowledgement
UNSUBSCRIBE	10	Client To Server	Unsubscribe request
UNSUBACK	11	Server To Client	Unsubscribe Acknowledgement
PINGREQ	12	Client To Server	Ping Request
PINGRESP	13	Server To Client	Ping Response
DISCONNECT	14	Client To Server Or Server To Client	Disconnect Notification
AUTH	15	Client To Server Or Server To Client	Authentication Exchange

Flag Bit:

MQTT Control Packet	Fixed Header Flags	Bit 3	Bit 2	Bit 1	Bit 0
CONNECT	Reserved	0	0	0	0
CONNACK	Reserved	0	0	0	0
PUBLISH	MQTT v5.0	DUP	QoS	RETAIN
PUBACK	Reserved	0	0	0	0
PUBREC	Reserved	0	0	0	0
PUBREL	Reserved	0	0	0	0
PUBCOMP	Reserved	0	0	0	0
SUBSCRIBE	Reserved	0	0	0	0
SUBACK	Reserved	0	0	0	0
UNSUBSCRIBE	Reserved	0	0	0	0
UNSUBACK	Reserved	0	0	0	0
PINGREQ	Reserved	0	0	0	0
PINGRESP	Reserved	0	0	0	0
DISCONNECT	Reserved	0	0	0	0
AUTH	Reserved	0	0	0	0

Variable Header Parsing

The variable header component is included as an optional field in certain MQTT control packet types. This field is located between the fixed header and the payload. The content of the variable header is determined by the packet type. Many packet identifier fields common to various packet types can be found in the variable header. The packet identifier field is a 2-byte integer included in the variable header component of many MQTT control packet types. The characteristics of the packet identifier field are:

• PUBLISH
• PUBACK
• PUBREC
• PUBREL
• PUBCOMP
• SUBSCRIBE
• SUBACK
• UNSUBSCRIBE
• UNSUBACK

Features of the packet identifier field:

1. If the QoS (Quality of Service) value is set to zero, the PUBLISH packet should not include the packet identifier field. This means that if the QoS value is greater than zero, the packet identifier field will only appear in the PUBLISH packet.
2. The client should assign a currently unused non-zero packet identifier when sending new SUBSCRIBE, UNSUBSCRIBE, or PUBLISH MQTT control packets.
3. The server should assign a currently unused non-zero packet identifier when sending new PUBLISH MQTT control packets.
4. PUBACK, PUBREC, PUBREL, and PUBCOMP are PUBLISH command acknowledgment packets, and the packet identifier is the same as that of the PUBLISH packet.
5. SUBACK and UNSUBACK are acknowledgment packets for SUBSCRIBE and UNSUBSCRIBE, respectively. The SUBACK and UNSUBACK packets share the same packet identifier as the SUBSCRIBE and UNSUBSCRIBE packets.
6. After processing the corresponding acknowledgment packet, the packet identifier can be reused. Here is a definition:

A. If QoS is set to 1, the acknowledgment packet for PUBLISH will be PUBACK. If PUBACK is processed, the PUBACK packet identifier can be reused. B. If the QoS value is 2, the acknowledgment packet for PUBLISH will be PUBCOMP or PUBREC.

Payload

The payload is the last MQTT control packet in the message format. This field holds the data to be sent.

MQTT Control Packet	Payload
CONNECT	Required
CONNACK	None
PUBLISH	Optional
PUBACK	None
PUBREC	None
PUBREL	None
PUBCOMP	None
SUBSCRIBE	Required
SUBACK	Required
UNSUBSCRIBE	Required
UNSUBACK	Required
PINGREQ	None
PINGRESP	None
DISCONNECT	None
AUTH	None

New Features of MQTT 5.0

The MQTT protocol is the most widely used and popular TCP/IP IoT protocol in the world today. Unsurprisingly, the widespread adoption of the protocol has led to a high demand for further development of the MQTT specification. MQTT 5 aims to meet this demand. Popular use cases for MQTT v5 include manufacturing systems and logistics, smart home applications, as well as enterprise IoT applications and mobile applications. The new features in MQTT 5.0 are designed to achieve the following goals:

• Large-scale system performance: Communication between thousands (if not millions) of devices is now more streamlined. There are no protocol constraints, but the appropriate architecture for MQTT 5 is needed to organize so many devices.
• Error reporting: The return code in MQTT v5.0 has been renamed to reason code, which can indicate a wider range of failures.
• Common interactions implemented: The current version has standardized many ways for devices to interact with each other. The system now includes the ability to define the capabilities of participating devices and how they respond to queries.
• Includes scalability mechanisms: Custom attributes, as well as content types or payload formats, can now be specified.
• Better support: Particularly suitable for small users who wish to use this latest protocol version to increase efficiency.

What Are the Differences Between MQTT 5 and MQTT 3

MQTT v5.0 is the successor to MQTT 3.1. Most importantly, MQTT v5.0 does not support backward compatibility (like v3.1.1). Due to the introduction of so many new items, existing implementations must be reevaluated. Here are some differences between V5 and V3:

• The Session Expiry Interval attribute can be used to implement session expiration. For example, the subscription time may be included in the topic, as well as the amount of time the message will be retained.
• Limitations on MQTT clients and servers can include maximum packet size (broadcast byte count) and maximum reception limits (number of messages sent simultaneously by the client or server).
• The Will Delay Interval is an attribute that allows you to send messages on a timer.
• Server redirection or server reference is an attribute that helps transfer packets between brokers or servers.
• The Message Expiry Interval is the amount of time a message is stored before expiration.
• The payload format indicator and content type – the types of payload flags that can be used are defined by this attribute: byte (binary), UTF-8, or MIME type.
• Topic alias – for example, alias 1 assigned to a topic. This feature can help you reduce the number of packets you need.
• Response topic – when using a response topic, the protocol works similarly to HTTP protocols with a request-response scheme.
• Non-local publishing – users can choose whether to receive messages from clients.
• Retained Message Control – the order of messages is controlled by this option.
• Subscription identifiers for identifying subscriptions on the server.
• Shareable subscriptions allow for more flexible subscriptions with additional symbols and filtering options.
• Reason codes on all ACK messages – errors can occur at any time during the process. In MQTT v3.1.1, there was little information from the server about what went wrong at various stages such as establishing communication, sending messages, or subscribing to topics.
• Server disconnection – unlike 3.1, MQTT 5.0 allows sending disconnect data packets from both the client and server sides.
• User attributes – keys can be used to store values of various attributes.
• Enhanced authentication – a mechanism for query/response authentication methods that performs mutual authentication between client and server.

For most IoT use cases, the MQTT 5 specification has become a logical choice. The new features of MQTT 5 successfully address the limitations of version 3 while also paving the way for future development. The adoption of the protocol is expected to surge across all industries in the coming years, including industrial, automotive, critical infrastructure, logistics, smart cities, and more. MQTT is set to become the industry standard for all things connected to the IoT.