Beginner Friendly | Quick Steps to Achieve MQTT Communication with Cloud Platform

The MQTT protocol is currently the most popular Internet of Things protocol in the world, bar none. It provides devices with a stable, reliable, and easy-to-use communication foundation. As of now, there are over a hundred million devices connected via the MQTT protocol, widely used in fields such as IoT and M2M.

Currently, the mainstream versions of MQTT are MQTT 3.1.1 and MQTT 5. MQTT 5 is fully compatible with MQTT 3.1.1 and is an enhancement based on MQTT 3.1.1. As of now, more people still use MQTT 3.1.1, so this article will explain using MQTT 3.1.1.

MQTT (Message Queuing Telemetry Transport) is an asynchronous communication message protocol built on the TCP/IP protocol stack. It is a lightweight, client-server architecture, publish/subscribe messaging protocol.

The initial version of the MQTT protocol was established in 1999, invented by Andy Stanford-Clark and Arlen Nipper. MQTT was developed to monitor oil pipelines using satellite communication, indicating that MQTT is designed specifically for low bandwidth, high latency, or unreliable networks.

Features of the MQTT protocol:

• Simple and easy to use, convenient for integration

• Secure and reliable, supporting TLS/SSL encryption and authentication mechanisms

• Lightweight, occupies little bandwidth, supports various messaging transmission modes

• Flexibility, can know device connection status, controllable data transmission quality

In MQTT protocol communication, the two most important roles are the server and the client. The client publishes messages to a “topic,” and the server processes and pushes them to other clients that have “subscribed” to that “topic.”

Is this a bit confusing? Let me give you an analogy by comparing the entire MQTT to a familiar video software, with the following one-to-one correspondence.

If you are Zhang San, an ordinary Douyin user, and you follow Li Si’s Douyin account. Here, Zhang San and Li Si do not directly interact; instead, they go through the Douyin server. The Douyin server is the “server,” and all Douyin users are the “clients.” Your action of following Li Si is called “subscribing.”

If Li Si “publishes” a video, then Zhang San, Wang Wu, Lao Liu, and all other fans who follow Li Si will receive this video push. This is because there is no concept of a topic in Douyin; as long as Li Si posts a video, fans will receive the push.

This is the basic concept of MQTT.

2.1 Server

The MQTT server is usually a server that acts as the central node for MQTT information transmission. Its main function is to receive information from MQTT clients and pass it on to other MQTT clients. Additionally, the MQTT server is responsible for managing clients, ensuring smooth communication between them, and ensuring that MQTT messages are correctly received and accurately delivered.

The server is generally a cloud platform, such as OneNET, Alibaba Cloud, Tencent Cloud, etc.; it can also be self-built using EMQ or Mosquitto.

2.2 Client

The MQTT client can send information to the server by “publishing” and can also receive information by “subscribing” from the server.

The client is usually our microcontroller, such as STM32, C51, Raspberry Pi, etc.

2.3 Topic

In MQTT communication, clients subscribe to various “topics.” The MQTT server uses “topics” to control information communication.

2.4 Characteristics of Publish and Subscribe

Independence: Clients are mutually independent, with no direct connections and do not need to know each other’s status or condition.

Space separation: Clients can communicate as long as they connect to the same MQTT communication network, whether it is the Internet or a local area network.

Time asynchronous: The publishing and subscribing of clients do not need to be synchronized. If a client disconnects, the server saves the information and pushes it when the client comes online.

The MQTT protocol communicates by exchanging predefined MQTT control messages. MQTT control messages are referred to as MQTT messages, and I will detail MQTT messages next.

3.1 Message Structure

An MQTT message consists of three parts: fixed header, variable header, and payload:

The fixed header (Fixed header) is present in all MQTT messages and indicates the message type and grouping class identifier. The variable header (Variable header) is present in some MQTT messages, and its specific content depends on the message type. The payload/message body (Payload) is present in some MQTT messages and contains the specific content of the message. The diagram is as follows:

With the overall message structure introduced, I will now detail each part. If you don’t understand, that’s okay; there will be examples later that will clarify things.

3.2 Fixed Header (Fixed header)

3.2.1 Message Type (message type)

Located in bits 7-4 of byte 1, indicating the MQTT message type, there are several types:

3.2.2 Flags (DUP, QoS Level, RET)

Located in bits 3-0 of byte 1, indicating the grouping class identifier of the MQTT message. In message types that do not use flags, the flags are reserved. If invalid flags are received, the receiving end must close the network connection.

DUP: Duplicate of the published message

QoS: Quality of service for the published message

RETAIN: Retain flag for the published message

3.2.3 Remaining Length

Located in bits 3-0 of byte 2, indicating the current remaining byte count, including the variable header and payload data. The remaining length does not include the bytes used to encode the remaining length field itself.

3.3 Variable Header (Variable header)

Some MQTT messages have a variable header. It is located between the fixed header and the payload. The content of the variable header varies depending on the message type. The variable header message identifier field exists in multiple types of messages.

The variable header will be detailed in subsequent message examples.

3.4 Payload (Payload)

The payload is the application message, but not all messages have a payload; only some MQTT messages have a payload, as detailed below:

QoS (Quality of Service). In the process of data communication, some messages are very important and cannot be lost; others are not important, and losing them is acceptable. Therefore, in MQTT, QoS can be configured to provide different service qualities for messages of varying importance.

The MQTT protocol has three levels of service quality:

QoS = 0: At most once

QoS = 1: At least once

QoS = 2: Exactly once. For messages of different importance, QoS = 1 and QoS = 2 are used for more important messages.

4.1 QoS = 0: At Most Once

This quality of service message is sent at most once. The receiver will not send a response, and the sender will not retry. The message may be delivered once or may not be delivered at all.

Imagine you are a courier, and you need to deliver packages (messages) to different recipients (subscribers). The QoS level is like the delivery service level between you and the recipient, determining the guarantees you provide during the delivery process.

QoS 0 (at most once) is like you leaving a package at the recipient’s door without any confirmation receipt. You simply leave the package at the door and walk away. In this case, you cannot confirm whether the package was successfully received by the recipient or if someone else stole it.

4.2 QoS = 1: At Least Once

This quality of service ensures that the message is delivered at least once. The variable header of the QoS 1 PUBLISH message contains a message identifier, which requires a PUBACK message for confirmation.

QoS 1 (at least once) is like you asking the recipient for a receipt confirmation after delivery. You deliver the package to the recipient and wait for them to give you a receipt confirming that they have received the package. If you do not receive the receipt, you will retry delivery until you receive the receipt. This way, you can ensure the package is received by the recipient, but it may add some delay and workload.

4.3 QoS = 2: Exactly Once

This is the highest level of service quality, where message loss and duplication are unacceptable. Using this service quality level incurs additional overhead. The variable header of QoS 2 messages contains a message identifier. Recipients of QoS 2 PUBLISH messages use a two-step confirmation process to confirm receipt.

QoS 2 (exactly once) is like confirming with the recipient that they can receive the package before sending it. Before delivery, you send a message to the recipient asking if they are home. If the recipient tells you they are home, you deliver the package and wait for them to give you a receipt confirming they have received it. If they do not reply and are not home, you continue to message them until they respond that they are home before delivering the package.

The MQTT heartbeat mechanism can be compared to the heartbeat of a human body; both are meant to maintain normal operation and connection stability.

When the MQTT client periodically sends heartbeat packets, it is like our heart, regularly sending signals to the server to indicate its presence and health status. If the server does not receive a heartbeat packet within a certain time, it will assume the client has encountered an issue or is offline, similar to how a doctor checks the heartbeat to assess the health condition when there is a problem.

The client periodically sends heartbeat requests (PINGREQ) to the server to indicate, “I am still connected to you.” When the server receives the heartbeat request, it responds with a heartbeat response (PINGRESP) to inform the client, “I know you are still connected to me.”

Through the heartbeat mechanism, MQTT can monitor the connection status of clients in real time, promptly discover and handle abnormal situations, ensuring the reliability and stability of communication. Just as we rely on our heart to maintain normal bodily functions, the MQTT heartbeat mechanism is one of the important mechanisms to ensure smooth communication links.

The last will, like the previous heartbeat, proves that the client is still connected to the server. The last will becomes vivid; the client first sends its last will to the server, so if the client goes down, the server can execute the last will.

MQTT Last Will is a mechanism that allows clients to set and send a last will message while they are “alive,” so that it can be published by the server in case of an unexpected disconnection.

Unexpected disconnection refers to when the client loses the heartbeat signal without sending a DISCONNECT message, which usually occurs in cases of network failure or battery depletion. At this time, the server will detect the abnormal disconnection of the client and publish the client’s last will message. However, if the client disconnects normally and sends a DISCONNECT message, the last will will not be triggered, and the server will not publish the client’s last will message.

By properly setting and using the MQTT last will mechanism, the client’s role in server management can be enhanced, providing real-time device status information.

Log into OneNet platform and create a DHT11 device as follows:

Record:

Device ID: dht11_01

Product ID: 16o95O4eF8

Device Key:

bk9vd3h3VzFxTk1PMHdKMHNYbHdsUmFIcFdKZFZxd1g=

ESP8266 Initialization:

void esp8266_init(uint32_t baudrate){    printf("esp8266 initialization starting...\r\n");    esp8266_uart_init(baudrate);        // Other ESP8266 initialization    printf("1. Testing if esp8266 exists...\r\n");    while(esp8266_at_test())        delay_ms(500);        printf("2. Setting working mode to STA...\r\n");    while(esp8266_set_mode(ESP8266_STA_MODE))        delay_ms(500);        printf("3. Setting single connection mode...\r\n");    while(esp8266_connection_mode(ESP8266_SINGLE_CONNECTION))        delay_ms(500);        printf("4. Connecting to Wi-Fi, SSID: %s, PWD: %s\r\n", WIFI_SSID, WIFI_PWD);    while(esp8266_join_ap(WIFI_SSID, WIFI_PWD))        delay_ms(1500);        printf("5. Connecting to TCP server, server_ip:%s, server_port:%s\r\n", TCP_SERVER_IP, TCP_SERVER_PORT);    while(esp8266_connect_tcp_server(TCP_SERVER_IP, TCP_SERVER_PORT))        delay_ms(500);        printf("6. Entering transparent mode..\r\n");    while(esp8266_enter_unvarnished())        delay_ms(500);        printf("ESP8266 has connected to the TCP server and entered transparent mode\r\n");    printf("ESP8266 initialization completed\r\n");}

void mqtt_init(void){    mqtt_login_init(PRODUCT_KEY,DEVICE_NAME,DEVICE_SECRET);    // Buffer assignment    mqtt_rxbuf = _mqtt_rxbuf;    mqtt_rxlen = sizeof(_mqtt_rxbuf);    mqtt_txbuf = _mqtt_txbuf;    mqtt_txlen = sizeof(_mqtt_txbuf);    memset(mqtt_rxbuf,0,mqtt_rxlen);    memset(mqtt_txbuf,0,mqtt_txlen);        // Unconditionally disconnect first    mqtt_disconnect();    delay_ms(100);    mqtt_disconnect();    delay_ms(100);}

// Cloud server device certificate#define PRODUCT_KEY "16o95O4eF8"#define DEVICE_NAME "dht11_01"#define DEVICE_SECRET "J6OmW6IjqBcXO9beNPOdkxI3Ves%3D"
// Subscribe and publish topics#define RELY_PUBLISH_TOPIC  "$sys/16o95O4eF8/dht11_01/thing/property/set_reply"  #define SET_TOPIC  "$sys/16o95O4eF8/dht11_01/thing/property/set"#define POST_TOPIC "$sys/16o95O4eF8/dht11_01/thing/property/post"

Obtaining DHT11 identifier and implementing JSON parsing data upload to cloud platform:

#include "sys.h"
#include "delay.h"
#include "uart1.h"
#include "dht11.h"
#include "esp8266.h"
#include "onenet.h"
int main(void){    uint8_t data_send[512] = {0};    uint8_t dht11_data[4] = {0};        while(1)    {             memset(dht11_data, 0, 4);        dht11_read(dht11_data);        sprintf((char *)data_send, "{\"id\":\"1386772172\",\"version\":\"1.0\",\"params\":{\"CurrentTemperature\":{\"value\":%d.%d},\"CurrentHumidity\":{\"value\":%d.%d}}}",            dht11_data[2], dht11_data[3], dht11_data[0], dht11_data[1]);              mqtt_publish_data(POST_TOPIC, (char *)data_send, 0);                delay_ms(3000);                printf("\r\n~~~~~~~~~~~~~~~~~Sending heartbeat packet~~~~~~~~~~~~~~~~~\r\n");        mqtt_send_heart();        printf("\r\n~~~~~~~~~~~~~~~~~Heartbeat packet ended~~~~~~~~~~~~~~~~~\r\n");    }}

Experimental Results: