Translator | Li RuiReviewer | Sun ShujuanThe data created by humans on the internet and computers is not always accurate; human typing, scanning, photographing, or recording is not always reliable. But what if there are sensors on machines that collect data and can communicate with other machines? What if there is a protocol that allows medical and personal devices, appliances, and other electronic devices to send and receive data from each other?For example, such a system means that people can accurately track the items in their inventory. Or the parts or items that need to be replaced, replenished, or repaired in their stores. Since sensors in devices collect data, this means the data is more accurate and reliable.With the advent of the Fourth Industrial Revolution, this system has become possible. It has brought about the Internet of Things, robotics, artificial intelligence, and other emerging disruptive technologies.
Overview of MQTTSmart TVs, smartwatches, and other “smart” devices and gadgets that can send and receive information from each other are now as common as cookies on websites. These smart devices, which can collect data through sensors and send it to other devices, are known as Internet of Things (IoT) devices.How do IoT devices achieve this? Their ability to communicate with each other is made possible through a messaging technology called Message Queue Telemetry Transport (MQTT). It is a lightweight messaging protocol based on IP that supports the Internet of Things. It enables controllers, sensors, and other gadgets or devices to “talk” to each other.Moreover, it also supports devices that are not always online, such as automation devices equipped with microcontrollers.The MQTT messaging protocol was created in 1999 by Andy Clark of IBM and Arlen Nipper of Eurotech. It was initially a proprietary protocol primarily used in the oil and gas industry, later becoming an open-source protocol. MQTT is designed to work in low-bandwidth networks. This feature is particularly suitable for machine-to-machine communication and IoT devices with a small digital footprint. It is not only suitable for low-bandwidth networks but also for high-latency environments.MQTT is used in manufacturing, healthcare, telecommunications, automotive, and other industries. It is now recognized as the standard for IoT messaging. The messaging system operates on a publisher/subscriber principle using an MQTT server. The sender and receiver do not need to be directly connected to each other.The source data is published and received by clients that subscribe to such data. The relationship between publishers and subscribers is as follows:
- The publisher (data source) transmits data or messages on a topic.
- The client subscribes to the topic to receive messages.
- The server matches publishers with subscribers.
- If no match is found, the message is discarded.
This arrangement is similar to broadcasting or television broadcasting—you can only view the transmission of the channel you are tuned to. In MQTT, the data source “publishes” data, and the receiver obtains data because they are registered subscribers.
MQTT Architecture and Data Transmission ProtocolMQTT is designed to be a simple publish/subscribe protocol for lightweight devices, consisting of two main components: the broker/server and the client.
MQTT Broker/ServerThe computer running the software that receives messages from the data source or publisher is called the broker or server. The server/broker then sends the data from the source to its correct destination, the subscriber. This software can be installed on a cloud-based broker/server using proprietary or open-source software, on a desktop computer, or on a Raspberry Pi. However, if dealing with IoT projects that can be controlled over the internet, using a cloud-based server is more appropriate.The server can manage thousands of simultaneously connected MQTT clients, depending on the implementation. Therefore, scalability and integration are essential considerations when choosing an MQTT server. In addition to receiving and sending messages, the server also has other capabilities, such as:
- Security
MQTT needs to be secured. It should require its clients to provide strong usernames and passwords. Additionally, the Transmission Control Protocol (TCP) connection should be encrypted using Secure Sockets Layer (SSL) and Transport Layer Security (TLS). These security measures should ensure the security and privacy of messages during transmission.
- Quality of Service (QoS) Levels
The Quality of Service (QoS) level is a protocol between the message source and the intended receiver. This protocol defines the delivery guarantees for specific messages. There are three levels of Quality of Service (QoS) in MQTT:(1) At most once (0)At this level, delivery is not guaranteed. The most that can be guaranteed at this level is “best effort” delivery. The message is neither stored nor resent, and the recipient does not send an acknowledgment of receipt. This QoS level is used when the internet is reliable and fast data delivery is needed.(2) At least once (1)This guarantees that the message is delivered to the subscriber at least once. The publisher stores the message until the receiver sends an acknowledgment of receipt.The server also confirms the message sent by the publisher, letting the client know that the message has been received. If the client does not receive the acknowledgment, the message will be sent again.The acknowledgment informs the subscriber that all messages related to the topic subscribed by the client have been published. While this level guarantees receipt of messages, duplicate messages may occur if the acknowledgment is not received in time.If the client needs to receive all messages on the subscribed topic, this level 2 is used. However, many commercial servers do not support level 2 because it is relatively slow and resource-intensive.(3) Exactly once (2)Level 2 is the highest Quality of Service level in MQTT. Each message is guaranteed to be received by the subscriber exactly once. It is also the most secure but slowest of the three QoS levels. While level 1 may lead to duplicate messages, level 2 guarantees that data is received only once.This guarantee is provided by two (at least two) “request and response” flows between the publisher and subscriber, indicating that the message has been received. Once the subscriber confirms receipt of the message, the publisher can discard the data packet. If the publisher does not receive a “message received” signal from the subscriber, the message will continue to be sent until it is confirmed.A higher QoS level is more reliable but involves higher bandwidth requirements and increased latency, which can slow down data transmission speed. This level is used when fast messaging is not required and duplicate messages may cause issues.In terms of QoS, messaging has two aspects: from the publisher to the server and from the server to the subscriber. Each time data is sent to the server, the source/publisher of the data sets the QoS level of the data. The server then sends the data to the subscriber client based on the client’s QoS level (set at the beginning of the subscription process). If the client’s QoS setting is lower than that of the publishing client, the server will send the lower QoS data.This feature allows MQTT to provide additional messaging Quality of Service, ensuring that data is sent according to the client’s requirements.
- Store and Forward
Some MQTT messages can be stored and forwarded to subscribers even if the subscriber client cannot receive the messages. These messages are called “store and forward topics” that clients can subscribe to. When the subscriber can receive messages, clients that have subscribed to the store and forward feature can receive the stored data.To make this feature work effectively, the MQTT server needs to perform the following actions:(1) Track the client ID of each subscriber.(2) Track the identifier of the last message successfully sent to each client.The MQTT broker automatically deletes expired messages unless configured to retain them. The broker stores configured topics in case future clients subscribe to them. These messages will then be delivered to subscribers in the order they were originally received.
MQTT ClientMQTT clients can be both publishers and subscribers. The labels “publisher” and “subscriber” refer to the clients that publish messages or subscribe to receive messages. In MQTT, the publish/subscribe functionality can be implemented in the same client. Clients can perform different functions during the communication phase, such as subscribing, unsubscribing, publishing, connecting, and disconnecting.Additionally, these clients can be any device or application, from desktop computers to fully committed cloud-based brokers/servers. Clients can be compact devices with minimal libraries and limited resources, using wireless network connections.It can be said that any device capable of running MQTT over a TCP/IP stack can be a client. The implementation of MQTT is simplified and straightforward, making it very suitable for small devices that use minimal resources.
ConclusionMQTT is a technology that enables smart appliances, smart gadgets, and other Internet of Things (IoT) devices to “converse” and share information with other machines or systems. This messaging technology allows smartphones, refrigerators, wearables, thermostats, remote sensors, and other smart gadgets to exchange data with each other.The data flow in MQTT works through a simple publisher/subscriber system with the help of the server/broker.The publisher transmits data about the topic through the server; then, the server transmits the data to the clients subscribed to that topic.
Original link:
https://www.smartdatacollective.com/how-data-flow-works-in-mq-telemetry-transport-mqtt/