Zaiming (Stone) Shi: Head of EMQX European R&D Center, with years of experience in developing large-scale distributed MQTT messaging systems, dedicated to collaborating with the EMQ Stockholm R&D team to make EMQX the world’s best MQTT broker.
MQTT is the standard protocol for IoT messaging, utilizing an extremely lightweight publish-subscribe messaging model to connect IoT devices in a scalable, reliable, and efficient manner.
Since IBM released MQTT over twenty years ago, and since EMQ released the open-source MQTT message server EMQX on GitHub in 2012, a decade has passed. Now, in 2023, as emerging technologies are rapidly advancing,with the growing scale of MQTT usage in IoT and increasing scenario diversity, we can foresee the following 7 development trends in the MQTT technology field.
QUIC (Quick UDP Internet Connections) is a new transport protocol developed by Google, running on top of UDP, aimed at reducing latency in establishing new connections, improving data transfer rates, and addressing some limitations of TCP.
The next-generation internet protocol HTTP/3 uses QUIC as its underlying transport protocol, providing lower latency and a better loading experience compared to HTTP/2.
MQTT over QUIC is the most innovative advancement in the MQTT protocol since the release of the MQTT 5.0 specification in 2017. With advantages such as multiplexing, faster connection establishment, and migration, it has the potential to become the next-generation MQTT protocol standard.
MQTT 5.0 defines three transport types: TCP, TLS, and WebSocket. In IoT security best practices, MQTT over TLS/SSL is widely used in production environments to secure communication between clients and brokers. However, it is slow and has high latency, requiring 3.5 RTTs, including 3 TCP handshakes and 4 TLS handshakes, to establish a new MQTT connection.
Compared to MQTT over TLS/SSL, MQTT over QUIC is faster and has lower latency, needing only 1 RTT for initial connection establishment, and can utilize 0 RTT connection recovery features to speed up reconnections. The QUIC protocol stack can be customized for various use cases, such as in unstable network environments or low-latency communication scenarios from client to server. It can play a significant role in IoV (Internet of Vehicles) under mobile networks and IIoT (Industrial IoT) scenarios that require extremely low latency, effectively enhancing the user experience.
The open-source MQTT message server EMQX introduced support for MQTT over QUIC in its latest version 5.0, making it the world’s first MQTT message server to support MQTT over QUIC. Currently, EMQ is actively promoting the standardization of MQTT over QUIC as a member of the OASIS MQTT Technical Committee, and it is foreseeable that in the near future, MQTT will also use QUIC as its primary transport layer, just like HTTP/3.
The rise of the Serverless model in cloud computing marks a groundbreaking paradigm shift in how applications are designed, developed, deployed, and operated. In this model, developers can focus on the business logic of applications without managing infrastructure, thereby increasing agility, scalability, and cost-effectiveness.
Serverless MQTT message servers will be a cutting-edge architectural innovation in 2023. Traditional IoT applications may take minutes or even hours to deploy MQTT messaging services in the cloud or in private enterprise environments, whereas Serverless MQTT can complete the deployment of MQTT services in just a few clicks.
In addition to extremely fast deployment speed, the greater value of Serverless MQTT lies in its unparalleled flexibility: seamless scaling of resources based on user demand and a pay-as-you-go pricing model that matches this elastic architecture. Serverless MQTT is expected to drive wider application of MQTT, reduce operational costs, and stimulate innovative collaboration across various industries. We may even see every IoT and IIoT developer having access to a free Serverless MQTT message server.
In March 2023, EMQX Cloud launched the world’s first Serverless MQTT service, providing users with 5-second rapid deployment and more flexible billing options, helping users efficiently develop IoT applications at a lower cost.
Multitenant architecture is an important foundation for implementing Serverless MQTT services. IoT devices from different users or tenants can connect to the same large-scale MQTT cluster while keeping their data and business logic isolated from other tenants.
Multitenant architecture is common in SaaS applications, where a single application serves multiple customers or tenants. It typically has two different implementation methods:
-
Tenant Isolation: Providing each tenant with a separate application instance running on a server or virtual machine.
-
Database Isolation: Multiple tenants share a single application instance, but each tenant has its own database schema to ensure data isolation.
In the multitenant architecture of MQTT Brokers, each device and tenant has a separate, isolated namespace, including a unique topic prefix and an access control list (ACL) to define which topics users can publish or subscribe to.
A multitenant MQTT message server can reduce management overhead and flexibly support complex scenarios or large-scale IoT application scenarios. For example, departments and applications within a large organization can use the same MQTT cluster as different tenants.
MQTT Sparkplug is an open standard specification designed by the Eclipse Foundation, with its latest version being MQTT Sparkplug 3.0, which defines a unified data access specification for industrial devices, enabling connection of various industrial sensors, actuators, programmable logic controllers (PLCs), and gateways via the MQTT protocol.
MQTT Sparkplug 3.0 was released in November 2022 and features the following key new functionalities and improvements:
-
Support for MQTT 5.0: Added support for MQTT 5.0, including new features such as shared subscriptions, message expiration, and traffic control.
-
Optimized Data Transmission: Optimized data transmission using more compact data encoding and compression algorithms.
-
Extended Data Model: Introduced an extended data model that allows for more detailed device information communication and supports the transmission of configuration data and device metadata.
-
Higher Security: Included several improvements to security, such as support for mutual TLS authentication and optimized access control mechanisms.
-
Simplified Device Management: Included automatic device registration and discovery, simplified device configuration, and improved diagnostics.
MQTT Sparkplug aims to simplify the connection and communication between different industrial devices, enabling efficient industrial data collection, processing, and analysis. With the release of the new version, MQTT Sparkplug 3.0 is expected to see wider application in the field of industrial IoT.
The Unified Namespace is a solution architecture built on MQTT Brokers aimed at industrial IoT and Industry 4.0. It provides a unified namespace for MQTT topics and a centralized repository for messages and structured data.
The Unified Namespace uses a central MQTT Broker to connect industrial devices, sensors, and applications, such as SCADA, MES, and ERP, in a star topology. The Unified Namespace significantly simplifies the development of industrial IoT applications with its event-driven architecture.
In traditional industrial IoT systems, OT and IT systems are often separate, with their data, protocols, and tools running independently. By adopting a Unified Namespace, OT and IT systems can exchange data more effectively, ultimately achieving the unification of OT and IT in the IoT era.
Now, users can build a Unified Namespace architecture supported by the most advanced technologies from the IT industry through the open-source MQTT message server EMQX or NanoMQ, combined with the industrial protocol gateway software Neuron.
MQTT Geo-Distribution is an innovative architecture that allows MQTT Brokers deployed in different regions or clouds to work together as a single cluster. Through Geo-Distribution, MQTT messages can be automatically synchronized and transmitted between MQTT Brokers in different regions.
There are two methods to achieve MQTT Geo-Distribution:
-
Single Cluster, Multiple Regions: A single MQTT cluster with each node running in different regions.
-
Multiple Clusters, Multi-Cloud: Multiple MQTT clusters distributed across different clouds connected together.
We can combine these two methods to create a reliable IoT data infrastructure among MQTT Brokers deployed across regions. With MQTT Geo-Distribution, enterprises can establish a global MQTT access network across multiple clouds. Regardless of their physical location, devices and applications can communicate with each other through the nearest node.
MQTT Streams is a highly anticipated extension capability of the MQTT protocol that enables real-time processing of massive, high-frequency data streams within MQTT Brokers. This further enhances the capabilities of traditional MQTT Brokers based on the publish-subscribe messaging model. With MQTT Streams, clients can produce and consume MQTT messages in a streaming manner like Apache Kafka, enabling historical message replay. This is crucial for event-driven processing, ensuring eventual data consistency, auditability, and compliance.
Stream processing is essential for extracting business value in real-time from the massive amounts of data generated by IoT devices. Previously, this process was achieved through an outdated and complex big data stack that required integration between MQTT Brokers and Kafka, Hadoop, Flink, or Spark.
With built-in stream processing, MQTT Streams simplifies the IoT data processing architecture, improving data processing efficiency and response time, and providing a unified messaging and stream processing platform for IoT. Through features like message deduplication, message replay, and message expiration, MQTT Streams achieves high throughput, low latency, and fault tolerance, making it a powerful tool for real-time data stream processing in MQTT-based IoT applications.
In summary, these 7 technical trends of MQTT reflect the advancements of emerging technologies and their significant role in driving the development of IoT.
As a standard messaging protocol that has evolved for over twenty years, the importance of MQTT continues to grow. With the increasing widespread application of IoT across industries, the MQTT protocol is also continuously evolving to meet new challenges, including the demand for lower-latency connections, more convenient deployment of MQTT services, flexible management in complex scenarios or large-scale IoT applications, and access for industrial devices. As the nervous system of the vast IoT, MQTT will undoubtedly play a crucial role in key areas such as industrial IoT and IoV in 2023 and beyond..
