If the number of devices reaches one million, the database will have to handle two million authentication records, which will significantly affect the database’s performance. Is there a batch way to define ACL authentication?
1. ACL Authentication Rules
Under normal business usage, the behavior of clients can be restricted using ACL. For example, Client A can only subscribe to messages from the /A/get queue and publish content to /A/set. However, processing such authentication in MySQL requires writing two records. If there are one million devices, the database will have to handle two million authentication records, which will significantly affect performance. So, is there a batch way to define ACL authentication?
In the MySQL ACL authentication configuration file, the SQL for using authentication can be edited, meaning you can implement batch ACL authentication rules via SQL.
> vim /usr/local/emqttd/etc/plugins/emq_auth_mysql.conf
# At the bottom, there is a configuration like this
auth.mysql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = ‘%a’ or username = ‘%u’ or username = ‘$all’ or clientid = ‘%c’
Here, the author implements that each device can by default subscribe to messages from the /A/get queue and publish to /A/set.
The current rule is that the client can only write messages to hello, and other operations are not allowed. We will first add two records.
Then modify the default ACL authentication SQL statement as follows (using username as the dynamic topic name, other fields can also be used):
select allow, ipaddr, username, clientid, access, REPLACE(topic,’$user’,’%u’) from mqtt_acl where ipaddr = ‘%a’ or username = ‘%u’ or username = ‘$all’ or clientid = ‘%c’
This way, even if there is no independent configuration for /A/set, the client as user A can still write messages and listen to messages from /A/get.
2. Shared Subscriptions
In common usage scenarios for queues, there are cases where a message is desired to be received by multiple listeners. Possible scenarios include:
-
One program processes, while another logs the results.
-
Batch pushing.
Multiple messages may need to be processed by one of several programs. For example:
-
In concurrent situations, time-consuming operations can be processed in parallel to improve system throughput.
By default, when multiple clients listen to an event, they will receive the same message. But how do we implement shared subscriptions? EMQ supports two usage methods for shared subscriptions:
-
$queue/ like: $queue/topic
-
$share/<group>/ like: $share/group/topic
Both methods can achieve shared subscriptions (the author tested and found that using share completed the subscription), listening and monitoring.
-
Multiple servers listen to $share/group/topic.
-
Clients send messages to the topic.
3. Differences Between QoS 0/1/2
At most once delivery: Messages are transmitted over TCP/IP networks. There is no acknowledgment, and the protocol does not define retransmission semantics. Messages may reach the server once, or may not reach at all.
At least once delivery: When the server receives a message, it is acknowledged by sending a PUBACK message. If a recognizable transmission failure occurs, whether due to communication connection issues or the sending device, or if the acknowledgment is not received within a certain time, the sender will set the DUP flag in the message header to 1 and resend the message.
Messages will reach the server at least once. SUBSCRIBE and UNSUBSCRIBE both use QoS level 1. If the client does not receive the PUBACK message (whether due to application-defined timeout or detecting failure and restarting the communication session), the client will resend the PUBLISH message and set the DUP flag to 1.
When the server receives duplicate data from the client, it will resend the message to the subscribers and send another PUBACK message.
The author implemented a consumer that blocks for 2 seconds to consume one content, while the publisher publishes one content every second. After EMQ’s maximum congestion is used up, there will be many duplicate messages in EMQ’s cache.
Exactly once delivery: The protocol flow associated with QoS level 1 ensures that duplicate messages are not delivered to the receiving application. This is the highest level of delivery and is used when duplicate messages are not allowed. This increases network traffic, but it is generally acceptable since the message content is critical. QoS level 2 includes a Message ID in the message header.
4. Password Salting
In user authentication, hash methods such as plain | md5 | sha | sha256 | bcrypt can be used (the default is sha256). However, for security reasons, EMQ also supports password salting. You can uncomment and use one of the salting methods.
The corresponding stored password must be salted as well.
5. EMQ Offline Messages
-
Retained Messages: When an MQTT client publishes (PUBLISH) a message to the server, it can set the Retained Message flag. A Retained Message will reside on the message server, and later subscribers to the topic can still receive that message. For example, when the mosquitto command line publishes a retained message to the topic ‘a/b/c’: mosquitto_pub -r -q 1 -t a/b/c -m ‘hello’, a connecting MQTT client subscribing to ‘a/b/c’ will still receive that message: $ mosquitto_sub -t a/b/c -q 1 hello. There are two ways to clear Retained Messages: a client publishes an empty message to a topic with a retained message: mosquitto_pub -r -q 1 -t a/b/c -m ”; the message server sets a timeout for the retained message.
-
cleanSession: Clearing Sessions: When an MQTT client initiates a CONNECT request to the server, it can set the session through the ‘Clean Session’ flag. Setting ‘Clean Session’ to 0 creates a persistent session, which retains offline messages when the client disconnects until the session times out. Setting ‘Clean Session’ to 1 creates a new temporary session, which is automatically destroyed when the client disconnects.
6. Summary
There are still many details to pay attention to in the use of EMQ and MQTT. Attention to detail is essential for further progress.
Note: The author has limited ability and may have made incorrect statements. Please feel free to point them out, and I hope for more communication!
Official EMQ address:http://emqtt.com/
EMQ Chinese documentation:http://emqtt.com/docs/v2/guide.html
Microsoft Releases Custom Linux Kernel and Distribution
MySQL 8.0 Official Release 8.0.11
Oracle Launches Another Infringement Case, This Time Against JavaScript!
A Markdown Editor That Can Compete in Memes, Interested?
Qt Company Changes Qt Support for Python: Qt for Python is Coming!
