Learning MQTT Protocol: 006 - Subscribe Topic and Corresponding Messages (SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK)

Background

Previously, we discussed how to publish messages and their corresponding packets; now let’s look at how the packets for subscribing to a topic are structured.

SUBSCRIBE – Subscribe to a Topic

The client sends a SUBSCRIBE packet to the server to create one or more subscriptions. Each subscription registers one or more topics of interest to the client. To forward application messages to the topics matching those subscriptions, the server sends PUBLISH packets to the client. The SUBSCRIBE packet also specifies the maximum QoS level for each subscription, and the server sends application messages to the client based on this level.

Fixed Header of SUBSCRIBE

Bit	7	6	5	4	3	2	1	0
byte 1	MQTT Control Packet Type (0x8)	Reserved Bit (0x2)
1	0	0	0	0	0	1	0
byte 2	Remaining Length

The 3rd, 2nd, 1st, and 0th bits of the fixed header of the SUBSCRIBE control packet must be set to 0, 0, 1, and 0, respectively. The remaining length field equals the length of the variable header (2 bytes) plus the length of the payload.

Variable Header of SUBSCRIBE

The variable header of SUBSCRIBE contains only the Packet Identifier field.

Packet Identifier occupies 2 bytes. There is no new knowledge point here, so it will not be introduced further.

Payload of SUBSCRIBE

The payload of the SUBSCRIBE packet must contain at least one pair of Topic Filter and QoS Level fields combined.

Description	7	6	5	4	3	2	1	0
Topic Filter
byte 1	Length MSB
byte 2	Length LSB
byte 3..N	Topic Filter
Requested QoS
Reserved Bit	QoS Level
byte N+1	0	0	0	0	0	0	X	X

The current version of the protocol does not utilize the high six bits of the Requested QoS byte. If any of these bits are non-zero, or if QoS is not equal to 0, 1, or 2, the server must consider the SUBSCRIBE packet invalid and close the network connection.

Each filter is followed by a byte, known as the Requested QoS, which indicates the maximum QoS level allowed for the server to send application messages to the client.

Topic Filter List: Represents the topics the client wants to subscribe to, must be a UTF-8 string. The server should support topic filters that include wildcards. If the server chooses not to support topic filters with wildcards, it must reject any subscription requests that include wildcard filters.

The body of the SUBSCRIBE contains the list of topics the client wants to subscribe to, where each item consists of the subscription topic name and corresponding QoS. The topic name can include wildcards, single-level wildcard ‘+’ and multi-level wildcard ‘#’. Using topic names that include wildcards can subscribe to all topics that meet the matching conditions. (The same goes for the following). To distinguish from the topics in PUBLISH, we refer to the topic names in SUBSCRIBE as Topic Filters. There is an introduction to Topic Filters at the end of the article.

Maximum Requested QoS Level: The field is encoded as one byte,

Topic Filter and QoS Level combinations are packed consecutively.

Example of SUBSCRIBE Packet Content

# Command used: mosquitto_sub -v  -u admin -P root -t 'topic' -q 2  -t 'a\b'

MQ Telemetry Transport Protocol, Subscribe Request
    Header Flags: 0x82, Message Type: Subscribe Request
        1000 .... = Message Type: Subscribe Request (8)
        .... 0010 = Reserved: 2
    Msg Len: 20
    Message Identifier: 1
    Topic Length: 7
    Topic: 'topic'
    Requested QoS: Exactly once delivery (Assured Delivery) (2)
    Topic Length: 5
    Topic: 'a\b'
    Requested QoS: Exactly once delivery (Assured Delivery) (2)

0040   82 14 00 01 00 07 27 74 6f 70 69 63 27 02 00 05   ......'topic'...
0050   27 61 5c 62 27 02                                 'a\b'.

SUBSCRIBE Response

When the server receives a SUBSCRIBE packet from the client, it must respond with a SUBACK packet. The SUBACK packet must have the same packet identifier as the SUBSCRIBE packet awaiting confirmation.

The server is allowed to start sending PUBLISH packets that match the subscription before sending the SUBACK packet.

For QoS levels and processes, please refer to: “QoS Levels and Sessions”. What is the Packet Identifier in the PUBLISH packet, and what is the Packet Identifier below.

If the server receives a SUBSCRIBE packet with a topic filter that matches an existing subscription’s topic filter, it must completely replace the existing subscription with the new subscription. The new subscription’s topic filter is the same as the previous subscription’s, but its maximum QoS value may differ. Any existing retained messages matching this topic filter must be resent, but the publishing process must not be interrupted.

If the topic filter differs from any existing subscription’s filter, the server will create a new subscription and send all matching retained messages.

If the server receives a SUBSCRIBE packet with multiple topic filters, it must handle it as if it received a series of multiple SUBSCRIBE packets, except that it needs to combine their responses into a single SUBACK packet [MQTT-3.8.4-4].

The SUBACK packet sent by the server to the client must include a return code for each pair of topic filters and QoS levels. This return code must indicate the maximum QoS level granted for that subscription, or indicate that the subscription failed [MQTT-3.8.4-5]. The server may grant a lower QoS level than what the subscriber requested. The QoS of the payload of messages sent in response to the subscription must be the minimum of the original published message’s QoS and the QoS granted by the server. If the original message’s QoS is 1 and the granted maximum QoS is 0, the server is allowed to resend a copy of the message to the subscriber [MQTT-3.8.4-6].

Non-standard Example For a specific topic filter, if the maximum QoS level granted to the subscribing client is 1, then application messages with QoS level 0 that match this filter will be distributed to this client at QoS level 0. This means the client will receive at most one copy of this message. On the other hand, messages published to the same topic with QoS level 2 will be downgraded by the server to QoS level 1 before being distributed to the client, so the client may receive duplicate copies of the message.

If the maximum QoS level granted to the subscribing client is 0, then application messages originally published to the client with QoS level 2 may be lost during busy times, but the server should not send duplicate copies of the message. Messages published to the same topic with QoS level 1 may be lost or duplicated when transmitted to the client.

Subscribing to a topic filter with QoS level 2 means: I want to receive messages matching this filter according to their published QoS level. This means that determining the maximum possible QoS level during message distribution is the responsibility of the publisher, while the subscriber can request the server to lower the QoS to a more suitable level for them.

SUBACK – Subscription Acknowledgment Packet

The server sends a SUBACK packet to the client to confirm that it has received and is processing the SUBSCRIBE packet.

The SUBACK packet contains a list of return codes that specify the maximum QoS level granted for each subscription in the SUBSCRIBE request.

Fixed Header of SUBACK

Bit	7	6	5	4	3	2	1	0
byte 1	MQTT Control Packet Type (0x9)	Reserved Bit (0x0)
1	0	0	1	0	0	0	0
byte 2	Remaining Length

Remaining Length Field: Equals the length of the variable header plus the length of the payload.

Variable Header of SUBACK

The variable header contains the packet identifier of the SUBSCRIBE packet awaiting confirmation, occupying 2 bytes.

Payload of SUBACK

The payload contains a list of return codes (n return codes). Each return code corresponds to a topic filter in the SUBSCRIBE packet awaiting confirmation.

Each return code occupies 1 byte, allowed return code values:

0x00 – Maximum QoS 0
0x01 – Success – Maximum QoS 1
0x02 – Success – Maximum QoS 2
0x80 – Failure

Return codes other than 0x00, 0x01, 0x02, and 0x80 are reserved and must not be used.

The order of return codes must match the order of topic filters in the SUBSCRIBE packet.

Example of SUBACK Packet

MQ Telemetry Transport Protocol, Subscribe Ack
    Header Flags: 0x90, Message Type: Subscribe Ack
        1001 .... = Message Type: Subscribe Ack (9)
        .... 0000 = Reserved: 0
    Msg Len: 4
    Message Identifier: 1
    Granted QoS: Exactly once delivery (Assured Delivery) (2)
    Granted QoS: Exactly once delivery (Assured Delivery) (2)

0040   90 04 00 01 02 02                                 ......

UNSUBSCRIBE – Unsubscribe Packet

The client sends an UNSUBSCRIBE packet to the server to unsubscribe from a topic.

Without introducing the format of the UNSUBSCRIBE packet, can readers guess what the format of the UNSUBSCRIBE packet is like?

If you are familiar with the SUBSCRIBE packet, then clever readers might know right away. In fact, in the UNSUBSCRIBE packet, except that the payload does not contain QoS levels, everything else is very similar to UNSUBSCRIBE. So what about the format of the UNSUBACK packet?

Fixed Header of UNSUBSCRIBE

Bit	7	6	5	4	3	2	1	0
byte 1	MQTT Control Packet Type (0xa)	Reserved Bit (0x2)
1	0	0	0	0	0	1	0
byte 2	Remaining Length

The fixed header of the UNSUBSCRIBE control packet’s 3rd, 2nd, 1st, and 0th bits must be set to 0, 0, 1, and 0, respectively. The remaining length field equals the length of the variable header (2 bytes) plus the length of the payload.

Variable Header of UNSUBSCRIBE

The variable header of UNSUBSCRIBE contains only the Packet Identifier field.

Packet Identifier occupies 2 bytes. There is no new knowledge point here, so it will not be introduced further.

Payload of UNSUBSCRIBE

The payload of the UNSUBSCRIBE packet contains the Topic Filter List that the client wants to unsubscribe from. The topic filters in the UNSUBSCRIBE packet must be packed consecutively. The payload of the SUBSCRIBE packet must contain at least 1 Topic Filter.

Topic Filter List: Represents the topics the client wants to unsubscribe from, must be a UTF-8 string. The server should support topic filters that include wildcards. If the server chooses not to support topic filters with wildcards, it must reject any subscription requests that include wildcard filters.

Response of UNSUBSCRIBE

The topic filters provided by the UNSUBSCRIBE packet (whether or not they include wildcards) must be compared character by character with the current set of topic filters held by the server for this client. If any filter matches completely, then it (the server) will delete its own subscription; otherwise, no further processing will occur.

If the server deletes a subscription:

It must stop distributing any new messages to this client.
It must complete the distribution of any QoS 1 and QoS 2 messages that have already started sending to the client.
It may continue sending any existing cached messages prepared for distribution to the client.

The server must send an UNSUBACK packet in response to the client’s UNSUBSCRIBE request. The UNSUBACK packet must contain the same packet identifier as the UNSUBSCRIBE packet.

Even if no topic subscriptions are deleted, the server must send an UNSUBACK response.

If the server receives an UNSUBSCRIBE packet with multiple topic filters, it must handle it as if it received a series of multiple UNSUBSCRIBE packets, except that it will combine their responses into a single UNSUBACK packet.

In other words, it will send many UNSUBACK packets back.

Example of UNSUBSCRIBE Packet

# mosquitto_sub -v  -u admin -P root -t 'topic' -q 2  -t 'a\b'  -U 'topic'
## -U represents unsubscribing from a topic.

MQ Telemetry Transport Protocol, Unsubscribe Request
    Header Flags: 0xa2, Message Type: Unsubscribe Request
        1010 .... = Message Type: Unsubscribe Request (10)
        .... 0010 = Reserved: 2
    Msg Len: 11
    Message Identifier: 2
    Topic Length: 7
    Topic: 'topic'

0040   a2 0b 00 02 00 07 27 74 6f 70 69 63 27            ......'topic'

UNSUBACK – Unsubscribe Acknowledgment Packet

The server sends an UNSUBACK packet to the client to confirm receipt of the UNSUBSCRIBE packet.

The UNSUBACK packet is a response to the UNSUBSCRIBE packet.

The UNSUBACK packet consists of (no payload) = a fixed header (0xb 0x02) + Packet Identifier (from the UNSUBSCRIBE’s Packet Identifier).

Example of UNSUBACK Packet Capture

#mosquitto_sub -v  -u admin -P root -t 'topic' -q 2  -t 'a\b'  -U 'topic' -U 'a\b'
## I do not know whether the internal processing mechanism of mosquitto_sub sends individual Unsubscribe packets, but the result shows it is like this.
687	17.900459	::1	::1	MQTT	168	Subscribe Request (id=1) ['topic'] ['a\b']
689	17.900477	::1	::1	MQTT	150	Unsubscribe Request (id=2)
691	17.900492	::1	::1	MQTT	146	Unsubscribe Request (id=3)
693	17.900538	::1	::1	MQTT	136	Subscribe Ack (id=1)
695	17.900576	::1	::1	MQTT	132	Unsubscribe Ack (id=2)
697	17.900605	::1	::1	MQTT	132	Unsubscribe Ack (id=3)

Appendix: Topic Wildcards and Topic Filters

When we subscribe to topics, we can use wildcards to match multiple subscribed topics. MQTT topics have a hierarchical concept, separated by ‘/’ between different levels.

Single-level Wildcard ‘+’: Used to represent any one level.

For example, ‘home/2ndfloor/+/temperature’ can match:

home/2ndfloor/201/temperature
home/2ndfloor/202/temperature

Cannot match:

home/2ndfloor/201/livingroom/temperature
home/3ndfloor/301/temperature

Multi-level Wildcard ‘#’: Can specify any number of levels

The difference between ‘#’ and ‘+’ is: 1) ‘+’ is used to represent any one level; while ‘#’ can specify any number of levels. 2) However, ‘#’ must be the last character of the Topic Filter, and it must follow ‘/’ unless the Topic Filter only contains ‘#’ as a single character.

For example, ‘home/2ndfloor/#’ can match:

home/2ndfloor
home/2ndfloor/201
home/2ndfloor/201/temperature
home/2ndfloor/202/temperature
home/2ndfloor/201/livingroom/temperature