1. Introduction
As described in “Message Bus Guarantees Message Delivery”, the core architectural design of MQ message delivery has two main points:
(1) Message Persistence
(2) Message Timeout, Retransmission, Acknowledgment
Upon reviewing the core architecture of the message bus, it consists of four main components: Sender, Server, Persistent Storage, Receiver.
To ensure message reachability, the timeout, retransmission, and acknowledgment mechanisms may lead to the message bus or the business receiving duplicate messages, which can impact the business.
For example:
When purchasing a membership card, the upstream payment system is responsible for charging the user, and the downstream system is responsible for issuing the card, using MQ for asynchronous notification. Whether it’s the loss of ACK in the first half, causing MQ to receive duplicate messages, or the loss of ACK in the second half, causing the card purchasing system to receive duplicate purchase notifications, it can lead to situations where the upstream charges once and the downstream issues multiple cards.
The design of idempotency in the message bus is crucial and is the focus of this article.
2. Idempotency Design in the First Half
MQ message sending in the first half, as shown in the above diagram from 1-3
1, the sender MQ-client sends the message to the server MQ-server
2, the server MQ-server persists the message
3, the server MQ-server sends an ACK back to the sender MQ-client
If 3 is lost, the sender MQ-client will retransmit the message after timing out, which may cause the server MQ-server to receive duplicate messages.
At this time, the retransmission is initiated by MQ-client, while the processing of the message is handled by MQ-server. To avoid the issue of step 2 persisting duplicate messages, the MQ system must generate an inner-msg-id for each message, which serves as the basis for deduplication and idempotency. The characteristics of this internal message ID are:
(1) Globally unique
(2) Generated by MQ, business-agnostic, shielding both the message sender and receiver
With this inner-msg-id, it ensures that even if retransmissions occur in the first half, only 1 message is persisted in the MQ-server‘s DB, achieving idempotency in the first half.
3. Idempotency Design in the Second Half
MQ message sending in the second half, as shown in the above diagram from 4-6
4, the server MQ-server sends the message to the receiver MQ-client
5, the receiver MQ-client sends an ACK back to the server
6, the server MQ-server deletes the persisted message
It is important to emphasize that the receiver MQ-client sending an ACK back to the server MQ-server is an active call from the message consumer, and cannot be automatically initiated by MQ-client, because MQ does not know when the consumer has successfully consumed the message.
If 5 is lost, the server MQ-server will retransmit the message after timing out, which may lead to MQ-client receiving duplicate messages.
At this time, the retransmission is initiated by MQ-server, while the message processing is handled by the business consumer. The retransmission of messages will inevitably lead to the consumer processing the same message multiple times (such as charging once and issuing multiple cards), to ensure business idempotency, the business message body must contain a biz-id, which serves as the basis for deduplication and idempotency. The characteristics of this business ID are:
(1) Globally unique for the same business scenario
(2) Generated by the business message sender, business-related, transparent to MQ
(3) Responsible for deduplication by the business message consumer to ensure idempotency
The most common business ID includes: paymentID, orderID, postID, etc.
Specifically for the payment card purchase scenario, the sender must include the payment ID in the message body, and the consumer must deduplicate based on the same payment ID to ensure idempotency in card purchasing.
With this business ID, it ensures that the message consumer in the second half, even if receiving duplicate messages, only processes 1 message, ensuring idempotency.
4. Conclusion
To ensure message delivery, MQ may send duplicate messages in both the first and second halves. How to ensure message idempotency?
In the First Half
MQ-client generates inner-msg-id, ensuring idempotency in the first half. This ID is globally unique and business-agnostic, guaranteed by MQ.
In the Second Half
The business sender includes biz-id, and the business receiver deduplicates to ensure idempotency. This ID is unique to a single business and business-related, transparent to MQ.
The conclusion: Idempotency is not only a requirement for MQ but also for upstream and downstream business processes.
==【End】==
Related Reading:
How to Implement “Delayed Messages”
Can the Message Bus Ensure Message Delivery