Source: Xiaolin Coding
Author: Xiaolin Coding
From HTTP/1.1 to HTTP/2, the HTTP protocol has always used TCP as its transport protocol. However, in the latest HTTP/3, TCP has been completely abandoned, reaching out to the isolated and helpless UDP, and based on the UDP protocol, a reliable transport protocol has been implemented at the application layer – QUIC.
Many may wonder, TCP has been used for decades with HTTP, and it is such a mature and reliable transport protocol, with features like timeout retransmission, in-order delivery, flow control, and congestion control. Why suddenly abandon TCP? What exactly is wrong with TCP? Is it nitpicking? So today, let’s discuss the reasons why TCP is considered “not good enough”.
TCP Has Head-of-Line Blocking Issues
The head-of-line blocking issue in TCP can be viewed from two perspectives: one is head-of-line blocking in the sending window, and the other is head-of-line blocking in the receiving window.1. Head-of-line blocking in the sending window.Data sent by TCP must be acknowledged in order; only after all data has been acknowledged in order can the sending window slide forward. For example, in the diagram below, the sender has sent all the data within the sending window, but the available window size is 0, indicating that the available window is exhausted, and no further data can be sent until an ACK is received.
Then, when the sender receives the ACK for bytes 32-36, the sliding window moves right by 5 bytes because 5 bytes of data have been acknowledged, and the bytes 52-56 become available, allowing the subsequent sending of these 5 bytes of data.
However, if a data packet is lost or its corresponding ACK packet is lost in the network, the sender cannot move the sending window, and thus cannot send new data. It can only retransmit the lost data packet until the ACK for this retransmitted packet is received, at which point the sending window can move and continue sending.For example, in the diagram below, the client is the sender and the server is the receiver.
The client sends data for bytes 5-9, but the ACK for byte 5 is lost in the network. Therefore, even if the client receives the ACK for bytes 6-9, the sending window will not move forward.At this point, byte 5 is equivalent to the “head of the line”; because the ACK for the “head of the line” has not been received, the sending window cannot move forward, and the sender cannot continue sending subsequent data, effectively pausing the sending action. This is the head-of-line blocking issue in the sending window.2. Head-of-line blocking in the receiving window.The receiver can only accept data within the range of the receiving window. If data outside this range is received, it will be discarded. For example, in the diagram below, the receiving window range is 32-51 bytes, and any data received above byte 52 will be discarded.
When can the receiving window slide? The receiving window can only slide forward when it receives ordered data, and then the already received and acknowledged “ordered” data can be read by the application layer.However, when the data received by the receiving window is not ordered, for example, if bytes 33-40 are received but byte 32 is missing, the receiving window cannot slide forward, and even if bytes 33-40 are received first, these data cannot be read by the application layer. Only when the sender retransmits byte 32 and it is received by the receiver will the receiving window slide forward, allowing the application layer to read bytes 32-40.Now, we have covered the head-of-line blocking issues in both the sending and receiving windows. The reason for these two issues is that TCP must process data in order; that is, to ensure the order of data, the TCP layer can only slide the window forward after processing the ordered data; otherwise, it remains stationary.
- Staying at the “sending window” prevents the sender from continuing to send data.
- Staying at the “receiving window” prevents the application layer from reading new data.
In fact, we cannot blame the TCP protocol; its original design purpose was to ensure the order of data.
Head-of-Line Blocking in HTTP/2
HTTP/2 abstracts the concept of Streams to achieve concurrent transmission of HTTP. A Stream represents a request and response in HTTP/1.1.
In an HTTP/2 connection, frames from different Streams can be sent out of order (thus allowing concurrent transmission of different Streams), because the header of each frame carries Stream ID information, allowing the receiver to assemble HTTP messages in order based on Stream ID. However, frames within the same Stream must be strictly ordered.However, multiple Stream requests in HTTP/2 are transmitted over a single TCP connection, which means that multiple Streams share the same TCP sliding window. Therefore, when data loss occurs, the sliding window cannot move forward, blocking all HTTP requests. This is a form of head-of-line blocking at the TCP layer.
QUIC Without Head-of-Line Blocking
QUIC also borrows the concept of Streams from HTTP/2, allowing multiple HTTP requests (Streams) to be sent concurrently over a single QUIC connection.However, QUIC allocates an independent sliding window for each Stream, which means that there are no dependencies between multiple Streams on a connection; they each control their own sliding windows independently.If a UDP packet from Stream2 is lost, it will only affect the processing of Stream2 and will not impact other Streams. In contrast, in HTTP/2, if a packet in a stream is lost, other streams will also be affected.
TCP Connection Establishment Delay
For HTTP/1 and HTTP/2 protocols, TCP and TLS are layered, belonging to the transport layer implemented in the kernel and the presentation layer implemented in the OpenSSL library, respectively. Therefore, they are difficult to merge, requiring multiple handshakes: first TCP handshake (1 RTT), then TLS handshake (2 RTT), resulting in a total of 3 RTT delays before data can be transmitted. Even if session reuse is employed, at least 2 RTTs are still needed, which increases data transmission delay to a certain extent.The delays of TCP three-way handshake and TLS handshake are shown in the figure:
Although HTTP/3 requires a QUIC handshake before transmitting data, this handshake process only requires 1 RTT. The purpose of the handshake is to confirm the “connection ID” of both parties, and connection migration is based on the connection ID.However, the QUIC protocol in HTTP/3 is not layered with TLS, as QUIC is also an application-layer protocol. Therefore, it can combine the handshake processes of QUIC and TLS. QUIC internally includes TLS, and its frames carry records from TLS. Additionally, since QUIC uses TLS 1.3, it can complete both connection establishment and key negotiation in just 1 RTT. Furthermore, during the second connection, application data packets can be sent along with QUIC handshake information (connection information + TLS information), achieving a 0-RTT effect.As shown in the right part of the figure, when HTTP/3 session resumes, effective payload data can be sent together with the first packet, achieving 0-RTT (as shown in the lower right corner of the figure):
Upgrading TCP is Challenging
The TCP protocol was born in 1973, and it is still being implemented with more new features today.However, the TCP protocol is implemented in the kernel, and applications can only use it but cannot modify it. If we want to upgrade the TCP protocol, we can only upgrade the kernel.Upgrading the kernel is a cumbersome task, not because the operation itself is complicated, but because kernel upgrades involve updates to underlying software and runtime libraries. Our service programs need to be regression tested for compatibility with the new kernel version, so server kernel upgrades are also relatively conservative and slow.Many new features of the TCP protocol require both the client and server to support them simultaneously to take effect. For example, the TCP Fast Open feature was proposed in 2013, but many versions of Windows still do not support it. This is because the upgrade of PC systems is significantly lagging, with many users still using Windows XP, which has been around for nearly 20 years.Therefore, even if TCP has good feature updates, it is difficult to promote them quickly, and users often have to wait years or even decades to experience them.In contrast, QUIC operates at the application layer, so upgrading the QUIC protocol is as easy as upgrading software. Moreover, QUIC can set different congestion control algorithms for different applications, providing high flexibility, which TCP cannot achieve because changing the congestion control algorithm in TCP affects all applications in the system and cannot be set differently for different applications.
Network Migration Requires Re-establishing TCP Connections
HTTP protocols based on TCP transport are determined by a four-tuple (source IP, source port, destination IP, destination port).
Therefore, when a mobile device switches from 4G to Wi-Fi, it means the IP address has changed, necessitating a disconnection and re-establishment of the TCP connection.
The connection establishment process includes the delays of TCP three-way handshake and TLS four-way handshake, as well as the slowdown process of TCP slow start, giving users the impression that the network suddenly lags for a moment. Therefore, the cost of connection migration is very high.The QUIC protocol does not use the four-tuple method to “bind” connections but instead uses connection IDs to mark the two endpoints of communication. The client and server can each choose a set of IDs to mark themselves. Therefore, even if the mobile device’s network changes, resulting in an IP address change, as long as the context information (such as connection ID, TLS keys, etc.) is retained, the original connection can be “seamlessly” reused, eliminating the reconnection cost and avoiding any lag, achieving the function of connection migration.
Conclusion
After HTTP/3 abandoned TCP, the reliable transport protocol based on UDP, QUIC, brings four benefits:
- Reduced connection time: Achieving 0-RTT connection establishment when the client has cached data.
- More flexible congestion control: Configuring different congestion control strategies for each request in user space.
- Multiplexing without head-of-line blocking: Each request stream independently has its own sliding window, unaffected by others.
- Connection migration: Network switching does not interrupt data transmission.
However, HTTP/3 also faces some challenges. The QUIC protocol, implemented in user space based on UDP, may not be recognized by some network devices. In the eyes of these network devices, it is just a UDP protocol.Almost all telecom operators “discriminate” against UDP packets, which is easy to understand, as several notorious DDoS attacks in history were based on UDP. In some areas, certain broadband providers directly prohibit non-port 53 UDP packets, while others, even without blocking UDP, strictly limit its flow.Since QUIC was officially released in 2013, it has developed for nearly 10 years by 2023. There are already many popular open-source projects online. Apart from Google, which has completed support for its own search engine and also brought Gmail, YouTube, and other sites on board, most domestic sites still use HTTP/2 protocol. The road to HTTP/3 seems to be stuck in the Tang Dynasty.Including my illustration website, which uses HTTP/2 protocol, I heard that Nginx has publicly stated: “We have supported QUIC protocol”. I should find time to upgrade Nginx and try to upgrade my website to HTTP/3, making the originally smooth website even smoother.
.
-End-
