Introduction
Hello everyone, I am MAI Mai Zao
The reason is that a high-concurrency collection service went live, and the 100m upstream bandwidth was quickly saturated. Since this is a dedicated line and only this service is using it, it can be confirmed that it is the cause.
However! This request is just a GET request, and it does not have a large request body. Why is that?
So, after using <span>charles</span> to capture the packets again, it was found that a <span>request</span> actually occupies 1.68kb in size! Among them, the TLS Handshake occupies 1.27kb.
What is TLS Handshake, and why is it so large?
First, it is important to know that HTTPS stands for:HTTP over TLS, and each time a new TCP connection is established, a complete TLS Handshake is usually required. During the handshake process, the client and server need to exchange certificates, public keys, encryption algorithms, and other information, which consumes a significant number of bytes.
TLS Handshake mainly includes:
- • Random numbers from the client and server
- • Supported encryption algorithms and TLS version information
- • The server’s digital certificate (including the public key)
- • The “Pre-Master Secret” used to generate the symmetric key
This process is not only time-consuming but also consumes bandwidth and CPU resources.
Therefore, the most straightforward solution is quite simple: just use HTTP, eliminating the TLS Handshake process, and naturally, there will be no TLS transmission.
So, is it really effective? Let’s verify it.
After changing the request protocol to HTTP:
It can be seen that the request header indeed does not contain TLS Handshake!
The entire request is only 0.4kb, saving 70% in size. Goal achieved.
Therefore, it can be concluded that in scenarios where HTTPS is not strictly necessary, using HTTP can save bandwidth. Additionally, due to the reduction of the encryption process, it can be observed that under the same concurrency, the server load is significantly reduced.
Now the question arises
If the interface must use HTTPS, what should be done? Of course, there is another solution, which is to use Keep-Alive.
<span>headers</span> add <span>Connection: keep-alive</span> to use it.
By enabling <span>Keep-Alive</span>, multiple HTTPS requests can be sent over the same TCP connection without having to perform a complete <span>TLS Handshake</span> each time, but the certificate and key exchange still need to be transmitted during the first handshake. This is also very suitable for high-concurrency scenarios.
It should be noted that
Keep-Alive has a timeout, and if the time exceeds, the connection will be closed, requiring a new connection to be established for subsequent requests.Nginx has a default <span>keep-alive</span> timeout of 75 seconds, while Apache HTTP Server typically has a default <span>keep-alive</span> timeout of 5 seconds.
PS: If your collection program uses a large number of proxy IPs, then the effect of Keep-Alive may not be significant~~ The best case is still to use HTTP.