Background
The HTTP protocol is a stateless protocol, meaning each request is independent of others. Therefore, its initial implementation was to open a TCP socket connection for each HTTP request, which would be closed after the interaction was complete.
HTTP is a full-duplex protocol, so establishing and closing connections requires three-way handshakes and four-way handshakes. Clearly, in this design, sending each HTTP request consumes a lot of additional resources, namely the establishment and destruction of connections.
Thus, the HTTP protocol has evolved to allow socket connection reuse through persistent connections.
From the image, we can see:
-
In serial connections, each interaction requires opening and closing connections.
-
In persistent connections, the first interaction opens a connection, and after the interaction ends, the connection is not closed, saving the connection establishment process for the next interaction.
There are two implementations of persistent connections: HTTP/1.0+ keep-alive and HTTP/1.1 persistent connections.
HTTP/1.0+ Keep-Alive
Since 1996, many HTTP/1.0 browsers and servers have extended the protocol with the “keep-alive” extension.
Note that this extension was introduced as an “experimental persistent connection” to supplement 1.0. Keep-alive is no longer in use, and the latest HTTP/1.1 specifications do not mention it, although many applications continue to use it.
Clients using HTTP/1.0 add “Connection:Keep-Alive” to the header, requesting the server to keep a connection open. If the server agrees to keep the connection open, it will include the same header in the response. If the response does not include the “Connection:Keep-Alive” header, the client will assume the server does not support keep-alive and will close the current connection after sending the response message.
Through the keep-alive extension, persistent connections were established between clients and servers, but some issues still existed:
-
In HTTP/1.0, keep-alive is not a standard protocol; clients must send Connection:Keep-Alive to activate the keep-alive connection.
-
Proxy servers may not support keep-alive, as some proxies are “blind relays” that cannot understand the meaning of the header and merely forward it hop by hop. This can lead to situations where both the client and server maintain connections, but the proxy does not accept data on that connection.
HTTP/1.1 Persistent Connections
HTTP/1.1 adopted persistent connections to replace Keep-Alive.
By default, connections in HTTP/1.1 are persistent. To explicitly close them, the header Connection:Close must be added to the message. In HTTP/1.1, all connections are reused.
However, like Keep-Alive, idle persistent connections can also be closed at any time by the client and server. Not sending Connection:Close does not imply that the server commits to keeping the connection open forever.
How HttpClient Generates Persistent Connections
HttpClient uses a connection pool to manage held connections, allowing reuse on the same TCP link. HttpClient achieves connection persistence through a connection pool.
In fact, the “pool” technique is a general design, and its design philosophy is not complex:
-
When a connection is used for the first time, establish the connection.
-
At the end, do not close the corresponding connection; return it to the pool.
-
The next time a connection to the same destination can be obtained from the pool.
-
Regularly clean up expired connections.
All connection pools follow this idea, but we focus on two aspects of the HttpClient source code:
-
The specific design scheme of the connection pool for future custom connection pool reference.
-
How to correspond to the HTTP protocol, i.e., how theoretical abstractions translate into code implementations.
Implementation of HttpClient Connection Pool
The handling of persistent connections in HttpClient can be concentrated in the following code, extracted from MainClientExec, with other parts removed:
public class MainClientExec implements ClientExecChain {
@Override
public CloseableHttpResponse execute(
final HttpRoute route,
final HttpRequestWrapper request,
final HttpClientContext context,
final HttpExecutionAware execAware) throws IOException, HttpException {
// Get a connection request from the connection manager HttpClientConnectionManager
final ConnectionRequest connRequest = connManager.requestConnection(route, userToken);final HttpClientConnection managedConn;
final int timeout = config.getConnectionRequestTimeout();
// Get a managed connection HttpClientConnection from the connection request ConnectionRequest
managedConn = connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS);
// Hold the connection manager HttpClientConnectionManager and the managed connection HttpClientConnection with a ConnectionHolder
final ConnectionHolder connHolder = new ConnectionHolder(this.log, this.connManager, managedConn);
try {
HttpResponse response;
if (!managedConn.isOpen()) {
// If the current managed connection is not open, it needs to be re-established
establishRoute(proxyAuthState, managedConn, route, request, context);
}
// Send the request through the connection HttpClientConnection
response = requestExecutor.execute(request, managedConn, context);
// Determine if the connection can be reused through the connection reuse strategy
if (reuseStrategy.keepAlive(response, context)) {
// Get the connection's validity period
final long duration = keepAliveStrategy.getKeepAliveDuration(response, context);
// Set the connection's validity period
connHolder.setValidFor(duration, TimeUnit.MILLISECONDS);
// Mark the current connection as reusable
connHolder.markReusable();
} else {
connHolder.markNonReusable();
}
}
final HttpEntity entity = response.getEntity();
if (entity == null || !entity.isStreaming()) {
// Release the current connection back to the pool for the next call
connHolder.releaseConnection();
return new HttpResponseProxy(response, null);
} else {
return new HttpResponseProxy(response, connHolder);
}
}
Here we see that the handling of connections during the HTTP request process is consistent with the protocol specifications, and we need to elaborate on the specific implementation.
PoolingHttpClientConnectionManager is the default connection manager for HttpClient. First, it obtains a connection request through requestConnection():
public ConnectionRequest requestConnection(
final HttpRoute route,
final Object state) {final Future<CPoolEntry> future = this.pool.lease(route, state, null);
return new ConnectionRequest() {
@Override
public boolean cancel() {
return future.cancel(true);
}
@Override
public HttpClientConnection get(
final long timeout,
final TimeUnit tunit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException {
final HttpClientConnection conn = leaseConnection(future, timeout, tunit);
if (conn.isOpen()) {
final HttpHost host;
if (route.getProxyHost() != null) {
host = route.getProxyHost();
} else {
host = route.getTargetHost();
}
final SocketConfig socketConfig = resolveSocketConfig(host);
conn.setSocketTimeout(socketConfig.getSoTimeout());
}
return conn;
}
};
}
It can be seen that the returned ConnectionRequest object actually holds a Future, where CPoolEntry is the actual connection instance managed by the connection pool.
From the above code, we should focus on:
-
Futurefuture = this.pool.lease(route, state, null)how to obtain an asynchronous connection from the connection pool CPool. -
HttpClientConnection conn = leaseConnection(future, timeout, tunit)how to obtain a real connection HttpClientConnection through the asynchronous connection Future.
Future
Let’s see how CPool releases a Future, with the core code of AbstractConnPool as follows:
private E getPoolEntryBlocking(
final T route, final Object state,
final long timeout, final TimeUnit tunit,
final Future<E> future) throws IOException, InterruptedException, TimeoutException {
// First, lock the current connection pool; this lock is reentrant this.lock.lock();
try {
// Obtain the connection pool corresponding to the current HttpRoute; for HttpClient's connection pool, there is a total pool size, and each route corresponds to a connection pool, so it is a "pool within a pool"
final RouteSpecificPool<T, C, E> pool = getPool(route);
E entry;
for (;;) {
Asserts.check(!this.isShutDown, "Connection pool shut down");
// Loop to obtain connections
for (;;) {
// Get a connection from the route-specific pool; it may be null or a valid connection
entry = pool.getFree(state);
// If null is obtained, exit the loop
if (entry == null) {
break;
}
// If an expired or closed connection is obtained, release the resources and continue the loop to obtain
if (entry.isExpired(System.currentTimeMillis())) {
entry.close();
}
if (entry.isClosed()) {
this.available.remove(entry);
pool.free(entry, false);
} else {
// If a valid connection is obtained, exit the loop
break;
}
}
// If a valid connection is obtained, exit
if (entry != null) {
this.available.remove(entry);
this.leased.add(entry);
onReuse(entry);
return entry;
}
// If no valid connection is obtained, a new one needs to be generated
final int maxPerRoute = getMax(route);
// The maximum number of connections for each route is configurable; if exceeded, some connections need to be cleaned up using LRU
final int excess = Math.max(0, pool.getAllocatedCount() + 1 - maxPerRoute);
if (excess > 0) {
for (int i = 0; i < excess; i++) {
final E lastUsed = pool.getLastUsed();
if (lastUsed == null) {
break;
}
lastUsed.close();
this.available.remove(lastUsed);
pool.remove(lastUsed);
}
}
// The number of connections in the current route pool has not reached the upper limit
if (pool.getAllocatedCount() < maxPerRoute) {
final int totalUsed = this.leased.size();
final int freeCapacity = Math.max(this.maxTotal - totalUsed, 0);
// Determine if the connection pool is over the upper limit; if so, some connections need to be cleaned up using LRU
if (freeCapacity > 0) {
final int totalAvailable = this.available.size();
// If the number of idle connections is greater than the remaining available space, idle connections need to be cleaned up
if (totalAvailable > freeCapacity - 1) {
if (!this.available.isEmpty()) {
final E lastUsed = this.available.removeLast();
lastUsed.close();
final RouteSpecificPool<T, C, E> otherpool = getPool(lastUsed.getRoute());
otherpool.remove(lastUsed);
}
}
// Create a connection based on the route
final C conn = this.connFactory.create(route);
// Add this connection to the corresponding "small pool" of the route
entry = pool.add(conn);
// Add this connection to the "large pool"
this.leased.add(entry);
return entry;
}
}
// If no valid connection is obtained and a new connection needs to be established, the current route connection pool has reached its maximum number, meaning connections are in use but unavailable to the current thread
boolean success = false;
try {
if (future.isCancelled()) {
throw new InterruptedException("Operation interrupted");
}
// Put the future into the route pool to wait
pool.queue(future);
// Put the future into the large connection pool to wait
this.pending.add(future);
// If a signal is received, success will be true
if (deadline != null) {
success = this.condition.awaitUntil(deadline);
} else {
this.condition.await();
success = true;
}
if (future.isCancelled()) {
throw new InterruptedException("Operation interrupted");
}
} finally {
// Remove from the waiting queue
pool.unqueue(future);
this.pending.remove(future);
}
// If no signal was received and the current time has timed out, exit the loop
if (!success && (deadline != null && deadline.getTime() <= System.currentTimeMillis())) {
break;
}
}
// If no valid connection was obtained, throw an exception
throw new TimeoutException("Timeout waiting for connection");
} finally {
// Release the lock on the large connection pool
this.lock.unlock();
}
}
The logic in the code above has several important points:
-
The connection pool has a maximum number of connections, and each route corresponds to a small connection pool, which also has a maximum number of connections. -
Regardless of whether it is the large connection pool or the small connection pool, when the number exceeds, some connections need to be released using LRU. -
If a valid connection is obtained, it is returned for upper-level use. -
If no valid connection is obtained, HttpClient will determine whether the current route connection pool has exceeded the maximum number. If it has not reached the upper limit, a new connection will be created and added to the pool. -
If the upper limit has been reached, it will wait in line until a signal is received; if the signal is not received, a timeout exception will be thrown. -
Obtaining connections through the thread pool must be locked using ReetrantLock to ensure thread safety.
By this point, the program has either obtained a usable CPoolEntry instance or terminated with an exception.
HttpClientConnection
protected HttpClientConnection leaseConnection(
final Future<CPoolEntry> future,
final long timeout,
final TimeUnit tunit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException {
final CPoolEntry entry;
try {
// Obtain CPoolEntry from the asynchronous operation Future<CPoolEntry>
entry = future.get(timeout, tunit);
if (entry == null || future.isCancelled()) {
throw new InterruptedException();
}
Asserts.check(entry.getConnection() != null, "Pool entry with no connection");
if (this.log.isDebugEnabled()) {
this.log.debug("Connection leased: " + format(entry) + formatStats(entry.getRoute()));
}
// Obtain a proxy object for CPoolEntry, all operations use the same underlying HttpClientConnection
return CPoolProxy.newProxy(entry);
} catch (final TimeoutException ex) {
throw new ConnectionPoolTimeoutException("Timeout waiting for connection from pool");
}
}
How does HttpClient reuse persistent connections?
In the previous chapter, we saw that HttpClient obtains connections through the connection pool, acquiring them from the pool when needed.
This corresponds to the issues discussed in Chapter 3:
-
When a connection is used for the first time, establish the connection.
-
At the end, do not close the corresponding connection; return it to the pool.
-
The next time a connection to the same destination can be obtained from the pool.
-
Regularly clean up expired connections.
We saw in Chapter 4 how HttpClient handles points 1 and 3. Now, how does it handle the second point?
That is, how does HttpClient determine whether a connection should be closed after use or placed back in the pool for reuse? Let’s take a look at the code in MainClientExec:
// Send the HTTP connection
response = requestExecutor.execute(request, managedConn, context);
// Determine whether the current connection should be reused based on the reuse strategy
if (reuseStrategy.keepAlive(response, context)) {
// If the connection needs to be reused, get the connection timeout, based on the timeout in the response
final long duration = keepAliveStrategy.getKeepAliveDuration(response, context);
if (this.log.isDebugEnabled()) {
final String s;
// The timeout is in milliseconds; if not set, it is -1, meaning no timeout
if (duration > 0) {
s = "for " + duration + " " + TimeUnit.MILLISECONDS;
} else {
s = "indefinitely";
}
this.log.debug("Connection can be kept alive " + s);
}
// Set the timeout; when the request ends, the connection manager will decide whether to close or return it to the pool based on the timeout
connHolder.setValidFor(duration, TimeUnit.MILLISECONDS);
// Mark the connection as reusable
connHolder.markReusable();
} else {
// Mark the connection as non-reusable
connHolder.markNonReusable();
}
It can be seen that after a connection is used for a request, a connection reuse strategy determines whether the connection should be reused. If it should be reused, it is given to the HttpClientConnectionManager to place in the pool.
So what is the logic of the connection reuse strategy?
public class DefaultClientConnectionReuseStrategy extends DefaultConnectionReuseStrategy {
public static final DefaultClientConnectionReuseStrategy INSTANCE = new DefaultClientConnectionReuseStrategy();
@Override
public boolean keepAlive(final HttpResponse response, final HttpContext context) {
// Get the request from the context
final HttpRequest request = (HttpRequest) context.getAttribute(HttpCoreContext.HTTP_REQUEST);
if (request != null) {
// Get the Connection header
final Header[] connHeaders = request.getHeaders(HttpHeaders.CONNECTION);
if (connHeaders.length != 0) {
final TokenIterator ti = new BasicTokenIterator(new BasicHeaderIterator(connHeaders, null));
while (ti.hasNext()) {
final String token = ti.nextToken();
// If it contains the Connection:Close header, it indicates that the request does not intend to keep the connection, ignoring the response's intention; this header is part of the HTTP/1.1 specification
if (HTTP.CONN_CLOSE.equalsIgnoreCase(token)) {
return false;
}
}
}
}
// Use the parent's reuse strategy
return super.keepAlive(response, context);
}
}
Let’s take a look at the parent class’s reuse strategy:
if (canResponseHaveBody(request, response)) {
final Header[] clhs = response.getHeaders(HTTP.CONTENT_LEN);
// If the response's Content-Length is not set correctly, do not reuse the connection
// Because for persistent connections, no need to re-establish connections between two transmissions, it needs to be handled correctly to avoid "sticky packet" issues
// So, if the response's Content-Length is not set correctly, the connection cannot be reused
if (clhs.length == 1) {
final Header clh = clhs[0];
try {
final int contentLen = Integer.parseInt(clh.getValue());
if (contentLen < 0) {
return false;
}
} catch (final NumberFormatException ex) {
return false;
}
} else {
return false;
}
}
if (headerIterator.hasNext()) {
try {
final TokenIterator ti = new BasicTokenIterator(headerIterator);
boolean keepalive = false;
while (ti.hasNext()) {
final String token = ti.nextToken();
// If the response has the Connection:Close header, it clearly indicates the intention to close, do not reuse
if (HTTP.CONN_CLOSE.equalsIgnoreCase(token)) {
return false;
// If the response has the Connection:Keep-Alive header, it clearly indicates the intention to persist, reuse
} else if (HTTP.CONN_KEEP_ALIVE.equalsIgnoreCase(token)) {
keepalive = true;
}
}
if (keepalive) {
return true;
}
} catch (final ParseException px) {
return false;
}
}
// If no relevant Connection headers are present in the response, all versions above HTTP/1.0 reuse the connection
return !ver.lessEquals(HttpVersion.HTTP_1_0);
In summary:
-
If the request header contains Connection:Close, do not reuse. -
If the response Content-Length is not set correctly, do not reuse. -
If the response header contains Connection:Close, do not reuse. -
If the response header contains Connection:Keep-Alive, reuse it. -
If none match, reuse if the HTTP version is higher than 1.0.
From the code, we can see that the implementation strategy is consistent with the constraints of the protocol discussed in Chapters 2 and 3.
How HttpClient Cleans Up Expired Connections
In versions of HttpClient prior to 4.4, when obtaining reusable connections from the connection pool, it would check for expiration and clean them up if expired.
In later versions, a separate thread scans the connections in the connection pool. If it finds a connection that has not been used for longer than the configured time, it will clean it up. The default timeout is 2 seconds.
public CloseableHttpClient build() {
// The cleaning thread will only start if expired and idle connection cleaning is specified
if (evictExpiredConnections || evictIdleConnections) {
// Create a cleaning thread for the connection pool
final IdleConnectionEvictor connectionEvictor = new IdleConnectionEvictor(cm,
maxIdleTime > 0 ? maxIdleTime : 10, maxIdleTimeUnit != null ? maxIdleTimeUnit : TimeUnit.SECONDS,
maxIdleTime, maxIdleTimeUnit);
closeablesCopy.add(new Closeable() {
@Override
public void close() throws IOException {
connectionEvictor.shutdown();
try {
connectionEvictor.awaitTermination(1L, TimeUnit.SECONDS);
} catch (final InterruptedException interrupted) {
Thread.currentThread().interrupt();
}
}
});
// Execute the cleaning thread
connectionEvictor.start();
}
It can be seen that when building the HttpClientBuilder, if the cleaning function is manually enabled, a connection pool cleaning thread will be created and run.
public IdleConnectionEvictor(
final HttpClientConnectionManager connectionManager,
final ThreadFactory threadFactory,
final long sleepTime, final TimeUnit sleepTimeUnit,
final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
this.connectionManager = Args.notNull(connectionManager, "Connection manager");
this.threadFactory = threadFactory != null ? threadFactory : new DefaultThreadFactory();
this.sleepTimeMs = sleepTimeUnit != null ? sleepTimeUnit.toMillis(sleepTime) : sleepTime;
this.maxIdleTimeMs = maxIdleTimeUnit != null ? maxIdleTimeUnit.toMillis(maxIdleTime) : maxIdleTime;
this.thread = this.threadFactory.newThread(new Runnable() {
@Override
public void run() {
try {
// Infinite loop, the thread keeps executing
while (!Thread.currentThread().isInterrupted()) {
// Sleep for a few seconds before executing, default is 10 seconds
Thread.sleep(sleepTimeMs);
// Clean up expired connections
connectionManager.closeExpiredConnections();
// If a maximum idle time is specified, clean up idle connections
if (maxIdleTimeMs > 0) {
connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS);
}
}
} catch (final Exception ex) {
exception = ex;
}
}
});
}
To summarize:
-
Only when manually set in HttpClientBuilder will the expired and idle connection cleaning be enabled. -
After manual settings, a thread will be started that executes in an infinite loop. Each time it executes, it sleeps for a while and calls the cleaning method of HttpClientConnectionManager to clean up expired and idle connections.
Conclusion
-
The HTTP protocol reduces the excessive connection issues of early designs through persistent connections. -
There are two methods for persistent connections: HTTP/1.0+ Keep-Alive and HTTP/1.1 default persistent connections. -
HttpClient manages persistent connections through connection pools, which consist of a total connection pool and a connection pool for each route. -
HttpClient uses an asynchronous Future<CPoolEntry> to obtain a pooled connection. -
The default connection reuse strategy is consistent with the HTTP protocol constraints, first checking the Connection:Close header to decide not to reuse, then checking the Connection:Keep-Alive header to decide to reuse, and finally reusing if the version is greater than 1.0. -
Only after manually enabling the expired and idle connection cleaning switch in HttpClientBuilder will connections in the pool be cleaned up. -
In versions after HttpClient 4.4, a thread in an infinite loop cleans up expired and idle connections, executing every few seconds to achieve periodic execution.