How to Elegantly Implement HTTP Logging?

How to Elegantly Implement HTTP Logging?How to Elegantly Implement HTTP Logging?Click the blue text to follow us

Recently, I encountered a requirement to implement HTTP logging myself. Adhering to the principle of not reinventing the wheel, I first consulted AI and found a framework called logbook. As an HTTP logging framework, logbook can fully capture the headers, parameters, and body of requests and responses. This article mainly introduces the usage, implementation principles, core components, and extensions of the logbook framework on the Servlet server (Why limit to Servlet server? Because this framework also supports Reactive servers and HTTP logging for clients like Apache HttpClient, OkHttp, Spring RestTemplate, etc., which I will cover in another article).

1. Quick Start

1. Maven Configuration

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-spring-boot-starter</artifactId>
    <version>${logbook.version}</version>
</dependency>

It is important to note the compatibility issue with the Tomcat version. Generally, Spring Boot 2.* should use logbook 2.*, and Spring Boot 3.* should use logbook 3.*. This article selects Spring Boot version 2.6.6 and logbook version 2.14.0.

2. Logbook Configuration

logbook:
  obfuscate: # Mask specified content
    headers:
      - accept
      - accept-encoding
format:
    style: json # Optional http/curl/splunk
strategy: default # Default to record complete request and response
include: # Limit specific URLs
    - /*
exclude: # Exclude specific URLs
    - /health
    - /actuator/**

logging:
level:
    org.zalando.logbook: TRACE  # The framework uses org.zalando.logbook to record TRACE level logs by default

The corresponding class for logbook configuration is org.zalando.logbook.autoconfigure.LogbookProperties, which indicates that logbook can limit request paths, specify log formats, and output strategies, and can also mask specified content.

3. Test Interface and Log Output Effect

Test Interface

@GetMapping("/hello")
public String hello(@RequestParam("name") String name){
    return "Hello, " + name;
}

Log Output

2025-04-15 17:44:46.988 TRACE 29972 --- [nio-8088-exec-2] o.z.l.Logbook                            : {"origin":"remote","type":"request","correlation":"9b16c99c8636db2b","protocol":"HTTP/1.1","remote":"127.0.0.1","method":"GET","uri":"http://127.0.0.1:8088/demo/restful/hello?name=test","host":"127.0.0.1","path":"/demo/restful/hello","scheme":"http","port":"8088","headers":{"accept":["XXX"],"accept-encoding":["XXX"],"cache-control":["no-cache"],"connection":["keep-alive"],"content-type":["application/json"],"host":["127.0.0.1:8088"],"user-agent":["PostmanRuntime-ApipostRuntime/1.1.0"]}}

2025-04-15 17:44:47.011 TRACE 29972 --- [nio-8088-exec-2] o.z.l.Logbook                            : {"origin":"local","type":"response","correlation":"9b16c99c8636db2b","duration":23,"protocol":"HTTP/1.1","status":200,"headers":{"Connection":["keep-alive"],"Content-Length":["11"],"Content-Type":["text/plain;charset=UTF-8"],"Date":["Tue, 15 Apr 2025 09:44:47 GMT"],"Keep-Alive":["timeout=60"]},"body":"Hello, test"}

2. Implementation Principles

1. How does logbook intercept the Tomcat request flow?

Logbook intercepts the Tomcat request flow through the implementation class of the interface javax.servlet.Filter, called LogbookFilter. The LogbookFilter is registered to Tomcat via LogbookAutoConfiguration#logbookFilter using a FilterRegistrationBean (this happens during Spring Boot startup).

2. How is the Body read multiple times?

  • • Problem Source: Normally, the Body stream of HttpServletRequest and HttpServletResponse can only be read once. Reading it in the logs will cause the Body stream to be empty, affecting normal business request processing and response.
  • • Spring’s Solution: Spring provides ContentCachingRequestWrapper and ContentCachingResponseWrapper to wrap requests and responses, allowing the Body stream to be read multiple times.
  • • Logbook’s Solution: Implements RemoteRequest and LocalResponse to wrap requests and responses, allowing the Body stream to be read multiple times.
  • • Implementation Principle: Whether it is Spring’s ContentCachingRequestWrapper and ContentCachingResponseWrapper, or Logbook’s RemoteRequest and LocalResponse, the core principle of achieving repeatable reading is to cache the byte stream.

3. Core Components

HttpFilter
Logbook
Strategy
Sink
HttpLogWriter
HttpLogFormatter
logbook
  • • HttpFilter: Provides the default implementation of javax.servlet.Filter, with LogbookFilter as its parent class, serving as the entry point for logbook to intercept the Tomcat request flow.
  • • Logbook: The entry point of the logbook framework, providing path filtering, request timing, trace ID generation, and other functions, with the default implementation being DefaultLogbook.
  • • Strategy: Controls response codes and Body reading (conditions for logging response codes, whether the Body can be read), configurable via logbook.strategy, with the default implementation being DefaultStrategy.
  • • Sink: Controls log recording, generally holding HttpLogFormatter and HttpLogWriter, with the default implementation being DefaultSink.
  • • HttpLogWriter: Executes log recording, can be used to control the log recording method, with the default implementation being DefaultHttpLogWriter, which uses the logger org.zalando.logbook to record TRACE level logs.
  • • HttpLogFormatter: Controls the specific log content, with the default implementation being JsonHttpLogFormatter.

4. Custom Extensions

By default, the logbook framework uses the logger org.zalando.logbook to record TRACE level logs in JSON format, with fixed content fields. If you want to adjust the default recording method and content, you need to implement the relevant interfaces yourself.

1. How to Customize Log Content?

By default, the log output content is controlled by JsonHttpLogFormatter. To customize the log output content, you need to implement the HttpLogFormatter interface. You can refer to the implementations provided by logbook, such as CurlHttpLogFormatter, DefaultHttpLogFormatter, and JsonHttpLogFormatter. Your custom HttpLogFormatter will override the default implementation of logbook.

// Custom log content
@AllArgsConstructor
public class BizLogFormatter implements StructuredHttpLogFormatter {
    private static final ZoneId UTC_8 = ZoneId.of("Asia/Shanghai");

    private final ObjectMapper mapper;

    @Override
    public String format(final Map<String, Object> content) throws IOException {
        return mapper.writeValueAsString(content);
    }

    @Override
    public Map<String, Object> prepare(final Precorrelation precorrelation, final HttpRequest request)
            throws IOException {
        final String correlationId = precorrelation.getId();

        MDC.put("traceId", correlationId);

        final Map<String, Object> content = new LinkedHashMap<>();

        content.put("time", precorrelation.getStart().atZone(UTC_8));
        content.put("type", "request");
        content.put("traceId", correlationId);
        content.put("protocol", request.getProtocolVersion());
        content.put("remote", request.getRemote());
        content.put("method", request.getMethod());
        content.put("uri", request.getRequestUri());
        content.put("host", request.getHost());
        content.put("path", request.getPath());
        content.put("scheme", request.getScheme());
        content.put("port", preparePort(request));

        prepareHeaders(request).ifPresent(headers -> content.put("headers", headers));
        prepareBody(request).ifPresent(body -> content.put("req", body));

        return content;
    }

    @Override
    public Map<String, Object> prepare(final Correlation correlation, final HttpResponse response) throws IOException {
        final Map<String, Object> content = new LinkedHashMap<>();

        content.put("time", correlation.getEnd().atZone(UTC_8));
        content.put("type", "response");
        content.put("traceId", correlation.getId());
        content.put("protocol", response.getProtocolVersion());
        content.put("status", response.getStatus());

        prepareHeaders(response).ifPresent(headers -> content.put("headers", headers));
        prepareBody(response).ifPresent(body -> content.put("rsp", body));

        return content;
    }
}

2. How to Customize Log Output?

Logs can be recorded not only to log files but also directly output to the console or recorded in a database. To adjust the log recording location, you can implement the HttpLogWriter interface yourself. You can refer to the implementations provided by logbook, such as StreamHttpLogWriter and DefaultHttpLogWriter. Your custom HttpLogWriter will override the default implementation of logbook.

// Custom log writer name and log level
public class BizLogWriter implements HttpLogWriter {

    private final Logger log = LoggerFactory.getLogger("biz");

    @Override
    public boolean isActive() {
        return log.isInfoEnabled();
    }

    @Override
    public void write(final Precorrelation precorrelation, final String request) {
        log.info(request);
    }

    @Override
    public void write(final Correlation correlation, final String response) {
        log.info(response);
    }
}

3. How to Log Both Request and Response Content?

By default, request-related information is logged during the request, and response-related content is logged during the response, resulting in two separate log entries. If you want to log both request and response-related content in a single log entry, for example, if you are particularly concerned about interface performance and success rate, you need to implement the Sink interface and extend the HttpLogFormatter interface.

// Allow simultaneous retrieval of values from request and response
public interface ReqRspHttpLogFormatter extends StructuredHttpLogFormatter {
    default String format(final Correlation correlation, final HttpRequest request, final HttpResponse response) throws IOException {
        return format(prepare(correlation, request, response));
    }

    Map<String, Object> prepare(final Correlation correlation, final HttpRequest request, final HttpResponse response) throws IOException;
}
@AllArgsConstructor
public class AccLogFormatter implements ReqRspHttpLogFormatter {
    private static final ZoneId UTC_8 = ZoneId.of("Asia/Shanghai");

    private final ObjectMapper mapper;

    @Override
    public String format(final Map<String, Object> content) throws IOException {
        return mapper.writeValueAsString(content);
    }

    @Override
    public Map<String, Object> prepare(Correlation correlation, HttpRequest request, HttpResponse response) throws IOException {
        final Map<String, Object> content = new LinkedHashMap<>();

        content.put("traceId", correlation.getId());
        content.put("remote", request.getRemote());
        content.put("method", request.getMethod());
        content.put("scheme", request.getScheme());
        content.put("host", request.getHost());
        content.put("port", preparePort(request));
        content.put("path", request.getPath());
        content.put("protocol", response.getProtocolVersion());
        content.put("rspCode", response.getStatus());
        content.put("reqTime", correlation.getStart().atZone(UTC_8));
        content.put("rspTime", correlation.getEnd().atZone(UTC_8));

        content.put("processTime", correlation.getDuration().toMillis());
        return content;
    }
}
// Custom Sink, here we do not care about the request content, do not log the request
@AllArgsConstructor
public class AccSink implements Sink {

    private final ReqRspHttpLogFormatter formatter;
    private final HttpLogWriter writer;

    @Override
    public boolean isActive() {
        return writer.isActive();
    }

    @Override
    public void write(final Precorrelation precorrelation, final HttpRequest request) throws IOException {
    }

    @Override
    public void write(final Correlation correlation, final HttpRequest request, final HttpResponse response)
            throws IOException {
        writer.write(correlation, formatter.format(correlation, request, response));
    }
}

4. How to Log Two Types of Logs for One Request

Now there is a scenario where we need to log a biz log, which contains specific request and response content, and another acc log, which only concerns performance and success rate-related elements, without needing to log the specific request and response content. The biz log and acc log have different recording locations and content. Intuitively, one might think to let the Sink hold two sets of HttpLogWriter and HttpLogFormatter, but this does not comply with the single responsibility principle. A single Sink implementation should only be responsible for one set of log output. Logbook provides a CompositeSink implementation to hold multiple Sinks.

@Configuration
@AutoConfigureAfter
public class LogBookConfiguration {

    @Bean("bizLogFormatter")
    public HttpLogFormatter bizLogFormatter(final ObjectMapper mapper) {
        return new BizLogFormatter(mapper);
    }

    @Bean("bizLogWriter")
    public HttpLogWriter bizLogWriter() {
        return new BizLogWriter();
    }

    @Bean("bizSink")
    public Sink bizSink(final HttpLogFormatter bizLogFormatter, final HttpLogWriter bizLogWriter) {
        return new BizSink(bizLogFormatter, bizLogWriter);
    }

    @Bean("accLogFormatter")
    public ReqRspHttpLogFormatter accLogFormatter(final ObjectMapper mapper) {
        return new AccLogFormatter(mapper);
    }

    @Bean("accLogWriter")
    public HttpLogWriter accLogWriter() {
        return new AccLogWriter();
    }

    @Bean("accSink")
    public Sink accSink(final ReqRspHttpLogFormatter accLogFormatter, final HttpLogWriter accLogWriter) {
        return new AccSink(accLogFormatter, accLogWriter);
    }

    @Bean("sink")
    public Sink sink(ObjectProvider<Sink> sinks) {
        return new CompositeSink(sinks.stream().collect(Collectors.toList()));
    }

    /**
     * logbook response body defaults to ISO_8859_1 encoding, to avoid Chinese garbled characters, add this filter
     */
    @Bean("utf8Filter")
    public FilterRegistrationBean utf8Filter() {
        Filter utf8Filter = (request, response, chain) -> {
            response.setCharacterEncoding(StandardCharsets.UTF_8.name());
            chain.doFilter(request, response);
        };
        FilterRegistrationBean utf8Registration = new FilterRegistrationBean(utf8Filter);
        utf8Registration.setName("utf8Filter");
        return utf8Registration;
    }
}

Previously, we have provided implementations for BizLogFormatter, BizLogWriter, AccLogFormatter, and AccSink. Here we will also supplement the implementations for BizSink and AccLogWriter.

@AllArgsConstructor
public class BizSink implements Sink {

    private final HttpLogFormatter formatter;

    private final HttpLogWriter writer;

    @Override
    public boolean isActive() {
        return writer.isActive();
    }

    @Override
    public void write(final Precorrelation precorrelation, final HttpRequest request) throws IOException {
        writer.write(precorrelation, formatter.format(precorrelation, request));
    }

    @Override
    public void write(final Correlation correlation, final HttpRequest request, final HttpResponse response)
            throws IOException {
        writer.write(correlation, formatter.format(correlation, response));
    }
}
public class AccLogWriter implements HttpLogWriter {

    private final Logger log = LoggerFactory.getLogger("acc");

    @Override
    public boolean isActive() {
        return log.isInfoEnabled();
    }

    @Override
    public void write(final Precorrelation precorrelation, final String request) {
    }

    @Override
    public void write(final Correlation correlation, final String response) {
        log.info(response);
    }
}

5. Supplement

1. About Chinese Garbled Characters in Responses

The response in logbook uses LocalResponse for encapsulation, which defaults to using ISO_8859_1 for encoding the Body. If the response contains Chinese characters, garbled characters will appear. Based on the implementation of org.zalando.logbook.servlet.LocalResponse#getCharset, there are two solutions.

  • • Solution One: Specify @RequestMapping with produces = MediaType.APPLICATION_JSON_UTF8_VALUE
@GetMapping(value = "/hello", produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public String hello(@RequestParam("name") String name) {
    return "Hello, " + name;
}
  • • Solution Two: Customize a Filter to specify the response body encoding type. For specific implementation, refer to LogBookConfiguration#utf8Filter.

Previous RecommendationsHow to Elegantly Implement HTTP Logging?Command Pattern: A Flexible Design for Encapsulating Requests as ObjectsHow to Elegantly Implement HTTP Logging?Understanding MySQL Execution Plans in One ArticleHow to Elegantly Implement HTTP Logging?SQL Optimization – How I Improved SQL Execution Performance by 10 TimesENDHow to Elegantly Implement HTTP Logging?Don’t forget tolike, share, and show your love↓↓↓

Related posts

  1. Basics of HTTP Protocol
  2. C++ std::async and std::future: Simplifying Asynchronous Programming!
  3. Introduction to MATLAB Basics
  4. Can HTTP/2 + SSE Completely Replace WebSocket?
  5. Building a Custom Linux System with Buildroot
  6. Analysis of Domestic and International Suppliers of Industrial Robot Controllers
  7. Getting Started with HTTP Requests Using Httpie in Python
  8. Providing HTTP Services with Prolog
  9. Understanding HTTP Status Codes: Troubleshooting Made Easy!
  10. An Overview of the HTTP Protocol
  11. Understanding the HTTP/3 Protocol
  12. The Most Powerful Python HTTP Server: Setting Up a Local Server with http.server
  13. How to Receive Messages in Real-Time via HTTP Like TCP?
  14. Go Language HTTP Middleware Chain: Onion Model Design | Handling Cross-Cutting Concerns
  15. Learning from Ele.me: How to Optimize the Network Layer Architecture of iOS Apps Using HTTP/2?
  16. Using PromQL in Prometheus HTTP API
  17. In-Depth Exploration: No AI Agent, No SaaS!
  18. Upgrading Automation Systems with PC Control and EtherCAT
  19. Implementing High Precision Motor Control Algorithms with Industrial PCs

Leave a Comment