Click the “blue text” above to follow us
Yesterday, a novice asked me: “Brother Feng, why does our Go service always run out of network bandwidth during peak times?” I smiled and said, isn’t this the classic “naked running” problem? Your HTTP response isn’t wearing a “compression coat”, and the data is like a fat person squeezing onto a bus, of course it takes up bandwidth! Today, I will guide you to implement a simple and efficient HTTP response compression middleware that can reduce your API size by over 50%!
1.
Content Compression: A Magic Trick to Halve Your Traffic Bill
Let me share some real data: our REST API service originally had a daily traffic of 300GB, and after adding gzip compression, it dropped directly to 120GB! The money saved is enough for the team to drink milk tea for a month. The core principle of HTTP compression is super simple: the server sends compressed content, and the browser receives and decompresses it. It’s like the delivery guy compressing your quilt into a package, and you restore it when you receive it. But many developers often forget this step, wasting traffic unnecessarily.
Don’t worry, implementing it in Go is super simple. In Go’s net/http package, we just need to wrap the handler function with gzip middleware, and all responses will automatically “wear a compression coat”. Is it really that simple? Yes, check out the code:
func GzipMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
next.ServeHTTP(w, r)
return
}
gz := gzip.NewWriter(w)
defer gz.Close()
w.Header().Set("Content-Encoding", "gzip")
next.ServeHTTP(gzipResponseWriter{Writer: gz, ResponseWriter: w}, r)
})
}
2.
Why Might Your Compression Middleware Perform Poorly?
Wait, don’t rush to copy and paste! Blindly using compression may backfire. I once saw a project where the API latency increased by 40% after compression! What happened? Because they compressed everything, including already compressed images, videos, and other binary files. It’s like vacuum-sealing a bag of snacks that has already been vacuum-sealed—it’s not only ineffective but also a waste of time!
What’s the smart approach? Adaptive Compression Strategy! Decide whether to compress based on content type and size. Let’s improve the code:
func AdaptiveGzipMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Capture response content
buf := &responseBuffer{ResponseWriter: w}
next.ServeHTTP(buf, r)
// Determine if compression is worthwhile
contentType := buf.Header().Get("Content-Type")
shouldCompress := strings.Contains(contentType, "text") ||
strings.Contains(contentType, "json") ||
strings.Contains(contentType, "xml")
// Responses smaller than 1KB are not worth compressing
if shouldCompress && len(buf.buffer) > 1024 &&
strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
w.Header().Set("Content-Encoding", "gzip")
gz := gzip.NewWriter(w)
gz.Write(buf.buffer)
gz.Close()
return
}
// Do not compress, return original content
w.Write(buf.buffer)
})
}
3.
Practical Experience: The Real Benefits of Compression
This compression middleware has been running in our production environment for over a year, and it’s not just about saving money. User experience has skyrocketed! Especially on mobile networks, the average page load time has decreased by 37%. One user jokingly said: “Did you change suppliers? Why is it suddenly so fast!”
However, compression also has costs. CPU usage will slightly increase, usually by about 3-5%. But considering the savings in bandwidth and the improvement in user experience, this cost is definitely worth it! If your server’s CPU is already over 90%, you might consider sacrificing some compression ratio for a faster compression level:
Here are some practical tips: compressing JSON and HTML type text yields the best results, usually reducing size by over 70%. For static files, consider pre-compression, which is more efficient than runtime compression. Responses smaller than 1KB may actually increase in size when compressed, so it’s better to send them directly. Using a compression pool (sync.Pool) to cache gzip.Writer objects can significantly reduce GC pressure and improve performance.
Finally, don’t be afraid to encounter pitfalls! Remember to handle “write after read” errors properly, and don’t let clients receive corrupted gzip data. Test with various clients, especially older browsers. If there are issues, add the Vary: Accept-Encoding header to help the CDN cache different versions correctly.
Alright, that’s it for today’s practical tips. Hurry up and add this “compression coat” to your Go service! Let your traffic bill be halved and user experience doubled. If you have any questions, feel free to ask me, and if you don’t understand something, Brother Feng will guide you step by step!
