Overview
A high-throughput, low-latency logging system relies on two key aspects: deferring synchronous write delays to the background (asynchronous) and minimizing memory copies (zero-copy). This article provides engineering-level implementation ideas, reusable code snippets, and performance comparisons, suitable for direct adaptation in production systems.
In Short
• Decouple log collection (format) from disk writing: the main thread places logs into a circular buffer or message queue, while a background thread writes them in batches;• Use a fixed-size circular buffer and pre-allocate memory to avoid malloc/free for each log;• Zero-copy technique: write log content directly to the circular buffer/memory-mapped area (mmap) to reduce intermediate buffer copies;• Batch writing (writev, pwrite, or mmap+msync) is more efficient than writing logs one by one;• Ensure a recoverable strategy in case of crashes/exceptions: periodic disk writes, timed flushes, and replayable log files.
Design Contract
• Input: Any thread must submit a log with O(1) cost, without blocking the business thread for more than a brief period (e.g., <500µs);• Output: Final disk output should be in line-oriented text or binary events; support rolling by size/time;• Error: In case of disk or background thread errors, it should be possible to fall back (to local files or configurable blocking/discard strategies).
Core Components and Engineering Diagram
• Frontend API (log_emit): Format or reference existing buffers and submit via handle;• Circular buffer (ring buffer) or lock-free queue: Transmit messages across threads;• Background writing thread: Batch read from the circular buffer and write to disk;• Persistence strategy: writev / mmap + msync / async I/O.
Key Technology 1 — Asynchronous Writing (Decoupling)
Asynchronous writing separates I/O delays from the business path. Common implementations include:
• Single Producer Multiple Consumers (SPMC) or Multiple Producers Single Consumer (MPSC) lock-free queues;
- • Fixed-size circular buffer (power-of-two size, using bitmask for modulo);
- • Background thread writes in batches: pull N logs from the circular buffer at once, merging into a single writev or write call.Advantages: Business threads experience almost no I/O waiting; disadvantages: logs not written to disk may be lost in case of a crash (trade-off required).Code Example: MPSC circular buffer based on simple locks (illustrative, can be replaced with lock-free implementation as needed)
// simple_ring.h - minimal API
typedef struct ringbuf ringbuf_t;
ringbuf_t *ring_create(size_t capacity); // capacity bytes
void ring_destroy(ringbuf_t *r);
// push returns 0 on success, 1 when not enough space
int ring_push(ringbuf_t *r, const void *data, size_t len);
// pop: copies up to buf_len into buf, returns popped bytes
size_t ring_pop(ringbuf_t *r, void *buf, size_t buf_len);Core Point: Business threads should avoid memory allocation as much as possible, pre-allocating a buffer pool or using thread-local formatting buffers.
Key Technology 2 — Zero-Copy (Reducing memcpy)
The goal of zero-copy is to write log text directly into a shared area that can be read by the background thread, avoiding back-and-forth copying:
• Construct log lines directly in the circular buffer (pre-allocate slots);• Use memory-mapped files (mmap) to treat files as circular buffers, with the background writing to disk via msync/munmap;• Use writev to write multiple iov directly, reducing the overhead of merging strings (though kernel-level copies will still occur).Example: Pre-allocating slots in the ring buffer (pseudocode)
// Reserve a slot in the ring, returning a pointer to the slot and writable length
// On success, the business thread writes to this memory and calls ring_commit(r, len)
void *ring_reserve(ringbuf_t *r, size_t want, size_t *avail);
void ring_commit(ringbuf_t *r, size_t written);
// Usage example:
size_t avail;
char *slot = ring_reserve(r, 256, &avail);
int n = snprintf(slot, avail, "%s %s\n", tag, payload);
ring_commit(r, (size_t)n);The key to this pattern is ensuring the availability and atomicity of the slot (avoiding concurrent writes to the same slot). An atomic counter or CAS can be used for protection.
Reusable Implementation (Complete but Concise Demo)
Below is a runnable minimal implementation scheme (MPSC, with reserve/commit interface). This is a foundational version that can be directly evolved in engineering:
// tinylog.c - minimal demo (non-production level complete error handling)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
#include <unistd.h>
#include <stdatomic.h>
typedef struct {
size_t cap; // power of two
char *buf;
atomic_size_t head; // write pos (producer side)
atomic_size_t tail; // read pos (consumer side)
} ring_t;
ring_t *ring_create(size_t cap) {
ring_t *r = calloc(1, sizeof(*r));
r->cap = cap;
r->buf = malloc(cap);
atomic_init(&r->head, 0);
atomic_init(&r->tail, 0);
return r;
}
void ring_free(ring_t *r) { free(r->buf); free(r); }
// reserve returns pointer and avail via out param; returns NULL if not enough
void *ring_reserve(ring_t *r, size_t want, size_t *out_avail) {
size_t head = atomic_load_explicit(&r->head, memory_order_relaxed);
size_t tail = atomic_load_explicit(&r->tail, memory_order_acquire);
size_t used = head - tail;
if (want > r->cap - used) return NULL; // full
size_t idx = head & (r->cap - 1);
size_t avail = r->cap - idx;
*out_avail = avail < want ? avail : want;
return r->buf + idx;
}
void ring_commit(ring_t *r, size_t written) {
atomic_fetch_add_explicit(&r->head, written, memory_order_release);
}
size_t ring_pop(ring_t *r, void *out, size_t out_len) {
size_t tail = atomic_load_explicit(&r->tail, memory_order_relaxed);
size_t head = atomic_load_explicit(&r->head, memory_order_acquire);
size_t have = head - tail;
size_t toread = have < out_len ? have : out_len;
if (toread == 0) return 0;
size_t idx = tail & (r->cap - 1);
size_t first = r->cap - idx;
size_t n = first < toread ? first : toread;
memcpy(out, r->buf + idx, n);
if (toread > n) memcpy((char*)out + n, r->buf, toread - n);
atomic_fetch_add_explicit(&r->tail, toread, memory_order_release);
return toread;
}
// Background thread example: batch writing out
void *writer_thread(void *arg) {
ring_t *r = arg;
FILE *f = fopen("logs.txt", "a+");
char tmp[4096];
while (1) {
size_t n = ring_pop(r, tmp, sizeof tmp);
if (n == 0) { usleep(1000); continue; }
fwrite(tmp, 1, n, f);
// Flush according to strategy
fflush(f);
}
fclose(f);
return NULL;
}
// Business side example
void emit(ring_t *r, const char *s) {
size_t avail;
char *p = ring_reserve(r, strlen(s), &avail);
if (!p) return; // Discard or degrade strategy
memcpy(p, s, strlen(s));
ring_commit(r, strlen(s));
}
// main omittedNote: The above implementation is an educational example; production should include boundary checks, alignment, padding length fields, message headers, and concurrency safety optimizations (e.g., using sequence/CAS stacks).
Performance Highlights and Microbenchmark Recommendations
- • Pre-allocating memory is better than mallocing each time;
- • writev performs well when merging small fragments;
- • mmap + msync performs excellently in high-throughput, low-latency scenarios, but mapping wrap-around and persistence semantics must be handled;
- • Batch size significantly affects throughput: too small leads to frequent syscalls, too large increases latency.Recommended test scenarios:
- • Single-threaded high-frequency logging (millions of strings per second), measuring throughput and p50/p99 latency;
- • Multi-threaded concurrent submissions (M producers) + 1 writer;
- • Crash recovery: simulate SIGSEGV, check the amount of unwritten logs.
Frequently Asked Questions (FAQ)
- • Q: Why not write files directly in the business thread?A: Writing small amounts of files in concurrent scenarios can lead to numerous syscalls and lock contention, reducing throughput and scattering latency.
- • Q: Does zero-copy always improve performance?A: Not necessarily. Zero-copy reduces user-space memcpy but may increase memory management complexity, synchronization costs, and implementation complexity; benchmarks should validate in specific scenarios.
Conclusion
Design the logging system as a core infrastructure. First, establish the correct contract (non-blocking, configurable, recoverable), then optimize performance with asynchronous and zero-copy techniques.