All articles

Making backpressure explicit in Go services

A bounded queue, deadlines, and clear overload responses can keep a small service responsive when demand exceeds capacity.

Making backpressure explicit in Go services

Measure the bottleneck before raising concurrency

A service often runs out of database connections, remote API quota, or memory before it runs out of goroutines. Raising concurrency can temporarily make more work appear active while increasing contention and queue time. Measure how many operations the slowest dependency sustains under a representative workload. Then choose a concurrency limit with room for other traffic and operational tasks.

Bound waiting as well as running work

Use a buffered channel when the queue is in-process and losing queued work at restart is acceptable. Give the queue a finite capacity and decide what a full queue means to the caller. An interactive endpoint can return 503 Service Unavailable with a short retry hint. A workflow that promises durable delivery needs persistent queue storage and a way to identify duplicate submissions.

Pass the request context to downstream calls. Accepted synchronous work should finish within the caller's deadline or stop when cancellation is observed. If a job is explicitly detached from the request, give it a separate job timeout and track its result independently. This prevents an abandoned client from holding scarce resources indefinitely.

Observe overload as a first-class event

Track queue depth, age of the oldest queued job, processing time, and rejected requests. Queue depth by itself can be misleading when work has different costs. A small queue that contains one very old job might indicate a blocked worker or a downstream timeout. When load testing, compare successful latency at several rates and verify that rejected work is explicit rather than timing out for every caller.

Plan shutdown deliberately. Stop admitting work, allow active jobs to finish within a bounded grace period, and make durable jobs visible again if the worker exits before acknowledging them. This gives capacity controls predictable behavior during both busy hours and deployments.