Go services · 3 MIN READ

Cache misses, retries and overload amplification

Design a cache-miss path that avoids multiplying identical work and ambiguous retries.

A cache is often added to reduce load, yet its failure path can multiply load. When a popular key expires, many callers may fetch the same value at once. When a timeout triggers retries at several layers, one user action can become many downstream operations.

Collapse duplicate work within a process

singleflight lets concurrent callers for the same key share one in-flight computation. The loader should have an explicit bounded lifetime. In this fragment loadCtx is owned by the service, not borrowed indefinitely from whichever request happened to arrive first.

result, err, _ := group.Do(key, func() (any, error) {
    ctx, cancel := context.WithTimeout(loadCtx, time.Second)
    defer cancel()
    return loadFromStore(ctx, key)
})
if err != nil { return err }
use(result)

Read the code

group is a shared singleflight.Group. This synchronous form waits for the shared result; callers that must independently stop waiting need a cancellation-aware design, for example DoChan with a select. Coalescing is per process, not a distributed lock across replicas.

Define cache freshness and failure policy

Choose TTLs from the data contract. Jitter can spread expirations, but it does not make stale data acceptable automatically. Serving a stale value may be useful for a catalogue and wrong for an authorization decision. Bound key count and value size, and ensure a miss cannot start unlimited fallback work.

Retry only a retryable, safe operation

Use a small attempt budget, backoff with jitter and an overall deadline. A timed-out write may already have committed, so an automatic retry can duplicate a side effect without an idempotency contract. Coordinate retries at one layer where possible. Record attempts and final outcomes so apparent success does not conceal amplified downstream load.

TAKE THIS WITH YOU

Coalesce duplicates, bound fallback work and retry under an explicit contract. A faster happy path does not excuse an unbounded failure path.

Check your understanding

Which statement is correct about singleflight.group?

Go to the source

Progress stays in this browser. No account needed.