Packages, modules and tooling · 3 MIN READ
Profile CPU, allocation and waiting as different bottlenecks
Use pprof and traces to investigate latency under load instead of increasing pools blindly.
A high-latency request does not necessarily consume much CPU. It may be waiting on a mutex, scheduling, a network response or a connection pool. Capture evidence for the same workload and time window, and choose a profile that can explain the suspected delay.
Packages and production tooling · A MAP OF THE IDEA
- Latency symptom
- CPU / allocation / wait
- Profile and trace
- One measured change
Useful when
- Use pprof and traces to investigate latency under load instead of increasing pools blindly.
Watch out
- A blocked request can spend most of its time waiting while consuming little CPU. Inspect queue, pool and synchronization delays as well as execution samples.
Implementation & operation cost: Profiles consume sampling overhead and need matching binaries, source revisions and a representative workload.
Capture a bounded benchmark profile
CPU profiles show sampled execution time. Heap and allocation profiles answer different memory questions, so choose the sample type deliberately. Keep the test binary with its profiles and source revision; otherwise symbol names and comparisons can become misleading.
go test ./internal/codec -run='^$' -bench=BenchmarkEncode -cpuprofile=cpu.out -memprofile=mem.out
go tool pprof -top cpu.out
go tool pprof -sample_index=alloc_space -top mem.outThe commands create local profile artifacts and show where time or allocated bytes accumulate in the measured benchmark. They do not show every blocked request or replace pool wait metrics from a real service.
Trace scheduling and explicit contention
Execution traces provide a timeline of goroutines and runtime activity. Block and mutex profiles need suitable runtime sampling configuration; collecting zero samples is not evidence that a service never waits. Profiling itself can affect the workload, so compare with the uninstrumented baseline.
go test ./internal/service -run TestBurst -trace=trace.out
go tool trace trace.out
go test ./internal/service -run TestBurst -blockprofile=block.out -mutexprofile=mutex.outReplace TestBurst with a bounded representative test. Inspect overlap among waiting, CPU work and goroutine growth before changing concurrency or blaming the garbage collector.
Turn evidence into one falsifiable change
If pool checkout dominates, inspect request arrival rate and slow operations before raising pool limits. If allocation dominates, find retained objects and hot allocation sites. Expose production profiling only through an appropriately restricted diagnostic path because profiles can reveal application data and implementation details.
TAKE THIS WITH YOU
Match the profile to the suspected bottleneck and compare one controlled change. CPU time, allocated bytes and waiting time are related but different measurements.
Check your understanding
Why can a slow service have a mostly idle CPU?
Go to the source
Progress stays in this browser. No account needed.