Architecture: consistency and evolution · 3 MIN READ
Observe a Go use case without drowning in cardinality
Connect structured logs, bounded metrics and traces to an explicit service objective.
A service can emit millions of log lines while providing no evidence about a failed business operation. Start with a question: is the checkout successful within its time budget, and where is the budget spent? Logs explain individual events, metrics describe distributions and rates, and traces connect sampled work across boundaries. They should share stable operation semantics without sharing every field.
system-design · A MAP OF THE IDEA
- Define objective
- Measure boundary
- Trace dependencies
- Act on evidence
Useful when
- Connect structured logs, bounded metrics and traces to an explicit service objective.
- Makes the owner and the failure boundary explicit.
Watch out
- User IDs, transaction hashes and raw URLs as metric labels create unbounded time series.
- Run native service and failure tests in your own development environment.
Implementation & operation cost: Additional state and tests must be justified by the demonstrated failure; this lesson is not a requirement for every Go program.
The contract and the failure
A service can emit millions of log lines while providing no evidence about a failed business operation. Start with a question: is the checkout successful within its time budget, and where is the budget spent? Logs explain individual events, metrics describe distributions and rates, and traces connect sampled work across boundaries. They should share stable operation semantics without sharing every field.
Work through the example
Define latency and error SLIs at the user-visible boundary. Measure dependency waiting separately from execution, and include retries in the end-to-end budget. Use bounded labels such as operation and outcome. Put trace/request identifiers in logs or exemplars. With OpenTelemetry, propagate context, end spans, record errors appropriately and flush exporters during bounded shutdown.
package main
import (
"context"
"log/slog"
"os"
)
func main() {
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
ctx := context.Background()
logger.InfoContext(ctx, "order rejected", "operation", "place_order", "reason", "insufficient_stock", "request_id", "req-example")
}The event has a fixed message and named fields, so a reader can search by operation and reason rather than parse prose. request_id is a log field, not a metric dimension. slog does not automatically create a distributed trace; tracing requires explicit instrumentation and propagation configured in the native service.
Production decisions
User IDs, transaction hashes and raw URLs as metric labels create unbounded time series. Logging tokens, keys or entire request bodies leaks secrets. A healthy average hides tail latency and errors hidden by retries. Test redaction, cancellation and exporter failure; telemetry failure must not accidentally become an unbounded request queue. Pick alerts that correspond to an actionable service objective.
Explain it in an interview
Where should a unique transaction hash appear: every metric label or selected logs/traces? Keep high-cardinality identifiers in appropriately protected logs or traces; metrics use a bounded set of operation and outcome labels. Describe the state before the operation, its owner, the failure between steps and the evidence you would test. Distinguish what this example demonstrates from the infrastructure a production service still needs.
TAKE THIS WITH YOU
Connect structured logs, bounded metrics and traces to an explicit service objective. Keep high-cardinality identifiers in appropriately protected logs or traces; metrics use a bounded set of operation and outcome labels.
Check your understanding
Where should a unique transaction hash appear: every metric label or selected logs/traces?
Go to the source
Progress stays in this browser. No account needed.