Architecture: consistency and evolution · 3 MIN READ

Outbox and inbox: close the database–broker gap

Commit a business change and its publication intent together, then deduplicate delivery.

Writing an order and publishing its event are two different side effects. A crash between them can leave a missing event; publishing first can announce a change that never commits. An outbox stores publication intent beside the business write in the same durable transaction. A separate relay retries it, and the consumer expects duplicates rather than claiming end-to-end exactly once.

system-design · A MAP OF THE IDEA

Business + outbox commit
Relay publishes
Possible duplicate
Inbox + effect commit

Useful when

Commit a business change and its publication intent together, then deduplicate delivery.
Makes the owner and the failure boundary explicit.

Watch out

A relay can crash after broker acknowledgement but before marking the row sent, so duplicates are normal.
Run native service and failure tests in your own development environment.

Implementation & operation cost: Additional state and tests must be justified by the demonstrated failure; this lesson is not a requirement for every Go program.

The contract and the failure

Work through the example

Give each outbox record an immutable event ID, aggregate ID, payload/schema version and publication state. Commit it with the order, then have a bounded relay claim pending rows. The relay marks delivery only after the broker acknowledges. A consumer uses a unique event ID constraint and updates its own state in the same local transaction.

package main

import "fmt"

type Event struct{ ID, Order string }
type Inbox struct {
	seen  map[string]bool
	total int
}

func (i *Inbox) Apply(e Event) bool {
	if i.seen[e.ID] {
		return false
	}
	// Teaching model: one owner. Persist both changes atomically in production.
	i.total++
	i.seen[e.ID] = true
	return true
}
func main() {
	i := Inbox{seen: map[string]bool{}}
	e := Event{"evt-1", "order-7"}
	fmt.Println(i.Apply(e), i.Apply(e), i.total) // true false 1
}

Read the code

The stable event ID, not a newly generated delivery ID, identifies the duplicate. The model applies the first event once and rejects its replay. A production inbox must persist the deduplication record and business change in one transaction; this map intentionally demonstrates neither crash safety nor distributed atomicity.

Production decisions

A relay can crash after broker acknowledgement but before marking the row sent, so duplicates are normal. Marking sent first risks permanent loss. State the ordering requirement per aggregate, retention period and poison-event policy. Test crashes on both sides of acknowledgement and a consumer crash between its effect and deduplication write; a process-local map does not pass those tests.

Explain it in an interview

The broker acknowledged, then the relay crashed before marking sent. What now? Retry may deliver the event again. A stable ID and an atomic consumer inbox/effect boundary prevent repeating the business effect. Describe the state before the operation, its owner, the failure between steps and the evidence you would test. Distinguish what this example demonstrates from the infrastructure a production service still needs.

TAKE THIS WITH YOU

Commit a business change and its publication intent together, then deduplicate delivery. Retry may deliver the event again. A stable ID and an atomic consumer inbox/effect boundary prevent repeating the business effect.

Check your understanding

The broker acknowledged, then the relay crashed before marking sent. What now?

Go to the source

Progress stays in this browser. No account needed.