Go architectures · 3 MIN READ
Web, queue, worker: make slow work durable
Separate request latency from background completion.
Separate request latency from background completion. A web-queue-worker system accepts a task, durably records it and lets a worker finish later. Trace the worked Go example, then compare the benefit with the cost before applying this design to your own project.
Architecture · A MAP OF THE IDEA
- HTTP accepts and durably records a job
- Queue leases or delivers the job
- Worker commits an idempotent result
Useful when
- Slow work should not hold an HTTP request
- Workers need independent throughput control
Watch out
- Acceptance and completion need separate status
- Leases, retries and duplicates require durable design
Implementation & operation cost: High
Fit depends on the task
Editorial suitability, not benchmarks or a universal ranking. Styles describe different axes and can be combined.
The design decision
A web-queue-worker system accepts a task, durably records it and lets a worker finish later. The response should distinguish accepted from completed. Job identities and status transitions let clients safely retry and observe progress without keeping an HTTP request open.
A complete Go example
Read this program from the entry point and follow the calls shown in the diagram. The output is included at the end so you can check the behavior without a compiler.
package main
import (
"errors"
"fmt"
)
type Queue struct{ jobs []string }
func (q *Queue) Accept(id string) error {
if id == "" {
return errors.New("job id required")
}
q.jobs = append(q.jobs, id)
return nil
}
func (q *Queue) Next() (string, bool) {
if len(q.jobs) == 0 {
return "", false
}
id := q.jobs[0]
q.jobs = q.jobs[1:]
return id, true
}
func main() {
q := &Queue{}
fmt.Println(q.Accept("report-1"))
id, ok := q.Next()
fmt.Println(id, ok)
}
// Output:
// <nil>
// report-1 true
The example exposes acceptance and later consumption as different operations. Its slice is a single-process teaching queue: a crash loses jobs and Next removes them before work completes. A durable broker needs acknowledgments or leases and redelivery handling.
Tradeoffs and the simpler alternative
Use a durable queue and idempotent worker for valuable work. Record status and propagate trace identifiers so accepted jobs do not disappear operationally. A goroutine can handle disposable local work, but cannot promise completion after the process exits.
Make accepting work a durable promise
An HTTP 202 response says work was accepted, not completed. Validate and authorize the request, allocate a job ID and persist the job before returning success. Expose a status endpoint with pending, running, succeeded and failed states. Put the relevant input or an immutable reference in durable storage. A channel living inside the web process loses queued work when that process exits.
Use leases, retries and idempotent work
A worker claims a job for a limited lease and renews it while progressing. If it crashes, another worker can reclaim after expiry. Both may briefly believe they own the same job during a partition, so use an ownership token or fencing where the effect requires it. Bound attempts and preserve diagnostic state for exhausted jobs. A retry should resume or reconcile a durable operation rather than issue the same non-idempotent external effect twice.
Operate and test the backlog
Monitor oldest pending age, queue depth, active leases, completion latency and exhausted retries. Test a crash after the effect but before status update, cancellation while waiting, and a worker that loses its lease. Use an outbox when job creation and an application state change must commit together. Separate web and worker deployment only when resource or scaling needs justify it; they can initially share one executable and a durable queue.
TAKE THIS WITH YOU
Separates acceptance from eventual job completion. Use a durable queue and idempotent worker for valuable work. Record status and propagate trace identifiers so accepted jobs do not disappear operationally. A goroutine can handle disposable local work, but cannot promise completion after the process exits.
Check your understanding
Which statement best describes Web, queue, worker: make slow work durable?
Go to the source
Progress stays in this browser. No account needed.