GopherForgeA WORKBENCH FOR GO

Around Go: MongoDB and Redis · 3 MIN READ

Redis Streams: acknowledge after the durable effect

Understand consumer groups, pending entries, redelivery and bounded retention.

A consumer group tracks delivery and acknowledgment, not automatic exactly-once application effects. A worker may crash after updating another database but before XACK. Redelivery is useful recovery, so the handler must recognize an operation it has already applied.

Around Go: data stores · A MAP OF THE IDEA

  1. Read group entry
  2. Apply idempotent effect
  3. Acknowledge
  4. Recover pending work

Useful when

  • Understand consumer groups, pending entries, redelivery and bounded retention.

Watch out

  • The effect may commit before a worker crashes without acknowledging. Redelivery can then repeat it; pending-entry recovery needs destination-side idempotency.

Implementation & operation cost: Consumer groups need retention, pending-work recovery, idempotent effects and explicit storage durability.

01

Read new entries with one group

In an isolated development instance, create a stream and group and append a demonstration event. XREADGROUP with > asks for new entries for this group. Acknowledge only after the application’s durable effect succeeds.

bash
redis-cli XGROUP CREATE demo:events workers 0 MKSTREAM
redis-cli XADD demo:events '*' operationID order-42 kind created
redis-cli XREADGROUP GROUP workers worker-1 COUNT 10 STREAMS demo:events '>'
Read the code

Replace worker names per process and retain the stream entry ID returned by the read. The demonstration group starts from 0 intentionally; starting at $ instead changes whether existing history is delivered.

02

Inspect and reclaim pending entries

A delivered but unacknowledged entry remains pending for the group. Inspect pending work and use a supported reclaim mechanism after an appropriate idle threshold. A slow healthy worker and a dead worker can look alike, so claiming work must not assume the old consumer has physically stopped.

bash
redis-cli XPENDING demo:events workers
redis-cli XAUTOCLAIM demo:events workers worker-2 60000 0-0 COUNT 10
# After the effect succeeds, acknowledge its actual returned entry ID:
# redis-cli XACK demo:events workers 1710000000000-0
Read the code

XAUTOCLAIM requires a compatible server version and returns a continuation cursor as well as claimed entries. Iterate with explicit work bounds; the sample acknowledgment ID is a placeholder, not a success shortcut.

03

Keep delivery, retention and durability distinct

Deduplicate the operation at the durable destination and treat replay as a normal path. Configure stream trimming so it does not discard work that still matters to consumers. Acknowledgment does not itself specify Redis persistence, replication acknowledgment or recovery guarantees; test those separately for the deployment.

TAKE THIS WITH YOU

Acknowledge after a durable idempotent effect, recover pending work deliberately, and budget retention and storage independently of consumer-group delivery.

Check your understanding

Why must a stream consumer make its external effect idempotent?

Go to the source

Progress stays in this browser. No account needed.