Around Go: MongoDB and Redis · 3 MIN READ

MongoDB cursors: bound the result and close the stream

Avoid loading an unbounded collection into application memory.

A cursor lets an application consume a result in batches; it is not permission to read the entire database without a budget. The query, projection, batch size, total limit and request deadline jointly determine network work and application memory. Check errors after iteration, not only when opening the cursor.

Around Go: data stores · A MAP OF THE IDEA

Bounded query
Batch from server
Process one document
Check and close

Useful when

Avoid loading an unbounded collection into application memory.

Watch out

Streaming only stays memory-bounded when the query and downstream consumer both have explicit limits.

Implementation & operation cost: Bounded iteration reduces retained memory but requires cursor ownership, cancellation and downstream backpressure.

Iterate with a deadline and a total limit

The v2 cursor’s Next can fail after some documents have already been decoded. Close the cursor on every path and propagate decode and cursor errors. A batch size controls transfer grouping; a total limit controls how much useful work the request is willing to perform.

cursor, err := notes.Find(ctx, filter, options.Find().SetLimit(100).SetBatchSize(25))
if err != nil { return err }
defer func() {
    cleanup, stop := context.WithTimeout(context.Background(), time.Second)
    defer stop()
    _ = cursor.Close(cleanup)
}()
for cursor.Next(ctx) {
    var note Note
    if err := cursor.Decode(&note); err != nil { return err }
    if err := consume(note); err != nil { return err }
}
return cursor.Err()

Read the code

The fragment assumes a bounded ctx, the application’s Note model and a consumer that does not retain every document. Cleanup gets a fresh bounded context so a canceled request does not prevent the cursor-close attempt.

Project only what this path needs

For a list view, return a small subset rather than large embedded payloads. Calling All on an unbounded cursor can materialize the whole result and exhaust memory. BatchSize does not impose a total row limit or protect you if consume appends every document to a slice.

javascript

db.notes.find({tenant: "demo"}, {body: 1, createdAt: 1})
  .sort({createdAt: -1, _id: -1}).limit(100).batchSize(25)

Read the code

The development shell query caps the result independently of the batch size. Keep a supporting index for the actual filter and sort and measure response bytes as well as document count.

Own downstream backpressure

If documents are sent to worker goroutines, use a bounded channel and cancellation path. Otherwise a streaming query can simply move the unbounded queue into Go memory. Do not concurrently use one cursor from multiple goroutines; give iteration one owner and distribute decoded values deliberately.

TAKE THIS WITH YOU

Bound total work separately from batch grouping, inspect cursor.Err, close on every path, and keep downstream buffering bounded.

Check your understanding

Why is a small batch size insufficient protection against memory growth?

Go to the source

Progress stays in this browser. No account needed.