bytes.Buffer and sync.Pool: borrowed bytes, owned results and safe reuse

A byte slice returned from a buffer often looks like a finished value. You can print it, send it through a channel, or put it into a cache. The danger is that the result may be a temporary view into storage that its owner will reuse. The compiler sees an ordinary []byte; it cannot infer your application's lifetime contract. Correctness therefore depends on deciding when a view stops being valid and where an independent copy is required.

This guide follows one practical boundary: constructing a response in a reusable bytes.Buffer, returning a result, and eventually adding sync.Pool. The goal is a result that remains correct after the next request, not the fewest possible allocations in a tiny isolated benchmark. We also connect the same reasoning to scanner tokens, asynchronous consumers and request cancellation. The slice storage guide supplies the underlying memory model if shared backing arrays are still unfamiliar.

The Bytes method exposes the buffer's unread bytes. Treat the returned slice as borrowed storage whose validity depends on what happens to the buffer next. A safe ownership boundary can be demonstrated without a pool, a race, or a scheduler-dependent test.

package main

import (
    "bytes"
    "fmt"
)

func main() {
    var b bytes.Buffer
    b.WriteString("alpha")
    borrowed := b.Bytes()
    owned := bytes.Clone(borrowed)
    borrowed[0] = 'A'
    fmt.Println(b.String())
    b.Reset()
    b.WriteString("omega")
    fmt.Println(string(owned))
}

This prints Alpha and alpha. The first line demonstrates shared storage while the borrow is valid. The second demonstrates that the explicit copy survives later reuse. We deliberately do not inspect the borrowed slice after reset: an example that prints a particular stale result could encourage dependence on behavior outside the documented lifetime. The important contract is independence of the owned result, not which bytes happen to remain visible through an invalid view.

This distinction makes a better test too. Assert that the copy stays unchanged across reuse. Do not write a test that requires invalid borrowed data to become a particular string. Invalid does not mean immediately erased, and unchanged stale bytes do not make a borrow safe.

Reset makes a buffer ready for another operation while retaining its storage for reuse. It does not promise to overwrite every old byte with zeros. If your code handles secrets, buffer reuse and secret erasure are separate design problems. Reducing length to zero does not establish a data sanitization guarantee, and putting an object into a pool does not make its contents confidential.

Draw a timeline with three moments: obtain the view, consume the view, then mutate or release the owner. Synchronous use inside that interval is often ideal. Returning the view to a caller moves consumption outside the interval unless the caller also receives ownership of the buffer. The API must then return a copy, transfer the owner, or expose a clearly documented release protocol.

The new borrowed-buffer lesson provides a shorter walkthrough with this timeline. It also helps distinguish two uses of the word “copy”: copying a slice descriptor preserves sharing, whereas copying its elements creates separate byte storage. That distinction is easy to lose in a code review focused only on return types.

A pool can reduce repeated allocation of temporary objects, but it does not know whether a returned byte slice still points inside an object. Once an object has been put back, the former borrower must stop touching it. A different goroutine may obtain it, or the runtime may discard it; neither possibility is under the original caller's control.

The following complete program copies the response before returning the buffer. Its size cutoff is an example policy for this program, not a universal recommended number.

package main

import (
    "bytes"
    "fmt"
    "sync"
)

var buffers = sync.Pool{New: func() any {
    return new(bytes.Buffer)
}}

func encode(name string) []byte {
    b := buffers.Get().(*bytes.Buffer)
    b.Reset()
    b.WriteString("hello, ")
    b.WriteString(name)
    result := bytes.Clone(b.Bytes())
    if b.Cap() <= 64*1024 {
        b.Reset()
        buffers.Put(b)
    }
    return result
}

func main() {
    first := encode("Ada")
    _ = encode("Grace")
    fmt.Println(string(first))
}

It prints hello, Ada. The returned result is owned independently of the pool. The cutoff avoids deliberately retaining a very large buffer through this reuse mechanism. It does not place a hard bound on total process memory: multiple callers can hold buffers simultaneously, response copies still allocate, and other application objects remain live. For a strict resource budget, admission control and bounded concurrency are separate requirements.

A cache exists to make previously stored information available later. A pool exists to reuse temporary objects when convenient. Do not rely on retrieving the object you most recently put in, on a minimum residence time, or on the pool retaining a specific number of entries. Tests that depend on those properties may pass locally and fail when garbage collection, parallelism or runtime details change.

Connection management adds even more responsibilities: liveness checks, capacity limits, timeouts, closing and server-side state. Database client libraries generally provide their own connection management. Replacing that machinery with a sync.Pool of network connections changes the problem rather than solving it. Likewise, a pool is not a queue whose elements must all be processed or returned.

The pool ownership lesson asks you to trace who may access an object at each step. Its central invariant is simple: a temporary object has one active owner, and external results must stop depending on it before it becomes reusable. This invariant remains valuable even if profiling later shows that pooling is unnecessary.

Streaming parsers often avoid allocation by returning views into an internal buffer. For example, bufio.Scanner.Bytes returns a token whose storage may be overwritten by a subsequent scan. Processing it immediately is different from appending that slice to a long-lived collection. Saving a descriptor saves a view, not a durable token.

package main

import (
    "bufio"
    "bytes"
    "fmt"
    "strings"
)

func main() {
    scanner := bufio.NewScanner(strings.NewReader("red\nblue\n"))
    var tokens [][]byte
    for scanner.Scan() {
        tokens = append(tokens, bytes.Clone(scanner.Bytes()))
    }
    if err := scanner.Err(); err != nil {
        panic(err)
    }
    for _, token := range tokens {
        fmt.Println(string(token))
    }
}

The durable copies print red and blue. The error check matters independently of ownership: a parser can retain tokens correctly while silently ignoring a read or token-size failure. Large token requirements need explicit scanner configuration and an application-level input bound, not an unbounded assumption that every line will fit. These are complementary contracts: one controls lifetime, and the other controls accepted input and resource use.

Suppose a handler passes a borrowed payload to a logging goroutine. The handler returns the buffer before the goroutine actually formats the message. Even if a channel send completed, the consumer may only have received the descriptor. A later request can reuse the same array before logging finishes. The result may be corrupted, and concurrent access can also become a data race.

An independent copy before enqueueing is usually the simplest contract. An ownership-transfer design can avoid that copy, but it needs a rule for returning the buffer after consumption and for handling a consumer that exits early. Cancellation must specify who owns the object after a failed enqueue, after a successful enqueue, and after shutdown. Without those transitions, a “zero-copy” optimization can create both leaks and double returns.

Keep asynchronous logging bounded as well. Correct ownership does not stop a producer from enqueueing data faster than the consumer can process it. Decide whether overload blocks, drops or rejects work, and test that policy. Our goroutine lifetime analysis is a useful companion when buffer ownership and worker shutdown meet in the same service.

Write a deterministic test for the public result: obtain the first payload, perform many later encodes, and confirm the first bytes remain unchanged. Add parallel calls under the race detector to exercise concurrent use. Do not assert that two calls receive the same buffer; a correct pool is allowed to return different objects, and a correct implementation must survive either case.

Also test the oversize path. A large request should produce the correct result whether or not its temporary buffer is returned for reuse. If you introduce a custom allocator interface for measurement, keep that interface narrow and avoid turning implementation counters into the public contract. Correct output and lifetime independence should remain the primary assertions.

Run ordinary tests and race-enabled tests separately from performance benchmarks. Race instrumentation changes execution cost, so its timings are unsuitable for making throughput claims. For benchmarking, use representative size distributions, report allocations, and inspect retained memory after large bursts. A workload of only tiny uniform responses will not reveal the retention behavior of occasional oversized requests.

Start with a correct unpooled implementation and a clear ownership comment. Then profile. If temporary allocation is insignificant compared with network latency, database work or retained application state, a pool adds a protocol without a useful payoff. If pooling does help, keep the ownership boundary visible in a short function so a future change cannot accidentally return b.Bytes() directly.

A good review note says that returned bytes are independent and may be retained after the function returns. A borrowing API instead says exactly which next operation invalidates the view. Both are legitimate designs. Ambiguity is what makes maintenance difficult, especially when an agent or automated refactor changes a synchronous call into asynchronous work.

Finish by drawing two rows: the temporary buffer and the returned owned result. Mark the copy between them, then draw the release arrow only from the temporary row. If you cannot identify where those rows separate, the result probably still depends on the object you plan to reuse. Continue with runtime memory lessons, standard-library lessons, or the allocation and retention article to evaluate the tradeoff with measurements.

When a buffer is passed to an io.Writer, ask whether consumption is complete when Write returns. The standard Writer contract does not permit retaining the supplied byte slice after the call. This supports efficient synchronous use: the caller can ordinarily reuse its input after the operation returns, subject to the actual interface contract. It does not mean that an arbitrary function accepting []byte has the same promise. Check the interface instead of inferring lifetime from a familiar parameter type.

An asynchronous adapter must reconcile that synchronous contract with its own queue. If its Write implementation merely stores the caller's slice and returns, it has not established an independent payload. It needs to copy before returning or use a different API that explicitly transfers ownership. This is a useful review question for logging, batching and telemetry adapters because their public surface can look synchronous while the implementation performs work later.

Also inspect short writes and errors. A correct ownership protocol does not establish that all bytes were accepted. Conversely, retrying a partially completed operation can duplicate output if the surrounding protocol lacks an idempotent or offset-aware design. Keep the byte-lifetime question separate from delivery semantics, then test both at their respective boundary. For a custom adapter, use a controlled fake consumer that delays processing until after Write returns; that makes accidental retention observable without relying on scheduler luck.

The small encoder earlier has no application-level failure branch, but real encoders often do. A validator can reject input, an upstream reader can fail, or cancellation can end a request before a result is delivered. Draw these exits on the same ownership timeline as the successful path. Every exit should leave the temporary object either owned by a specific consumer, safely reusable, or deliberately discarded. An unlabelled exit is where leaks and double returns tend to hide.

A deferred cleanup can make the release path easier to audit, provided the returned result has already become independent. Do not mistake defer for an ownership solution by itself: returning a borrowed slice and deferring Put still releases its storage before the caller can safely retain it. The order of operations remains copy, then release. If a callback receives a borrowed view, document whether retaining it is forbidden and keep that callback's scope easy to inspect.

For error tests, force failure before output construction, during construction and after a result is ready but before enqueueing. Check the public error and any resource invariant that the API actually promises. Avoid requiring a pool to return the same object as proof that cleanup occurred. If you introduce a dedicated test double for ownership accounting, assert exactly one logical release, while leaving the runtime pool's identity and retention behavior outside the test contract.

Finally, revisit the size cutoff after a burst of large failures as well as large successes. A failed operation can grow a temporary buffer just as much as a successful one. A workload that usually produces tiny responses but occasionally builds a large rejected payload can therefore retain unexpected capacity. Measure that distribution before deciding which objects are worth returning for reuse.