Patterns: concurrent Go · 3 MIN READ

Worker pool: bound simultaneous work

Control active work instead of starting one goroutine for every input.

Control active work instead of starting one goroutine for every input. A worker pool bounds the number of operations in progress. Trace the worked Go example, then compare the benefit with the cost before applying this design to your own project.

Concurrent Go pattern · A MAP OF THE IDEA

Producer feeds a bounded handoff
Fixed workers process independent jobs
Coordinator closes results after all workers exit

Useful when

Independent work has a concurrency budget
Downstream services must not be overloaded

Watch out

Early consumer exit needs cancellation
Queue capacity and worker count are separate limits

Implementation & operation cost: Medium

Fit depends on the task

Small tool or CLI4/5

Domain-rich application4/5

Independent distributed services5/5

Editorial suitability, not benchmarks or a universal ranking. Styles describe different axes and can be combined.

The design decision

A worker pool bounds the number of operations in progress. Queue capacity is a separate limit on waiting work; neither should grow without a budget. Use a pool for independent jobs when the downstream capacity or memory cost is more important than spawning freedom.

A complete Go example

Read this program from the entry point and follow the calls shown in the diagram. The output is included at the end so you can check the behavior without a compiler.

package main

import (
	"fmt"
	"sync"
)

func Squares(jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
	defer wg.Done()
	for n := range jobs {
		results <- n * n
	}
}

func main() {
	jobs := make(chan int)
	results := make(chan int)
	var wg sync.WaitGroup
	for i := 0; i < 2; i++ {
		wg.Add(1)
		go Squares(jobs, results, &wg)
	}
	go func() {
		defer close(jobs)
		for _, n := range []int{1, 2, 3, 4} {
			jobs <- n
		}
	}()
	go func() { wg.Wait(); close(results) }()
	sum := 0
	for n := range results {
		sum += n
	}
	fmt.Println(sum)
}

// Output:
// 30

Read the code

Two workers share one input channel; a coordinator closes results only after both have exited. The receiver drains results concurrently with production, so unbuffered sends can progress. The sum is stable even though individual result order is not.

Tradeoffs and the simpler alternative

This finite example drains every job. Production cancellation must reach job production and result sends so an early consumer exit cannot strand workers. Measure downstream limits and queue latency; more workers can overload a database rather than increase useful throughput.

TAKE THIS WITH YOU

Limits active work independently of queue capacity. This finite example drains every job. Production cancellation must reach job production and result sends so an early consumer exit cannot strand workers. Measure downstream limits and queue latency; more workers can overload a database rather than increase useful throughput.

Check your understanding

Which statement best describes Worker pool: bound simultaneous work?

Go to the source

Progress stays in this browser. No account needed.