gopherpace

Distributed, gossip-coordinated token-bucket throttling middleware for Go HTTP services.

gopherpace caps the aggregate request rate across every replica of a service — "1000 req/s total", not "1000 req/s per pod". Each replica runs a replica of one conceptual cluster-wide bucket: it enforces it locally on the hot path and gossips how many tokens it has admitted to its peers over memberlist.

Install

go get github.com/pedreviljoen/gopherpace

Quickstart

package main

import (
	"log"
	"net/http"

	"github.com/pedreviljoen/gopherpace"
)

func main() {
	limiter, err := gopherpace.New(gopherpace.WithRefillRate(100))
	if err != nil {
		log.Fatal(err)
	}
	defer limiter.Close()

	mux := http.NewServeMux()
	mux.HandleFunc("/work", handleWork)

	log.Fatal(http.ListenAndServe(":8080", limiter.Wrap(mux)))
}

With no seeds configured, New runs in standalone mode: a cluster of one node. Add WithSeeds to configure throttling across a cluster of service replicas:

import "github.com/pedreviljoen/gopherpace/seeds"

limiter, err := gopherpace.New(
	gopherpace.WithRefillRate(1000),
	gopherpace.WithSeeds(seeds.DNS{
		Host: "myservice-headless.default.svc.cluster.local",
		Port: 7946,
	}),
)

Supported configuration options

Option	Description	Default
WithRefillRate(refillRate int64)	Aggregate cluster-wide refill rate: how many whole tokens/second the bucket refills with. Required, must be > 0.	—
WithBucketSize(size int64)	Aggregate cluster-wide bucket capacity, in whole tokens.	equal to WithRefillRate
WithSeeds(p seeds.Provider)	Enables clustering: where to find gossip peers. Consulted afresh on every join attempt — see Seeds.	none (standalone)
WithGossipInterval(d time.Duration)	How often this node broadcasts its consumption. A node that has heard nothing from a peer for 4 intervals treats itself as partitioned and fails closed. Must be > 0.	500ms
WithBindPort(port int)	Gossip (memberlist) bind port, UDP + TCP. 0 means an OS-assigned port, which peers can then only learn via limiter.GossipAddr().	7946
WithNodeName(name string)	This node's identity in the cluster.	os.Hostname()
WithLogger(logger *slog.Logger)	Logger for all gopherpace logging (component=gopherpace).	slog.Default()

Under the hood mechanics

The mechanics is a plain token bucket: it holds up to WithBucketSize tokens, every request consumes one token, an empty bucket throttles (429), and tokens flow back in at the refill rate (tokens/second).

The configured refill rate and bucket size describe one conceptual cluster-wide bucket. With no central store, every node keeps a replica of it and the replicas are kept in step by gossip:

1. A request arrives and is evaluated against the tokens available in the local bucket.
2. The bucket refills at the full configured refill rate — every node's copy does, because every node's copy is the same bucket.
3. Every 500ms by default (WithGossipInterval), a node broadcasts its own cumulative count of admitted tokens to the cluster over memberlist (SWIM membership plus gossip transport, seeded by the seeds.Provider).
4. On hearing from a peer, a node subtracts the tokens that peer has admitted since it last heard from it, flooring at an empty bucket — tokens never go negative.
5. So in steady state each bucket drains at the cluster-wide admission rate and fills at the configured rate, and the aggregate converges on the configured rate — however the load balancer skews traffic, with no shares, no averages and no per-node rate to compute.

The refill rate is one constant value: it is never scaled or divided per node. The bucket has a capacity and a current value, and only the current value moves — requests decrement it, refill increments it.

Tokens and the refill rate are whole numbers (int64) end to end — there is no such thing as a partial request — and every rounding goes in the direction of admitting less: refill converts whole elapsed seconds only, and sub-second time carries into the next conversion, never credited early.

Failure telemetry

• Overshoot is bounded by roughly one gossip interval of cluster-wide traffic: what the cluster admits between a node's broadcasts is not yet charged to its peers.
• Partitions fail closed. A node that has clustering configured but is not in contact with any peer admits nothing: Allow returns (false, 1s) without touching the bucket and Wrap answers 429.

Entering the degraded state logs ERROR and repeats at most once every 30s while it lasts; recovery logs INFO. Nothing is ever logged per request.

Deployment / discovery examples

Kubernetes — headless Service

A headless Service (clusterIP: None) gives one DNS A record per pod IP, which is exactly what seeds.DNS expects:

apiVersion: v1
kind: Service
metadata:
  name: myservice-headless
spec:
  clusterIP: None
  selector:
    app: myservice
  ports:
    - name: gossip
      port: 7946
      protocol: UDP
    - name: gossip-tcp
      port: 7946
      protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myservice
spec:
  replicas: 5
  selector:
    matchLabels:
      app: myservice
  template:
    metadata:
      labels:
        app: myservice
    spec:
      containers:
        - name: myservice
          image: myservice:latest
          env:
            - name: GOPHERPACE_REFILL_RATE
              value: "1000"
            - name: GOPHERPACE_DNS
              value: "myservice-headless.default.svc.cluster.local"

limiter, err := gopherpace.New(
	gopherpace.WithRefillRate(1000),
	gopherpace.WithSeeds(seeds.DNS{
		Host: "myservice-headless.default.svc.cluster.local",
		Port: 7946,
	}),
)

ECS / Cloud Map

Register the service in AWS Cloud Map (Route 53 Service Discovery) so each task gets an A record under a shared DNS name:

{
  "family": "myservice",
  "containerDefinitions": [
    {
      "name": "myservice",
      "environment": [
        { "name": "GOPHERPACE_REFILL_RATE", "value": "1000" },
        { "name": "GOPHERPACE_DNS", "value": "myservice.internal" }
      ]
    }
  ]
}

limiter, err := gopherpace.New(
	gopherpace.WithRefillRate(1000),
	gopherpace.WithSeeds(seeds.DNS{Host: "myservice.internal", Port: 7946}),
)

Bare-metal/VM deployments without DNS-based discovery can use seeds.Static{"10.0.0.5:7946", "10.0.0.6:7946"} instead — or seeds.Fallback{seeds.DNS{...}, seeds.Static{...}} to try DNS first and fall back to a fixed list.

Usage examples

net/http

mux := http.NewServeMux()
mux.HandleFunc("/work", handleWork)
http.ListenAndServe(":8080", limiter.Wrap(mux))

Gorilla Mux

Limiter.Wrap is a standard func(http.Handler) http.Handler, so it works unchanged with Gorilla Mux's r.Use:

r := mux.NewRouter()
r.Use(limiter.Wrap)
r.HandleFunc("/work", handleWork)

Gin

The core module never imports a web framework, so Gin integration goes through Gin's own http.Handler adapter, or by calling Allow directly per request:

// Option 1: wrap the whole engine with net/http.
r := gin.Default()
r.GET("/work", handleWork)
http.ListenAndServe(":8080", limiter.Wrap(gin.WrapH(r)))

// Option 2: call Allow() per request inside a Gin middleware.
r.Use(func(c *gin.Context) {
	ok, retryAfter := limiter.Allow()
	if !ok {
		c.Header("Retry-After", strconv.Itoa(int((retryAfter+time.Second-1)/time.Second)))
		c.AbortWithStatus(http.StatusTooManyRequests)
		return
	}
	c.Next()
})

Tests & benchmarks

Run with go test -race ./.... All tests use the public API or direct in-package construction, no test hooks in the library code.

Package	Tests	Line coverage
gopherpace	33	92.5%
gopherpace/seeds	10	95.8%

Benchmark (go test -bench . -benchmem, Apple M-series, Go 1.26):

Benchmark	ns/op	allocs/op
BenchmarkAllow (hot path, admit)	~67	0