[ecv]

Why

Investigating #113 needed a heap and goroutine profile from a production indexer pod. The indexer had no pprof endpoint — net/http/pprof was imported nowhere and Run() started no HTTP server of any kind, not even for metrics or health. Getting a single profile therefore meant building a one-off image, pushing it, suspending Flux, patching the deployment, and rolling ten production pods before any data could be collected.

That work produced the findings in this comment: goroutines climbing 103 → 4,521 on one pod with 1,114 parked in meilisearch-go.waitForTask, a heap ~82% under the NATS connection read path, and 29.3% of CPU spent in jetstream.Metadata() from the trackMessage duplicate scan. None of it was reachable without this change.

What

Adds --pprof-bind-address, empty by default. When set, starts a net/http/pprof server on that address.

• Handlers are registered on a dedicated ServeMux, not by importing net/http/pprof for its side effects, so http.DefaultServeMux is left untouched.
• ListenAndServe errors other than http.ErrServerClosed are logged, never fatal — profiling must not be able to take down the indexer.
• Shuts down on context cancellation with a 5s timeout.
• ReadHeaderTimeout set to 10s.
• Validate() rejects a malformed value via net.SplitHostPort.

The deployment passes --pprof-bind-address=$(PPROF_BIND_ADDRESS) in the same $(VAR) style as every other flag, defaulted to "".

Security

Inert unless deliberately enabled. No containerPort and no Service are added, deliberately — binding to 127.0.0.1:6060 keeps the endpoint reachable only through kubectl port-forward, which runs inside the pod network namespace, and never from the cluster network.

Usage

kubectl -n search-system set env deployment/resource-indexer PPROF_BIND_ADDRESS=127.0.0.1:6060
kubectl -n search-system port-forward pod/<indexer-pod> 6060:6060

curl -s 'http://localhost:6060/debug/pprof/goroutine?debug=2' > goroutines.txt
go tool pprof -inuse_space -top http://localhost:6060/debug/pprof/heap
go tool pprof -top 'http://localhost:6060/debug/pprof/profile?seconds=25'

Notes for review

• Conflicts textually with fix: Keep the indexer's memory flat during Meilisearch stalls #114, which adds --metrics-bind-address and its own listener in the same flag block and Run(). The two listeners are independent; whichever merges second needs a small manual merge.
• No behaviour change to indexing. Nothing under internal/indexer/ is touched.
• Verified with go build ./..., go vet ./cmd/search/..., gofmt -l, and a smoke test confirming HTTP 200 on /debug/pprof/, heap, goroutine and cmdline plus clean shutdown on context cancel. It also ran in production for ~45 minutes during the Search stops updating when a delete fan-out outgrows the indexer's memory #113 investigation.

Refs #113

🤖 Generated with Claude Code

https://claude.ai/code/session_01XHThCUggHF6zu6wMxHTrN3

[ecv]

Closing. The incident that prompted it was fixed in #114, and a profiling server can come back if another one needs it.