Repository navigation
Add processor and gateway Prometheus metrics - #5
Merged
Merged
Conversation
Replaces the private-registry observability structs in messages-processor and delivery-processor with a simpler package-level Prometheus pattern using sync.Once on the default registry. Adds equivalent metrics.go to sms-processor (previously unmonitored) and a pgx DBQueryTracer to the gateway observability package. Key changes: - delivery-processor, messages-processor: delete observability.go (private registry struct + healthz/readyz HTTP server) and replace with metrics.go (global registry, sync.Once, startMetricsServer on APP_METRICS_ADDR) - sms-processor: add metrics.go with handler_results, latency, and retry counters; wire startMetricsServer into main - gateway/internal/observability: add dbtracer.go (DBQueryTracer for pgx), add dbQueriesTotal + dbQueryLatency to metrics.go + RecordDBQuery() - prometheus.yml: add messages-processor, delivery-processor, sms-processor scrape jobs; drop healthz-dependent scrape interval to 5s - docker-compose.yml: replace APP_HTTP_ADDR/:18088-18089 with APP_METRICS_ADDR/:9091-9093; replace /healthz healthcheck with port mappings; add sms-processor ports - messages-processor go.mod: remove pgxmock (tests that used it deleted) - sms-processor go.mod/go.sum: add prometheus/client_golang v1.23.2 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Prometheus metrics and DB tracing for gateway and processors: - Gateway DB query metrics/tracing (dbtracer.go) - Processor /metrics endpoints with result/retry/latency tracking - Prometheus scrape config and compose port wiring - Compatibility preservation for health endpoints and stage metrics Gateway observability tests: PASS Messages processor metrics tests: PASS Delivery processor metrics tests: PASS All builds: PASS Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds Prometheus metrics instrumentation across the gateway and processors, including DB query tracing/metrics, and updates the dev observability stack (Prometheus scrape config + docker-compose port wiring) to scrape the new endpoints.
Changes:
- Introduces
/metricsservers and new handler result/retry/latency metrics formessages-processor,delivery-processor, andsms-processor. - Adds PostgreSQL query tracing/metrics for the gateway and processors (via
pgxtracing). - Updates Prometheus scrape configuration and docker-compose wiring to target the new metrics ports/endpoints.
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| ohmf/services/sms-processor/go.sum | Updates module checksums to include Prometheus dependencies. |
| ohmf/services/sms-processor/go.mod | Adds Prometheus client dependency (and indirect deps). |
| ohmf/services/sms-processor/cmd/processor/metrics.go | Adds sms-processor Prometheus metrics and a metrics HTTP server. |
| ohmf/services/sms-processor/cmd/processor/main.go | Starts metrics server and records handler outcomes/retries/latency. |
| ohmf/services/messages-processor/cmd/processor/observability.go | Removes prior custom observability server/registry implementation. |
| ohmf/services/messages-processor/cmd/processor/observability_test.go | Removes tests tied to the removed observability server. |
| ohmf/services/messages-processor/cmd/processor/metrics.go | Adds new metrics server, stage counter compatibility, DB query tracing metrics, and helper recorders. |
| ohmf/services/messages-processor/cmd/processor/metrics_test.go | Adds unit tests for the new metrics helpers and metrics server. |
| ohmf/services/messages-processor/cmd/processor/main.go | Wires metrics server + pgx tracer; records new metrics during processing. |
| ohmf/services/messages-processor/cmd/processor/main_test.go | Removes processor observability wiring from tests. |
| ohmf/services/gateway/internal/observability/metrics.go | Adds gateway DB query metrics and SQL normalization helper. |
| ohmf/services/gateway/internal/observability/metrics_test.go | Adds tests asserting DB query metrics appear in /metrics. |
| ohmf/services/gateway/internal/observability/dbtracer.go | Adds pgx query tracer for gateway DB query metrics. |
| ohmf/services/delivery-processor/cmd/processor/observability.go | Removes prior custom observability server/registry implementation. |
| ohmf/services/delivery-processor/cmd/processor/metrics.go | Adds delivery-processor metrics server + DB query tracing metrics. |
| ohmf/services/delivery-processor/cmd/processor/main.go | Wires metrics server + pgx tracer; records new metrics during processing. |
| ohmf/services/delivery-processor/cmd/processor/main_test.go | Updates tests after observability removal and adjusts delivery processing expectations. |
| ohmf/infra/observability/prometheus.yml | Updates scrape intervals and adds scrape jobs for processors. |
| ohmf/infra/docker/docker-compose.yml | Wires processor metrics ports and updates healthcheck endpoints accordingly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+24
to
+28
| func (t *DBQueryTracer) TraceQueryEnd(ctx context.Context, _ *pgx.Conn, data pgx.TraceQueryEndData) { | ||
| start, _ := ctx.Value(dbTraceKey{}).(time.Time) | ||
| statement, _ := ctx.Value(dbStatementKey{}).(string) | ||
| RecordDBQuery(statement, data.Err, time.Since(start)) | ||
| } |
Comment on lines
+51
to
+63
| func startMetricsServer(addr string) { | ||
| initMetrics() | ||
| go func() { | ||
| server := &http.Server{ | ||
| Addr: addr, | ||
| Handler: promhttp.Handler(), | ||
| ReadHeaderTimeout: 5 * time.Second, | ||
| } | ||
| if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed { | ||
| log.Printf("sms-processor metrics server failed: %v", err) | ||
| } | ||
| }() | ||
| } |
Comment on lines
+16
to
+19
| // Reset the once so this test is isolated from other init paths. | ||
| // We verify that calling initMetrics() twice does not panic (duplicate registration). | ||
| initMetrics() | ||
| initMetrics() |
Comment on lines
+135
to
+138
| mux.HandleFunc("/readyz", func(w http.ResponseWriter, _ *http.Request) { | ||
| w.WriteHeader(http.StatusOK) | ||
| _, _ = w.Write([]byte("ok")) | ||
| }) |
Comment on lines
+84
to
+87
| mux.HandleFunc("/readyz", func(w http.ResponseWriter, _ *http.Request) { | ||
| w.WriteHeader(http.StatusOK) | ||
| _, _ = w.Write([]byte("ok")) | ||
| }) |
Comment on lines
+13
to
+17
| type DBQueryTracer struct{} | ||
|
|
||
| func NewDBQueryTracer() *DBQueryTracer { | ||
| return &DBQueryTracer{} | ||
| } |
Comment on lines
+23
to
+26
| - targets: | ||
| - messages-processor:9091 | ||
| - messages-processor-b:9091 | ||
|
|
Comment on lines
+27
to
+32
| - job_name: ohmf-sync-fanout-workers | ||
| metrics_path: /metrics | ||
| static_configs: | ||
| - targets: | ||
| - sync-fanout-worker-a:9094 | ||
| - sync-fanout-worker-b:9094 |
f8573
added a commit
that referenced
this pull request
Jun 11, 2026
* Add processor Prometheus metrics and gateway DB tracer Replaces the private-registry observability structs in messages-processor and delivery-processor with a simpler package-level Prometheus pattern using sync.Once on the default registry. Adds equivalent metrics.go to sms-processor (previously unmonitored) and a pgx DBQueryTracer to the gateway observability package. Key changes: - delivery-processor, messages-processor: delete observability.go (private registry struct + healthz/readyz HTTP server) and replace with metrics.go (global registry, sync.Once, startMetricsServer on APP_METRICS_ADDR) - sms-processor: add metrics.go with handler_results, latency, and retry counters; wire startMetricsServer into main - gateway/internal/observability: add dbtracer.go (DBQueryTracer for pgx), add dbQueriesTotal + dbQueryLatency to metrics.go + RecordDBQuery() - prometheus.yml: add messages-processor, delivery-processor, sms-processor scrape jobs; drop healthz-dependent scrape interval to 5s - docker-compose.yml: replace APP_HTTP_ADDR/:18088-18089 with APP_METRICS_ADDR/:9091-9093; replace /healthz healthcheck with port mappings; add sms-processor ports - messages-processor go.mod: remove pgxmock (tests that used it deleted) - sms-processor go.mod/go.sum: add prometheus/client_golang v1.23.2 * Add processor and gateway Prometheus metrics Adds Prometheus metrics and DB tracing for gateway and processors: - Gateway DB query metrics/tracing (dbtracer.go) - Processor /metrics endpoints with result/retry/latency tracking - Prometheus scrape config and compose port wiring - Compatibility preservation for health endpoints and stage metrics Gateway observability tests: PASS Messages processor metrics tests: PASS Delivery processor metrics tests: PASS All builds: PASS ---------
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Prometheus metrics and DB tracing for gateway and processors.
Scope
/metricsendpointsCompatibility
/healthzand/readyzremain available on processor metrics serversohmf_messages_processor_stage_events_totalremains functional with a deprecation commentValidation
go test ./internal/observability/...✓go test ./cmd/processor/...for messages-processor ✓go test ./cmd/processor/...for delivery-processor ✓go buildfor gateway, messages-processor, delivery-processor, and sms-processor ✓Remaining risk
A compose/dev-environment smoke test should verify
/metrics,/healthz, and/readyzreturn 200 at runtime.