Skip to content

Commit a5dfce4

Browse files
authored
fix: don't send prometheus unless remote url is specified (#2468)
1 parent e4ed400 commit a5dfce4

5 files changed

Lines changed: 55 additions & 10 deletions

File tree

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
'@hyperdx/otel-collector': patch
3+
---
4+
5+
fix(otel-collector): only enable the prometheus remote-write exporter in
6+
standalone mode when `CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT` is set
7+
8+
The standalone collector config used to unconditionally declare a
9+
`prometheusremotewrite` exporter and a `metrics/promql` pipeline. When
10+
`CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT` was unset the exporter rendered
11+
with an empty endpoint and every metrics batch failed to export.
12+
13+
The exporter and pipeline have been moved to
14+
`docker/otel-collector/config.standalone.promql.yaml`, which is now only
15+
loaded by `entrypoint.sh` when `CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT` is
16+
non-empty. This mirrors the OpAMP-managed gating in
17+
`packages/api/src/opamp/controllers/opampController.ts` (which already
18+
only adds the exporter when `IS_PROMQL_ENABLED` is true).
19+
20+
No action required if `CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT` is set; the
21+
behavior is unchanged. If it was unset, the collector now stops emitting
22+
the failing prometheus-remote-write attempts.

‎docker/otel-collector/Dockerfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ LABEL org.opencontainers.image.vendor="HyperDX" \
7575
COPY --chown=10001:10001 docker/otel-collector/config.yaml /etc/otelcol-contrib/config.yaml
7676
COPY --chown=10001:10001 docker/otel-collector/config.standalone.yaml /etc/otelcol-contrib/standalone-config.yaml
7777
COPY --chown=10001:10001 docker/otel-collector/config.standalone.auth.yaml /etc/otelcol-contrib/standalone-auth-config.yaml
78+
COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /etc/otelcol-contrib/standalone-promql-config.yaml
7879
COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl
7980
COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema
8081

@@ -97,6 +98,7 @@ LABEL org.opencontainers.image.vendor="HyperDX" \
9798
COPY --chown=10001:10001 docker/otel-collector/config.yaml /etc/otelcol-contrib/config.yaml
9899
COPY --chown=10001:10001 docker/otel-collector/config.standalone.yaml /etc/otelcol-contrib/standalone-config.yaml
99100
COPY --chown=10001:10001 docker/otel-collector/config.standalone.auth.yaml /etc/otelcol-contrib/standalone-auth-config.yaml
101+
COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /etc/otelcol-contrib/standalone-promql-config.yaml
100102
COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl
101103
COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema
102104

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Enables remote-write of OTLP metrics to a Prometheus-compatible endpoint.
2+
# Only included when CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT is set.
3+
#
4+
# Mirrors the OpAMP-managed behavior in
5+
# packages/api/src/opamp/controllers/opampController.ts (gated on
6+
# IS_PROMQL_ENABLED). When updating this file, keep it in sync with
7+
# buildOtelCollectorConfig().
8+
9+
exporters:
10+
prometheusremotewrite:
11+
endpoint: http://${env:CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT}/write
12+
tls:
13+
insecure: true
14+
resource_to_telemetry_conversion:
15+
enabled: true
16+
17+
service:
18+
pipelines:
19+
metrics/promql:
20+
receivers: [otlp/hyperdx]
21+
processors: [memory_limiter, batch]
22+
exporters: [prometheusremotewrite]

‎docker/otel-collector/config.standalone.yaml‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -53,12 +53,6 @@ exporters:
5353
initial_interval: 5s
5454
max_interval: 30s
5555
max_elapsed_time: 300s
56-
prometheusremotewrite:
57-
endpoint: http://${env:CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT}/write
58-
tls:
59-
insecure: true
60-
resource_to_telemetry_conversion:
61-
enabled: true
6256

6357
service:
6458
# Pipeline `processors:` lists live in the base config.yaml so users can
@@ -71,10 +65,6 @@ service:
7165
metrics:
7266
receivers: [otlp/hyperdx]
7367
exporters: [clickhouse]
74-
metrics/promql:
75-
receivers: [otlp/hyperdx]
76-
processors: [memory_limiter, batch]
77-
exporters: [prometheusremotewrite]
7868
logs/in:
7969
receivers: [otlp/hyperdx]
8070
exporters: [routing/logs]

‎docker/otel-collector/entrypoint.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,15 @@ if [ -z "$OPAMP_SERVER_URL" ]; then
5151
COLLECTOR_ARGS="$COLLECTOR_ARGS --config /etc/otelcol-contrib/standalone-auth-config.yaml"
5252
fi
5353

54+
# Enable prometheus remote-write of OTLP metrics only when an endpoint is
55+
# configured. Mirrors the OpAMP-managed IS_PROMQL_ENABLED gating in
56+
# packages/api/src/opamp/controllers/opampController.ts so the standalone
57+
# collector does not attempt to export to a missing endpoint.
58+
if [ -n "$CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT" ]; then
59+
echo "CLICKHOUSE_PROMETHEUS_METRICS_ENDPOINT is set, enabling prometheus remote-write"
60+
COLLECTOR_ARGS="$COLLECTOR_ARGS --config /etc/otelcol-contrib/standalone-promql-config.yaml"
61+
fi
62+
5463
# Add custom config file if specified
5564
if [ -n "$CUSTOM_OTELCOL_CONFIG_FILE" ]; then
5665
echo "Including custom config: $CUSTOM_OTELCOL_CONFIG_FILE"

0 commit comments

Comments
 (0)