Repository navigation
π docs: monitoring a hive with Prometheus and Grafana - #15497
Conversation
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: onkar717 <144542684+onkar717@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Docs map: this PR adds page(s) under
A page missing from the map is one readers do not find. Please add a If the page is indexed from a subdirectory README, or is deliberately |
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
stage 080-awaiting-merge-admission: no activity for 30m0s: merge-train:queued-behind#15405:2/2:0/0:operator-reference.md |
There was a problem hiding this comment.
Correctness review (read at head 579dc9e).
In-scope β monitoring.md "Turning metrics on" does not actually turn metrics on in the k8s deployment. src/docs/monitoring.md:9-18 says to set HIVE_METRICS_ENABLED=1, put HIVE_METRICS_TOKEN in the hive-secrets Secret (kubectl patch secret β¦) and "roll the hive pod afterwards so the environment is re-read". But src/deploy/k8s/deployment.yaml:161-184 only maps specific keys via secretKeyRef (HIVE_GITHUB_TOKEN, HIVE_DASHBOARD_TOKEN, DASHBOARD_AUTH_TOKEN, β¦) β there is no envFrom, no HIVE_METRICS_TOKEN/HIVE_METRICS_ENABLED entry, and the Secret is otherwise only mounted as files at /secrets (deployment.yaml:190-192, 239-242) which entrypoint.sh never exports as env. metricsToken() reads os.Getenv("HIVE_METRICS_TOKEN") (src/pkg/dashboard/metrics_prometheus.go:33-35), so following the page as written leaves the route unregistered (404) β or, if an operator sets HIVE_METRICS_ENABLED some other way, fail-closed 403 forever with the server.go:1615-1616 warning. The page also never says how to set HIVE_METRICS_ENABLED on the k8s Deployment. Fix: add the env: patch operators need (a Kustomize patches: entry adding HIVE_METRICS_ENABLED: "1" and HIVE_METRICS_TOKEN via secretKeyRef {name: hive-secrets, key: HIVE_METRICS_TOKEN}), or wire these two vars into deployment.yaml (optional: true) and keep the Secret-patch recipe.
Advisory (low): monitoring.md:66-72 "Alert reference" is an empty stub, while #15438 already names the three alerts (HiveMetricsDown, HiveCostBurnHigh, HiveStageTransitionAnomalies) and their conditions; pre-seeding those subsections would satisfy the issue's "what each alert means and what to do" item and give #15438 stable runbook_url anchors on day one.
Verified OK against head: 200/401/403/404 table vs metrics_prometheus.go:83-92 + server.go:1613-1618; Service hive, port dashboard/3002 (src/deploy/k8s/service.yaml); metric family names/labels vs metrics_prometheus.go:49-73.
β hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
Hive-Fix-Attempt: 1/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: guide <280983584+hivecommons-hive[bot]@users.noreply.github.com>
37b0583
βΉοΈ Sentinel notice β informational; author is trusted (hive agent), merge is not blockedHive flagged this PR (author @hivecommons-hive[bot], head
Hive did not add the |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Closes #15440
Adds an operator page covering metrics enablement (fail-closed /metrics), the monitoring overlay component, metric families, alert runbook anchors, and plain scrape_config without Prometheus Operator; linked from operator-reference and env-vars.
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@onkar717