Skip to content

🐛 fix(fleetreport): dampener suppressions are not unclassified runtime errors - #17109

Merged
hivecommons-hive[bot] merged 8 commits into
v5from
scanner/fix-17103
Nov 8, 2026
Merged

hivecommons-hive[bot] merged 8 commits into
v5from
scanner/fix-17103

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

The dampener WARN (stage transition suppressed by dampener) is the stage engine working as designed; counting it as a repeated unclassified error files fleet selfreports for benign flap suppression. The slogfeed handler now skips it (stays in local logs), with a test. Closes #17103

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88
Closes #17111

…errors

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 7, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Nov 7, 2026
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 7, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Hive filed review findings that do not block this PR as backlog items:

Cap: 10 backlog item(s) per PR per day.

Reviewed by reviewer-2 — no findings.

Reviewer notes

Read the diff against the v5 tree and the emitting site. Looks correct to me.

No findings.

Review findings from reviewer (folded here because this head already carries its hive review):

Read the emitter and the handler, not just the diff. Looks correct to me.

  • The matched string in stageDampenerSuppression (src/pkg/fleetreport/slogfeed/slogfeed.go:236) is byte-identical to the only emitter in the tree, slog.Warn("stage transition suppressed by dampener", …) at src/pkg/stagelabels/store.go:781; no other call site logs that message, so the skip neither misses a variant nor swallows anything else.
  • The emitter is the [scanner] stage reconciler oscillates 050↔070↔085 on a needs-decision hive PR, re-labelling every 1–5 min and firing ~140 workflow runs/hour (#16607: 2500 runs, 300+ label events) #16974 dampener refusing a flap (>3 moves between two stages in 30 min without a new head or human event) and keeping the current label — designed behaviour, same class as the sibling skips in feed().
  • Message-only matching follows the existing expectedKickHold/inferenceBlockedStall idiom; the test mirrors TestReviewRetryServerErrorLogsWithoutFleetReport (12 records, zero observations, all 12 reach the wrapped handler).
  • Changelog fragment and src/docs/fleet-report.md paragraph are present and name the function/file.

Not approving (that is a human's call), but I found nothing to block on.

Evidence bundle `#171098dba1c0` sha256:51087aad0a3f411fda9fb9455fb9cf28d88977879e9841ddadc7553e905ac4c9 — download from the dashboard or `hivectl review evidence #17109`.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the filter against the emitter: src/pkg/stagelabels/store.go:781 logs exactly "stage transition suppressed by dampener", so the exact-match in stageDampenerSuppression (slogfeed.go:235) catches it, and the early return at slogfeed.go:117 runs before Key()/LogFedClass, so no observation can leak through another class. Same shape as sentinelPacingSkip; test mirrors TestReviewRetryServerErrorLogsWithoutFleetReport. Looks correct to me.

One note, not a defect: #17103 showed 12 suppressions in 10 min on a single PR (#17073 flapping 071-needs-author ↔ 050-review-pending). Dropping the selfreport is right, but the underlying flap stays invisible to the fleet now — worth a separate look if it recurs.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

…slogfeed skip list

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at 7e05404. Looks correct to me.

  • The filter string in src/pkg/fleetreport/slogfeed/slogfeed.go:236 matches the only emitter byte-for-byte: src/pkg/stagelabels/store.go:781 (slog.Warn("stage transition suppressed by dampener", …), inside the #16974 dampener branch that returns tr.Suppressed = true without touching the label). No other call site logs that message, so nothing else is silenced.
  • stageDampenerSuppression follows the same shape as the neighbouring message-match helpers (automergeMergeGuardSkip at slogfeed.go:244, reviewRetryServerError at :298) and is wired into the same early-return chain in feed (:117), so the record still reaches the wrapped handler — the new test asserts both (12 records pass through, 0 observations).
  • src/docs/fleet-report.md and changelog.d/fixed-17103.md describe the behaviour as implemented.

CI is green apart from tide pending.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me.

Verified against v5 at the merge-base:

  • The only emitter of this message is src/pkg/stagelabels/store.go:781 (slog.Warn("stage transition suppressed by dampener", …)), so the exact-match predicate stageDampenerSuppression in src/pkg/fleetreport/slogfeed/slogfeed.go catches every instance and nothing else.
  • The predicate is wired into the early-return chain in feed() ahead of Key()/LogFedClass, the same shape as reviewRetryServerError and automergeMergeGuardSkip (slogfeed.go:236-291), so the WARN still reaches the wrapped handler via Handle() (slogfeed.go:80-86) and only the fleet observation is skipped — which is what the test asserts (12 records through, 0 observations).

One info-level note, not blocking: the new paragraph in src/docs/fleet-report.md cites pkg/fleetreport/slogfeed/slogfeed.go while the neighbouring paragraphs cite src/…-prefixed paths (src/cmd/hive/main_sweeps.go, src/crux/agent/manager_kick.go). The citation guard passed, so this is cosmetic.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the change against the v5 tree; looks correct to me.

  • The emitter is src/pkg/stagelabels/store.go:781 — slog.Warn("stage transition suppressed by dampener", ...), fired only from isDampenedLocked inside the #16974 flap guard, so the exact-string match in stageDampenerSuppression (slogfeed.go:235-237) catches precisely that record and nothing else.
  • The skip sits in the early-return chain at slogfeed.go:117, ahead of Key()/LogFedClass, same shape as expectedKickHold (slogfeed.go:138). The record still reaches the wrapped handler, which the test asserts (12 forwarded, 0 observations).
  • src/docs/fleet-report.md gains the matching paragraph, so Closes #17111 (the docs-currency follow-up) is accurate, not just #17103.

Nothing to request. Merge state is only waiting on tide/overlayfs-exec-guard.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me. Read the emitter and the only ingestion path:

  • src/pkg/stagelabels/store.go:781 emits exactly "stage transition suppressed by dampener" (with key/head/from/to/reason attrs), so the strings.TrimSpace(r.Message) == match in the new stageDampenerSuppression (slogfeed.go:233) is exact, not a prefix guess.
  • slogfeed.feed is the only producer of ClassUnclassifiedRepeated observations (fleetreport.IsBenignMessage has no other callers outside slogfeed), so short-circuiting in feed fully stops the selfreport; nothing else re-derives the class from logs.
  • The test fails without the fix: newFeed observes every WARN+ record, so 12 WARNs produce 12 observations and len(*seen) != 0 trips; the strings.Count(out, "\n") == 12 check confirms the records still reach the wrapped handler (local logs keep them), matching the stated intent.

One non-blocking idiom note: the three existing exact-message, attr-independent benign cases (fleetreport.go:895, :901, :905) live in IsBenignMessage, while the slogfeed predicates are used for matches that also need to inspect attrs. This one needs no attr inspection, so it could have gone in IsBenignMessage; either placement is functionally equivalent here.

Docs paragraph in src/docs/fleet-report.md and the changelog.d/fixed-17103.md fragment match the behaviour change.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit c6957b6 into v5 Nov 8, 2026
78 of 79 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-17103 branch November 8, 2026 01:27
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026
hivecommons-hive Bot pushed a commit that referenced this pull request Nov 8, 2026
…time error (#17147)

The stage engine's same-head reversal guard (suppressStageFlap) logs
"stage flap suppressed" at WARN each time it refuses to undo its last
label write. That is the guard working as designed, but the slogfeed skip
added in #17109 only matched the stage-label store's dampener message, so
repeated suppressions still filed unclassified_error_repeated selfreports.
Skip both messages in stageDampenerSuppression.

Fixes #17142

Hive-Run: #17142
Hive-Plan: selfreport-triage
Hive-Spec: fleet-report#slogfeed-skip-list

Signed-off-by: Karibhou <Karibhou@users.noreply.github.com>
Co-authored-by: Karibhou <Karibhou@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

0 participants