backlog: record #1211's second excursion -- the margin is widening, not repeating - #358
Merged
Merged
Conversation
…ot repeating #1211 said a recurrence gets evidence rather than another re-run. This is the evidence. Two excursions on windows-2025, same test, same SLO, hours apart on unrelated changes: PR #343 fixed_per_conn@N=24: 36 < prior 48.4 * 0.75 (36.30) short by 0.30 PR #355 fixed_per_conn@N=24: 34.9 < prior 48.2 * 0.75 (36.15) short by 1.25 NEITHER CHANGE COULD REACH THE SUBSYSTEM. #343 was the worktree gate and its tests; #355 was a docs guard, ci.yml and four CLAUDE.md lines. Zero overlap with connscale, the store or the pipeline in either case, so attribution to the diff is ruled out on blast radius rather than on "flake". THE NEW INFORMATION IS THE TREND, not the repeat. The miss went 0.30 -> 1.25 against a prior that barely moved (48.4 -> 48.2). One excursion is the band being tight; two excursions drifting further out is the ratio behaving differently, which is exactly what this item was left open to catch. AND IT IS NOT THE CLOCK: the suite grew 11,961 -> 12,025 passing between the runs, and the failing step took 40:46 against a 55:00 cap, so the leg had headroom and was not being killed. Slowness would have shown up as a cap kill, not as an SLO miss. I did NOT re-run to make it green, and I did not file a new number -- this item already exists for this exact metric, and a second instance belongs on it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only. #1211 stays OPEN. No new number -- this item already exists for this exact metric, and a second instance belongs on it rather than beside it.
#1211's own text says a recurrence gets evidence, not another re-run. This is that evidence.
Two excursions, hours apart, on unrelated changes
Both
windows-2025, bothtest_connscale_smoke.py::test_connscale_smoke_end_to_end, both theempty_claims_monotonicband.Attribution is ruled out on blast radius, not on the word "flake"
#343 touched
scripts/hooks/worktree_gate.ps1and three gate test files. #355 touched a docs guard, its tests,ci.yml, and four lines ofCLAUDE.md. Neither has any overlap with connscale, the store, or the pipeline. Neither change could reach the subsystem that failed.The new information is the trend, not the repeat
The miss went 0.30 -> 1.25 against a
priorthat barely moved (48.4 -> 48.2). One excursion is a tight band; two excursions drifting further out is the ratio behaving differently, which is what this item was left open to catch.And it is not the clock
The suite grew 11,961 -> 12,025 passing tests between the runs, and the failing step took 40:46 against a 55:00 cap -- the leg had headroom and was not being killed. Runner slowness would surface as a cap kill, not as an SLO miss.
I did not re-run it to get a green. A green re-run here is consistent with the diagnosis rather than evidence against it, and banking one would have destroyed the second data point that makes the trend visible.