Skip to content

docs(backlog): one witness per e2e attempt, and twenty seconds - #459

Merged
G1NG4R merged 1 commit into
mainfrom
docs/one-witness-per-attempt
Oct 17, 2026
Merged

G1NG4R merged 1 commit into
mainfrom
docs/one-witness-per-attempt

Conversation

@G1NG4R

@G1NG4R G1NG4R commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Backlog only — no code. The measurements around #458, including one swap that
got away and the rule that follows from it.

What it carries

  • The f0e5998 swap is unobserved, and the cause was my scheduling. The
    witness was armed while e2e attempt 1 was running; that attempt failed,
    so deploy was gated off and its 2700 s rail burned down on a window where no
    swap could occur. It expired at 03:46:06. The rerun's deploy began at
    03:46:26 — twenty seconds later. A second witness started at 03:51 and
    measured what was already over.
  • The rule: one witness per e2e attempt, re-armed until the target sha is
    actually serving. It worked immediately on the next swap — armed 04:13:34,
    twelve seconds after e2e began, and the swap came in clean: 934 × 200 per
    path, zero non-200
    , offset +15.8 s.
  • v0.47.3 is the night's first release, because fix is a releasing type
    where the six preceding chore and docs merges correctly produced none.
  • p95 read 241.2 ms, measured five minutes after the restart. That is the
    docs(backlog): the watcher waited for itself, and p95 was stale #452 shape, and the entry says so without claiming a second reading: both
    of my reads returned the same measuredAt, so this is explained by precedent
    and arithmetic, not disproven by a fresh measurement.
  • Two tooling errors of my own, same family: a kill that returned 0 while
    the witness kept sampling (I had killed the wrapper, not the witness — a
    return code is not proof), and printf '--- …', whose format string begins
    with a dash and is read as an option. The second is why the coverage figure
    for test: the witness tail is measured from the twin, and closes before #304's window #453's sixth data point is derived rather than read.

Tally: 6 drops in roughly 41 430 requests, thirteen swap runs in a row without
one. A fifth no-deploy window was discarded rather than counted — it had no
deploy ahead of it and was stopped so the next witness could not run beside it.

make check green.

The f0e5998 swap went unobserved and the cause was my own scheduling:
the witness was armed for an e2e attempt that then failed, so its 2700 s
rail burned down on a window where deploy was gated off by definition,
and the rerun's deploy began twenty seconds later. The rule is one
witness per e2e attempt, re-armed until the target sha is serving -- and
it worked on the next swap, 934 x 200 per path with zero non-200.

Also: v0.47.3 is the night's first release, because fix releases where
chore and docs correctly did not; p95 read 241.2 ms five minutes after
the restart, which is the #452 shape and is written down as explained
rather than disproven, since the snapshot had not refreshed for a second
reading. And two tooling errors of my own, both of the same family: a
kill that returned 0 without touching the witness, and a printf whose
format string began with a dash.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@G1NG4R
G1NG4R merged commit 206df65 into main Oct 17, 2026
13 checks passed
@G1NG4R
G1NG4R deleted the docs/one-witness-per-attempt branch October 17, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant