Repository navigation
fix(ci): an unreachable product endpoint is not documentation drift - #13
Merged
bitcoinuniverseadmin merged 3 commits intoAug 29, 2026
Merged
Conversation
The House Manual failed to publish twice this afternoon and once this morning, all three times on the same step, none of them because anything was wrong with the documentation: live check: https://forkedfelines.art/api/v1/product unreachable after 4 attempts: TypeError: fetch failed The check conflated two different events behind one exit code. A mismatch means the product moved and the documentation is now wrong, which must never publish. Unreachable means a runner could not open a socket, which says nothing about the facts in either direction. Treating them alike meant a network fault blocked the publication of documentation that was correct, while the site went on serving an older build. That is strictly the worse of the two outcomes, and it is what happened: the expired-order tone fix sat unpublished while the previous wording stayed live. Unreachable now exits 75, EX_TEMPFAIL, and both workflows turn that into a warning annotation and continue. A mismatch still exits 1 and still stops the publish. The facts are not unguarded while the endpoint is away. check-docs.mjs pins every one of them offline against hardcoded values and runs first in the Pages workflow and as a sibling job in CI, so an unreachable product leaves them guarded by the pinned check alone, which is a reasonable posture for a transient network fault where the old one was not. Deliberately not a wider retry budget: that trades a visible failure for a slower one and still fails whenever the path is out for longer than whatever number gets picked. A new publication-gate rule keeps the two exit codes distinct and rejects either workflow running the check bare, so this cannot quietly collapse back into one failure. Evidence that it is the network and not the product: at 12:04 two GitHub-hosted runners ran this identical check against this identical endpoint within the same minute. One reached it and passed. The other could not, for 72 seconds across four attempts. The endpoint answered 200 from the workstation throughout, in about 160ms, with no AAAA record and no rate-limit headers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of the first commit found a fail-open I had introduced. Three
conditions landed in one catch and left through one exit code, and after
moving that exit code to 75 they all published with a warning:
fetch threw, no response genuinely transient
response answered, !response.ok it ANSWERED
response.json() threw it ANSWERED
The second is the one that matters. If /api/v1/product is renamed, moved,
or version-bumped out from under the documentation, the endpoint answers
404, and the previous commit would have reported that as UNREACHABLE and
published the House Manual against facts nobody compared. That is exactly
the event this gate exists to catch, arriving in a network costume, and
it was worse than before my change rather than merely unfixed: a 404 used
to exit 1 and block.
The decision now reads what the last attempt observed:
no response 75 nothing was learned
5xx 75 it answered and is having a bad time, which is
also what a deploy window looks like
429 75 rate limited, not moved
any other 4xx 1 the contract is not where the docs say it is
2xx unparseable 1 it answered and it is not a contract
The last attempt wins rather than the worst one, because a 404 followed
by three dropped connections is better described as losing the path than
as the contract moving.
All five paths were exercised against a real local server rather than
asserted. The first harness reported two false failures because
spawnSync blocks the event loop, so the in-process server could never
answer and every case looked unreachable; re-run out-of-process, 404 and
the unparseable body exit 1, and 500, 429 and a refused connection exit
75.
The gate rule is tightened to match. Asserting that EX_TEMPFAIL exists
says nothing about which conditions reach it, so it now pins the
transient set and rejects any widening to 4xx, which was proved to bite
by widening it by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review found the one sequence where last-attempt-wins resolves the wrong way: attempt 1 404 the contract moved attempts 2-4 refused the path drops last wins -> 75 -> publishes That is the gate's own case arriving and then being masked by noise. Unlikely, and the only sequence where the ordering rule changes the outcome at all. Decided on the asymmetry rather than on likelihood. Wrongly exiting 1 costs a blocked publish: visible, recoverable, one re-run. Wrongly exiting 75 costs a silent publish against a contract nobody compared, which is precisely what this gate exists to prevent. Where the two error costs are that lopsided, be wrong in the direction someone notices. The change is one clause rather than worst-wins wholesale. A 4xx that is not 429 is sticky once seen; everything else keeps the last-attempt rule and the existing boundaries. Ordering never mattered among no-response, 5xx and 429 anyway, since all three answer 75 regardless. The counter-argument is recorded in the source because it is the thing that would change this: if some intermediary answered 404 while transiently failing, a healthy deploy would block a publish for no reason. Nothing in this path is known to do that. A container swap surfaces as 502 or 503, and a 404 from this endpoint means the route is not registered, which is the persistent condition rather than the transient one. Exercised against a server that answers 404 once and then stops listening, which is the exact sequence: exit 1, "answered 404 on at least one of 4 attempts". The gate rule pins the stickiness and was proved to bite by renaming the variable that carries it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bitcoinuniverseadmin
deleted the
fix/live-check-unreachable-is-not-drift
branch
August 29, 2026 12:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User problem
The House Manual failed to publish three times today, all on the same step, none of them because anything was wrong with the documentation:
10:57 failed then passed on re-run, 11:26 passed, 12:04 failed, 12:07 failed again on re-run. The immediate consequence is that the expired-order tone fix merged in PR #11 and promoted in PR #12 is still not on the live site: the House Manual is serving the previous build.
The actual defect
tools/check-facts-live.mjsconflated two different events behind one exit code:Both exited 1. So a runner that could not open a socket blocked the publication of documentation that was correct, while the site kept serving an older build. That is strictly the worse of the two outcomes.
The safety argument for failing closed is weaker than it looks, because the live check is the second layer.
check-docs.mjsalready pins every fact against hardcoded values with no network at all, and it runs first in the Pages workflow and as a sibling job in CI. An unreachable endpoint does not leave the facts unguarded; it leaves them guarded by the pinned check alone, which is a reasonable posture for a transient network fault where blocking a publish is not.Evidence that it is the network, not the product
At 12:04, within the same minute, two GitHub-hosted runners ran this identical check against this identical endpoint:
GitHub Actions 1000002116(Docs CIfacts-live), 12:04:57 to 12:05:03: passedGitHub Actions 1000002111(Pagesbuild), 12:04:50 to 12:06:02: unreachable, 72 seconds, four attemptsThroughout, the endpoint answered
200from the workstation in about 160ms across eight consecutive probes. There is noAAAArecord, so it is not an IPv6 fallback, and no rate-limit headers come back. One hosted runner reaching it while another cannot, in the same minute, rules out both "the product is down" and "the host blocks GitHub wholesale".What changed
tools/check-facts-live.mjs: unreachable exits 75 (EX_TEMPFAIL), mismatch still exits 1. Both messages now say which one happened.::warningannotation and continue; 1 still stops the publish.tools/check-docs.mjsgains a publication-gate rule that keeps the two exit codes distinct and rejects either workflow running the check bare, so this cannot quietly collapse back into one failure.Deliberately not a wider retry budget. That trades a visible failure for a slower one and still fails whenever the path is out for longer than whatever number gets picked.
Test evidence
Both paths exercised locally:
The new gate rule was proved to bite by reverting the exit code by hand:
And with it restored:
Risk
A mismatch is still a hard failure and still blocks publication, which is the case this gate exists for. The behaviour change is narrow: a publish now proceeds when the product cannot be reached, carrying a warning that names exactly what was not checked and how to restore it.
Credit to the parallel session working in this repository, which had the 10:57 occurrence and the reasoning about which of the two failures is a reason not to publish.