Promote: an unreachable product endpoint is not documentation drift - #14
Merged
Merged
Conversation
The House Manual failed to publish twice this afternoon and once this morning, all three times on the same step, none of them because anything was wrong with the documentation: live check: https://forkedfelines.art/api/v1/product unreachable after 4 attempts: TypeError: fetch failed The check conflated two different events behind one exit code. A mismatch means the product moved and the documentation is now wrong, which must never publish. Unreachable means a runner could not open a socket, which says nothing about the facts in either direction. Treating them alike meant a network fault blocked the publication of documentation that was correct, while the site went on serving an older build. That is strictly the worse of the two outcomes, and it is what happened: the expired-order tone fix sat unpublished while the previous wording stayed live. Unreachable now exits 75, EX_TEMPFAIL, and both workflows turn that into a warning annotation and continue. A mismatch still exits 1 and still stops the publish. The facts are not unguarded while the endpoint is away. check-docs.mjs pins every one of them offline against hardcoded values and runs first in the Pages workflow and as a sibling job in CI, so an unreachable product leaves them guarded by the pinned check alone, which is a reasonable posture for a transient network fault where the old one was not. Deliberately not a wider retry budget: that trades a visible failure for a slower one and still fails whenever the path is out for longer than whatever number gets picked. A new publication-gate rule keeps the two exit codes distinct and rejects either workflow running the check bare, so this cannot quietly collapse back into one failure. Evidence that it is the network and not the product: at 12:04 two GitHub-hosted runners ran this identical check against this identical endpoint within the same minute. One reached it and passed. The other could not, for 72 seconds across four attempts. The endpoint answered 200 from the workstation throughout, in about 160ms, with no AAAA record and no rate-limit headers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of the first commit found a fail-open I had introduced. Three
conditions landed in one catch and left through one exit code, and after
moving that exit code to 75 they all published with a warning:
fetch threw, no response genuinely transient
response answered, !response.ok it ANSWERED
response.json() threw it ANSWERED
The second is the one that matters. If /api/v1/product is renamed, moved,
or version-bumped out from under the documentation, the endpoint answers
404, and the previous commit would have reported that as UNREACHABLE and
published the House Manual against facts nobody compared. That is exactly
the event this gate exists to catch, arriving in a network costume, and
it was worse than before my change rather than merely unfixed: a 404 used
to exit 1 and block.
The decision now reads what the last attempt observed:
no response 75 nothing was learned
5xx 75 it answered and is having a bad time, which is
also what a deploy window looks like
429 75 rate limited, not moved
any other 4xx 1 the contract is not where the docs say it is
2xx unparseable 1 it answered and it is not a contract
The last attempt wins rather than the worst one, because a 404 followed
by three dropped connections is better described as losing the path than
as the contract moving.
All five paths were exercised against a real local server rather than
asserted. The first harness reported two false failures because
spawnSync blocks the event loop, so the in-process server could never
answer and every case looked unreachable; re-run out-of-process, 404 and
the unparseable body exit 1, and 500, 429 and a refused connection exit
75.
The gate rule is tightened to match. Asserting that EX_TEMPFAIL exists
says nothing about which conditions reach it, so it now pins the
transient set and rejects any widening to 4xx, which was proved to bite
by widening it by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review found the one sequence where last-attempt-wins resolves the wrong way: attempt 1 404 the contract moved attempts 2-4 refused the path drops last wins -> 75 -> publishes That is the gate's own case arriving and then being masked by noise. Unlikely, and the only sequence where the ordering rule changes the outcome at all. Decided on the asymmetry rather than on likelihood. Wrongly exiting 1 costs a blocked publish: visible, recoverable, one re-run. Wrongly exiting 75 costs a silent publish against a contract nobody compared, which is precisely what this gate exists to prevent. Where the two error costs are that lopsided, be wrong in the direction someone notices. The change is one clause rather than worst-wins wholesale. A 4xx that is not 429 is sticky once seen; everything else keeps the last-attempt rule and the existing boundaries. Ordering never mattered among no-response, 5xx and 429 anyway, since all three answer 75 regardless. The counter-argument is recorded in the source because it is the thing that would change this: if some intermediary answered 404 while transiently failing, a healthy deploy would block a publish for no reason. Nothing in this path is known to do that. A container swap surfaces as 502 or 503, and a 404 from this endpoint means the route is not registered, which is the persistent condition rather than the transient one. Exercised against a server that answers 404 once and then stops listening, which is the exact sequence: exit 1, "answered 404 on at least one of 4 attempts". The gate rule pins the stickiness and was proved to bite by renaming the variable that carries it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ble-is-not-drift fix(ci): an unreachable product endpoint is not documentation drift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes the verified head of
developso GitHub Pages publishes it, and so the Pages workflow itself stops failing on a network fault.Carries PR #13 and, behind it, the still-unpublished expired-order tone fix from PR #11 and #12. The House Manual has been serving a build from before that fix because the publish step kept failing on an unrelated network condition.
Why the publish was failing.
tools/check-facts-live.mjsconflated two events behind one exit code: the endpoint answering with facts that disagree (documentation drift, must never publish) and the endpoint not being reachable at all (says nothing about the facts). A runner that could not open a socket therefore blocked publication of documentation that was correct, while the site went on serving an older build.Unreachable now exits 75 and publishes with a warning annotation. A mismatch still exits 1 and still blocks. A 4xx is treated as a mismatch, not as unreachable, and stays counted even if the path drops afterwards: if the contract moves, the endpoint answers, and that is the event this gate exists to catch.
The facts are not unguarded meanwhile.
check-docs.mjspins all eleven values against hardcoded literals with no network, and runs first in the Pages workflow and as a sibling job in CI.Evidence it is the network, not the product. At 12:04, within one minute, two GitHub-hosted runners ran this identical check against this identical endpoint. One passed in six seconds; the other could not reach it across four attempts over 72 seconds. The endpoint answered 200 from the workstation throughout, in about 160ms, with no AAAA record and no rate-limit headers.
Every exit path was exercised against a real local server rather than asserted: 404 and a 2xx with an unparseable body exit 1; 500, 429 and a refused connection exit 75; and a 404 followed by a dead server still exits 1.