fix(ci): a retry budget sized on the outage that was measured, not on a guess

[stephrobert]

The nightly conformance run of 2026-09-25 died installing the Scaleway CLI:

curl: (22) The requested URL returned error: 504

from a step that already said --retry 3 --retry-connrefused. Three jobs died
the same way on 2026-09-14.

curl does retry on a 504. The budget is seven seconds.

The reflex reading is that curl ignores HTTP errors. Measured against a local
server answering 504 forever, --retry 3 sends four requests — 1s, 2s, 4s —
and gives up after seven seconds. --retry-all-errors changes nothing: 504 is
already in curl's own list of transient errors.

Retrying was never the gap. How long it retried was.

How long an outage actually lasts here

One real measurement, taken 2026-09-14 while this family of failure was killing
three jobs at once: terraform-provider-outscale v1.8.0 answered 200 to five
requests out of fifteen for roughly twenty minutes, then fifteen out of
fifteen. A 67% failure rate, not a total one.

Against that rate, taking attempts as independent — which they are not when a
nearby cache is what broke, so this is a floor rather than a promise:

attempts	chance of failing anyway
4 (what every step had)	19.8%
8	3.9%
10	1.7%

19.8% is exactly what was observed: passing most nights, failing some.

A fixed interval, not a backoff

Exponential backoff spares a service struggling under load. What breaks here is
a CDN answering 504 to everyone, and waiting longer does not help it. At an
equal time budget a fixed interval buys far more attempts: ten cost 135s
spaced 15s apart, against 511s doubling.

What is not retried

A 404 is a pin naming an asset that does not exist. Retrying it for two minutes
turns a clear error into a slow one, and the log then says the same thing for two
different problems.

What falsify found in this work

Every test drove the script with the budget lowered through the environment so
it would run in seconds — and none of them touched the default. Setting
attempts=4, the exact value that failed, left all of them green. The defaults
now have a test of their own, asserting the calculation rather than the file.

Proven

• Four tests driving the real script: a server that fails then recovers, one
  that never recovers, one that answers 404, and one that must leave no file
  behind.
• A guard refusing a workflow that downloads with curl instead of the helper,
  so the next tool added does not inherit the seven seconds. 33 call sites moved.
• Four mutations, each compiling and each biting.
• The five falsify specs mutating the seven workflows this touches: green, and
  falsify:lint finds all 1309 fragments across 209 specs — no guard was
  disarmed by moving the call sites.
• mise run prepush green.

🤖 Generated with Claude Code