fix(#499): preflight passed with 6 of 9 publish variables missing, and its disk floor was below the one it delegates to

[Polichinel]

Follow-up to #524, from a /falsify pass on "we are ready for a 40-lesson forecasting run reaching faoapi". Both defects are mine, both are in --preflight, and both are the exact failure that mode exists to prevent.

1. Preflight checked 3 of 9 publish variables

PredictionStoreConfig.from_environment() requires nine, measured by running it:

APPWRITE_ENDPOINT, APPWRITE_DATASTORE_PROJECT_ID, APPWRITE_DATASTORE_API_KEY,
APPWRITE_PROD_FORECASTS_BUCKET_ID, APPWRITE_PROD_FORECASTS_BUCKET_NAME,
APPWRITE_PROD_FORECASTS_COLLECTION_ID, APPWRITE_PROD_FORECASTS_COLLECTION_NAME,
APPWRITE_METADATA_DATABASE_ID, APPWRITE_METADATA_DATABASE_NAME

Only the first three are secrets. A missing identifier fails the publish exactly as hard as a missing secret, so preflight would have reported ready with six of nine absent — and the run would have died at the publish, after the entire roster trained.

Adding the six names would not fix it. The extra can be absent, the endpoint unreachable, the key expired (#359 — 2026-11-17). Preflight now constructs the config and imports the SDK, which answers the question instead of a proxy for it. Suggested by the views-pipeline-core session, which noted this is what their #557 argues the manager should do itself; until it does, doing it by hand costs seconds.

Operator trap found by the same measurement: pipeline-core no longer auto-loads a .env from the working directory (#346, register C-177) — a library reading whatever .env its caller is standing in is what the Appwrite seam contract §3 forbids. A correct /root/.secrets beside the operator is not enough; the nine must be exported. Without that line the failure presents as wrong credentials rather than unexported ones. Preflight now prints set -a; . /root/.secrets; set +a.

2. The disk floor was below the floor it delegates to

pod_run_fao_delivery.sh demanded 60 GB for eight models plus the pool, while pod_run_model.sh — which it delegates to eight times — refuses below 40 GB for one ("one model needs ~20GB"). Preflight could report ready and the delegated script refuse partway through the roster, after hours.

Raised to DISK_FLOOR_GB=80. The retained component is an estimate and the comment says so: one model's calibration output is ~2.5 GB per predictions directory here, but no forecasting run has ever completed on this roster, so a forecast's retained size is unmeasured. A forecast has one origin against calibration's thirteen so it should be smaller — "should be" is doing real work there. Trigger to revise: the first completed run.

A guard now compares the two floors directly, because the defect was not the number — it was that nothing related them.

Context the audit turned up

The eight-model roster has never completed a forecasting run. ensembles/rusty_bucket/data/generated/forecasting_log.txt is dated 2026-07-20 and lists temporary_crane and temporary_fox at Deployment Status: shadow — a different roster. Every hop after training executes for the first time.

Separately, #499 B3's entity-coverage claim is wrong for these models and is corrected on that issue: ADR-064's check is never constructed on the PredictionFrame path (model.py:1653-6, forecasting/stage.py:79). Deliberate per ADR-042, and already recorded in ADR-064's own "deliberately not covered" section — but B3 asserted the opposite.

Verification

ruff clean. 8102 passed, 209 skipped, 5 xfailed. The publish-config check was exercised with credentials absent and with fakes present, confirming it fails for the right reason in each case.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GG2cY2HaqmUMpwUyR5V82K