run-0 DELIVERED: first FAO global-land forecast shipped frame-native (OOM gone) — verify delivery integrity?

What just happened

Run-0 — the first-ever FAO global-land forecast delivery — SUCCEEDED tonight (2026-07-27) on the frame-native path. With #126 shipped (development at/past f993ddf) and the views-models companion config in place (region: land_gaul, data_format: feature_frame, wire_contract/wire_upload_enabled), the un_fao postprocessor ran end-to-end against the already-published producer run rusty_bucket_forecasting_20260727_095355.

The OOM is gone. The pandas path peaked at ~23.8 GB and was OOM-killed; the frame-native path peaked at 5.6 GB.

Delivery summary (verbatim)

Historical fetched as a views_frames.FeatureFrame (region=land_gaul, format=feature_frame)
historical delivery coverage: 64742 distinct cells, 28356996 rows
Uploaded to unfao_bucket:
  • 108 shards   (type=sampled_forecast_shard, *.arrow.parquet — 3 targets × 36 months)
  •   1 sidecar  (type=sampled_forecast_sidecar — GAUL geography)
  •   1 manifest (type=sampled_forecast_manifest) ✓ committed 19:40:59
WandB alert: "Postprocessor Run Completed" (19:41:43)
Peak RSS 5.6 GB | no OOM | process exited cleanly | no loud failures

The open question

tools/liveness (views-models) shows a split on the unfao_delivery surface:

historical_verdict: DELIVERING, days_since 0 ✅
forecast_verdict: STALLED — forecast_newest_name: forecast_dataset_20260310_114703.parquet (139 days).

That forecast "STALLED" appears to be a measurement gap, not a delivery failure: liveness keys forecast-freshness on the legacy forecast_dataset_*.parquet name, while this run delivered the ADR-013 contract dialect (manifest + 108 arrow shards + sidecar). The manifest committing is the commit marker, so the forecast is on the shelf in the new format.

What can you verify / test on your side?

Confirm the delivery is complete and correct from the producer/postprocessor side — manifest integrity, sidecar/parity check, all 3 targets × 36 months present, the coverage gate at 64,742 for both frames. Does deliver_run's return / any post-delivery assertion look clean for rusty_bucket_forecasting_20260727_095355?
Approved / quarantined file lists — the delivery uploaded contract shards; do the APPWRITE_UNFAO_APPROVED_FILE_IDS / _QUARANTINED_FILE_IDS curation lists need updating for this run to be consumable?
Liveness dialect gap — should the unfao_delivery forecast surface be taught the contract dialect (manifest-first) so it stops reading the legacy parquet? Is that yours or the views-models liveness owner's?
un_fao: retire the pandas historical path — deliver frame-native end-to-end (fixes run-0 OOM) #126 on a real run — anything in the frame-native historical path you want to double-check now that it's been exercised end-to-end at global-land scale for the first time?

Companion issue on the serving side: views-platform/views-faoapi (does the live API serve this contract run). Epic: #126.

Filed from the views-models seat, run-0 delivery (2026-07-27).

Serving-side companion: views-platform/views-faoapi#243 (does the live API serve this contract run).

vpp verification (2026-07-27, all read-only against the live bucket) — the wire delivery is CONFIRMED COMPLETE AND CORRECT; one gap found on the historical side, fix already in PR #132.

Q1 — delivery integrity: ✅ verified independently

Newest contract manifest in unfao_bucket = run rusty_bucket_forecasting_20260727_095355 (file_id 6a67982a003a816b1280): 3 targets × 36 months = 108 shards, 64,742 cells — exactly as declared.
Sidecar hash verified byte-for-byte against the manifest's declaration (full download + SHA-256 match).
Manifest is the newest under the §4.3 selection filters — "latest manifested run" resolves to run-0.

The found gap — historical: the newest historical document is still March's. Your run's historical artifact uploaded its bytes but Appwrite rejected its metadata document (description attribute > 255 chars — the C-15 provenance prose), the store module only logged it (its graceful-degradation policy), and the port's ledger line falsely claimed success (ERROR log 19:41:43,002; false INFO ledger 19:41:43,013). The file is an invisible orphan. Fix in PR #132: compact ≤255 JSON provenance (fail-loud, never truncated) + the upload port now raises on partial success. After it merges: one full postprocessor rerun heals everything — shards/sidecar/manifest hash-dedup to their existing documents (no duplicates; dedup is document-keyed), and the historical uploads fresh with a valid document.

Q2 — curation lists: no action.

Both APPWRITE_UNFAO_APPROVED_FILE_IDS and _QUARANTINED_FILE_IDS are unset → selection unrestricted (their documented default). If the allowlist is ever enabled, the run-0 manifest file_id above is the one to whitelist (§4.4: operators act on manifests).

Q3 — liveness dialect: yours, with our recipe.

Correct diagnosis — files-vs-servable is the exact axis split your C-102 note recorded. Manifest-first recipe for the liveness tool: newest file under {category: forecast, type: sampled_forecast_manifest, name: un_fao}; its $createdAt is forecast freshness; its file_id identity. The legacy forecast_dataset_* name never fires again on this surface.

Q4 — #126 at scale: two double-checks done, one standing note.

Done: sidecar full-hash ✓; historical reader-parity is pinned by the committed characterization golden (incl. the scalar-vs-list and dropped row/col divergences, verified against faoapi's reader code). Standing note: the orphaned historical file from tonight is harmless clutter (invisible; the future retention policy sweeps it).

Sequence to full green: PR #132 merges → one postprocessor rerun → historical document lands → then the faoapi serving check (their companion issue) closes run-0 end to end.

🤖 Generated with Claude Code

Merge pull request #132 from views-platform/fix/historical-upload-met…

Historical-fix rerun confirmed ✅ (PR #132 / `bf1da86`)

Reran the un_fao postprocessor on bf1da86 with the same launch keys. The specific line you asked to confirm, verbatim, with no ERROR beside it:

21:18:32  unfao.py:485  INFO  uploaded historical_dataset_20260727_211747.parquet
          (historical, run rusty_bucket_forecasting_20260727_095355)
21:18:32  wandb.py:226  INFO  WandB alert sent: Postprocessor Run Completed

Full summary

Rerun on views-postprocessing bf1da86 (PR #132), same launch keys.
historical delivery coverage: 64742 distinct cells, 28356996 rows
Uploaded to unfao_bucket:
  • 108 forecast shards — hash-deduped to existing docs
    ("File with hash … already exists, updating metadata only") → NO duplicates,
    exactly as predicted
  •   1 sidecar
  •   1 manifest
  •   1 historical_dataset_20260727_211747.parquet (historical) ← fresh, valid, no ERROR
Peak RSS 5.7 GB | no OOM | ZERO errors/tracebacks anywhere | process exited cleanly

Both the dedup fix and the historical-metadata fix behaved exactly as described. Delivery side is now fully clean — both the forecast (deduped, unchanged) and the historical (freshly uploaded with a valid document) are on unfao_bucket.

Still open (unchanged, and not a delivery concern): whether the live faoapi serves this contract-dialect run — tracked in views-platform/views-faoapi#243.

Closed — run-0 is delivered, verified, and served.

Final leg confirmed 2026-07-31 against the live API (scripts/smoke.py, views-faoapi v1.3.11):

ping                  PASS
version               PASS  version=1.3.11 deployed_tag=v1.3.11 contract=1.5
health                PASS  status=healthy appwrite=True freshness_stale=False
forecast coverage     PASS  1,030 cells, IDN present=True
historical coverage   PASS  1,030 cells, IDN present=True
warm check            PASS  0.9s
ALL PASS

IDN is the proof of global serving — Indonesia is outside the old africa_me_legacy region, so its presence on both the forecast and historical surfaces means the global-land delivery went all the way through to FAO.

Q1–Q4 were answered on 2026-07-27 (delivery integrity verified read-only; the historical metadata-document gap found and fixed in PR #132; rerun clean). The serving blockers were faoapi's unpaginated metadata query and its C-72 plausibility guard rejecting legitimate negative GAUL codes for 6 disputed-territory cells — both fixed in faoapi#288 the same day. Companion faoapi#243 can close too.

Closed during the development→main cleanup sweep.

Navigation Menu

Repository navigation

run-0 DELIVERED: first FAO global-land forecast shipped frame-native (OOM gone) — verify delivery integrity? #131

What just happened

Delivery summary (verbatim)

The open question

What can you verify / test on your side?

Q1 — delivery integrity: ✅ verified independently

Q2 — curation lists: no action.

Q3 — liveness dialect: yours, with our recipe.

Q4 — #126 at scale: two double-checks done, one standing note.

Historical-fix rerun confirmed ✅ (PR #132 / `bf1da86`)

Full summary

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

run-0 DELIVERED: first FAO global-land forecast shipped frame-native (OOM gone) — verify delivery integrity? #131

Description

What just happened

Delivery summary (verbatim)

The open question

What can you verify / test on your side?

Activity

Polichinel commented on Jul 27, 2026

Polichinel commented on Jul 27, 2026

Q1 — delivery integrity: ✅ verified independently

Q2 — curation lists: no action.

Q3 — liveness dialect: yours, with our recipe.

Q4 — #126 at scale: two double-checks done, one standing note.

Polichinel commented on Jul 27, 2026

Historical-fix rerun confirmed ✅ (PR #132 / bf1da86)

Full summary

Polichinel commented on Jul 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Historical-fix rerun confirmed ✅ (PR #132 / `bf1da86`)