release: 1.4.0 to main

CHANGELOG.md

@@ -9,6 +9,64 @@ This file exists because the version number was the only signal a consumer got
 (register C-111). Releases before 1.2.0 are summarised from their tags rather than
 reconstructed in detail.
 
+## 1.4.0 — 2026-09-29
+
+**Fixes an unservable delivery.** The first live UN-FAO run, earlier today, uploaded
+109 of 110 objects, reported success, and was refused by views-faoapi at ingest. If you
+are pinned below this version, a delivery can still complete successfully and be
+unservable.
+
+### What was wrong
+
+The C-94 findability guard checked **two** things — the run manifest and the historical
+artifact — by querying the newest document per *category*. Every wire object carries
+`category="forecast"` and the manifest is uploaded last, so that query always returned
+the manifest. The GAUL sidecar and the 108 shards were never asked about.
+
+The sidecar's bytes were identical to the previous run's, the content-addressed store
+correctly declined a second copy, the metadata document was updated against the **old**
+file, and the upload returned a real file id for the wrong document. Nothing observed it.
+
+### What changed for a consumer
+
+The guard now asks two questions instead of one, and **refuses in four situations rather
+than two**. A delivery that previously completed can now stop:
+
+| | |
+|---|---|
+| *selection* | does the consumer's own query land on this run? — unchanged |
+| *per-object* | does **every** uploaded artefact resolve under its own filename? — new |
+
+Both raise the existing `DeliveryNotFindableError`; no new exception types. A failed
+*check* still reports `FindabilityUnverifiedError` rather than condemning the delivery.
+
+**If one of these fires after upgrading it is reporting a condition that was already
+wrong and already invisible.** The refusal names every object that does not resolve, and
+says so explicitly when nothing in the run is servable.
+
+### Why this is worth taking promptly
+
+Pooling upstream is deterministic, and every artefact's filename embeds the run id. So a
+**re-run** writes new filenames over identical bytes, and the deduplication path that
+took one object takes **all 110 at once** — the store creates the documents, the count is
+right, and nothing is servable. Re-running is the documented remedy for a torn run, which
+makes this the realistic case rather than the exotic one.
+
+### Also in this release
+
+- The store port gained a fifth method, `documents()`, and its documented duck-typing
+  contract now lists it. A datastore built to the previous docstring would have raised
+  `AttributeError` mid-delivery.
+- `deliver_run`'s summary carries the upload ledger out, as `uploaded_objects`.
+
+### Known, and not ours
+
+`views-pipeline-core`'s `get_latest_file_id` documents *"the newest matching file based on
+creation timestamp"* and takes the first element of an unsorted result. The *selection*
+half of the guard has relied on that since August and can in principle raise a false
+alarm. This release removes the equivalent assumption from the per-object half, which no
+longer depends on document order at all. The upstream half is filed in pipeline-core.
+
 ## 1.3.0 — 2026-09-19
 
 **No new failure modes.** A launcher that ran 1.2.0 sees nothing new stop. This release

docs/CICs/UNFAOPostProcessorManager.md

@@ -3,7 +3,7 @@
 
 **Status:** Active  
 **Owner:** PRIO MD&D Team  
-**Last reviewed:** 2026-08-26  
+**Last reviewed:** 2026-09-29  
 **Related ADRs:** ADR-001, ADR-002, ADR-008, ADR-009  
 
 ---
@@ -90,7 +90,7 @@ Assumptions that are not met **must cause failure**, not fallback behavior. The
 - **Null values in required metadata columns:** Raises `ValueError` with null count and affected column name (C-01 resolved — validation active)
 - **Dataset initialization failure:** Raises `ValueError` in `_save()` if datasets are None
 - **Appwrite upload failure:** Propagates exception from `DatastoreModule`. **Inside the wire leg** it is wrapped as `contract.wire.sink.TornRunError` (C-105, 2026-08-19), naming the run, how many of how many objects landed, and their names and file ids — the consumer cannot see a torn run (the manifest is the commit marker and never landed), the listed objects are **not** removed, and a re-run uploads all of them again under the same names. **The historical leg is not covered by that wrapper**: it uploads after the wire run is committed, so a failure there leaves a visible, complete forecast run alongside the *previous* run's historical artifact. Recorded as remaining scope in C-105
-- **Delivery invisible to the consumer:** raises `delivery.findability.DeliveryNotFindableError` (C-94, added 2026-08-18). After both legs are uploaded, the manager queries the partner store as the consumer does — `name == product.CONSUMER_DOCUMENT_NAME`, per category — and asserts the newest document it finds **is the one this run just uploaded**. Two distinct refusals: nothing found at all, and *found the previous run's* (*"the newest forecast document is X, but this run uploaded Y"*). The run-scoping is the whole guard — asking merely whether any document exists is a question the previous delivery already answers yes to, so the check could never fail from delivery 2 onward. This is the one failure mode where every upload reports success and the consumer still sees nothing; run-0's historical leg stranded exactly that way (C-79). It runs only inside the §11.4 interlock, and queries through a store with pipeline-core's automatic `name == model_name` filter suppressed, so it verifies the declared name rather than the views-models directory name that happens to match (C-77). **It does not detect a delivery that never ran, or stale data served from the consumer's cache** — both recorded as gaps in C-94
+- **Delivery invisible to the consumer:** raises `delivery.findability.DeliveryNotFindableError` (C-94, added 2026-08-18). After both legs are uploaded, the manager queries the partner store as the consumer does — `name == product.CONSUMER_DOCUMENT_NAME`, per category — and asserts the newest document it finds **is the one this run just uploaded**. **Four distinct refusals, in two questions.** The *selection* question — does the consumer's own query land on this run — refuses when nothing is found at all, and when it finds the previous run's (*"the newest forecast document is X, but this run uploaded Y"*). The *per-object* question, added 2026-09-29 by #312, refuses when no document carries an uploaded artefact's filename, and when one does but carries a different file id. **Every artefact is checked, not the two legs**: the first live delivery uploaded 109 of 110 and reported success, because the sidecar's bytes matched the previous run's, the store declined a duplicate, and `update_document` ran against the old file — returning a real id for the wrong document. The per-object half resolves the way views-faoapi does (type-scoped query, filename matched in Python) because `filename` carries no index; a direct filename query would report UNVERIFIED forever. The run-scoping is the whole guard — asking merely whether any document exists is a question the previous delivery already answers yes to, so the check could never fail from delivery 2 onward. This is the one failure mode where every upload reports success and the consumer still sees nothing; run-0's historical leg stranded exactly that way (C-79). It runs only inside the §11.4 interlock, and queries through a store with pipeline-core's automatic `name == model_name` filter suppressed, so it verifies the declared name rather than the views-models directory name that happens to match (C-77). **It does not detect a delivery that never ran, or stale data served from the consumer's cache** — both recorded as gaps in C-94
 - **Wrong forecast selected:** structurally impossible since #149. Selection is by **run manifest** — a commit marker whose contents are hash-verified — not by scanning the bucket for the newest `category="forecast"` upload. Declared identity is additionally checked **per shard header** against the launched ensemble inside `TargetLease.load()` (`contract/wire/source_selection.py:73-81`), so identity comes from the artifact's own content. The metadata-field check this bullet used to describe (`delivery/identity.py`) was retired in #150 and the legacy reader it served in #149; register C-25 is closed as *superseded by mechanism* <!-- legacy-ok: retirement record -->
 - **Launch config incomplete:** raises `LaunchConfigError` naming the missing key. A launcher that omits `wire_contract` or declares a `data_format` other than `feature_frame` is **refused**, never quietly routed into a fallback (ADR-003, register C-63)
 - **Region coverage mismatch:** Raises `CoverageError` in `_check_coverage()` (called from `_validate()`) if a pinned region's delivered cell count is wrong (S1/C-34) or a GAUL-uncovered excluded cell leaks into the delivery (S4/C-30)

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "views-postprocessing"
-version = "1.3.0"
+version = "1.4.0"
 description = ""
 authors = [
     "Dylan Pinheiro <dylpin@prio.org>",