Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/ADRs/013_sampled_forecast_wire_contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -1116,6 +1116,34 @@ record execution progress against it.
and the provenance keyset closed (PR #136). **P3/#138** — §7(d) added: this ADR
now cross-references PLATFORM-001 by URL, never by copy.

- **2026-07-31 — the Hop-A LEGACY READER IS RETIRED; §11.4's transition is over on
this side (#149, epic #148).** §11.4 required a type-aware guard live at each
store **before the first contract artifact was uploaded there**. Both guards
merged 2026-07-15 and run-0 uploaded 2026-07-27, so the sequencing constraint was
satisfied. The legacy reader it protected has now been **deleted**: this repo
reads forecasts only through the contract path (manifest selection,
`unfao/wire/source_selection.py`) and historical actuals only as a
`views_frames.FeatureFrame` (#126).

**The retired guard, recorded here because the code that carried it is gone.**
The legacy reader selected `{"category": "forecast", "type": "ensemble"}` —
pinning the legacy `type` was what made pipeline-core#269's contract uploads
(`sampled_forecast_shard` / `_manifest` / `_sidecar`) unselectable by it. That
disjointness is the fact §11.4 turned on, and it held for the whole transition.

**Consequence for §11.4:** its Hop-A clause is now historical. **Its Hop-B clause
still binds** — views-faoapi's deployed selector is a separate deployment on its
own schedule (views-faoapi C-161). Nothing in this entry relaxes the consumer
side.

**The delivery mode is now DECLARED, not inferred.** Previously
`configs.get("wire_contract")` returning `None` because a launcher never mentioned
the key was indistinguishable from a deliberate `False`, and either quietly
selected the retired path. A launcher that does not declare `wire_contract: True`
and `data_format: feature_frame` is now **refused by name**
(`unfao/launch_config.py`) — ADR-003 applied to the launch config, and register
C-63 closed.

---

## Appendix A — Version history (review archaeology; non-normative)
Expand Down
24 changes: 22 additions & 2 deletions reports/technical_risk_register.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
| Project | views-postprocessing |
| Owner | Dylan Pinheiro / PRIO MD&D Team |
| Last Updated | 2026-07-31 |
| Total Concerns | 70 |
| Open Concerns | 33 |
| Total Concerns | 71 |
| Open Concerns | 34 |
| Resolved Concerns | 37 |

---
Expand Down Expand Up @@ -812,6 +812,26 @@ Cross-refs: **C-69** (the packaging split this rides along with), **C-59**/**C-6

---

### C-71: `appwrite_env.assert_env_declared` raises without logging — ADR-008 non-compliance in an entry-validation seam

| Field | Value |
|-------|-------|
| ID | C-71 |
| Tier | 4 — the raise is loud and its message is fully diagnostic, so nothing is silently swallowed today. What is missing is the persistent record ADR-008 requires, in the one seam whose whole job is to make a misconfigured launch visible. |
| Source | `review-diff` (2026-07-31) — S1/#149 review; found by mirroring this module when writing its sibling |
| Trigger | When a delivery run refuses on a missing Appwrite variable and the operator goes looking for *why* in the logs rather than the traceback — or when the next entry-validation module is written against this one as the pattern, as `unfao/launch_config.py` was |
| Location | `views_postprocessing/unfao/appwrite_env.py:44-51` (`assert_env_declared`) |

**ADR-008:48** requires that *"raised structural failures must be logged at `ERROR` level or higher"*, and **:51** that *"raising is not a substitute for logging."* `assert_env_declared` raises `EnvironmentError` naming every missing variable but never logs. A launcher misconfiguration is a structural failure by any reading of that ADR.

**How it was found, and why that matters:** `unfao/launch_config.py` (S1) was deliberately written to mirror this module — same shape, same failure style, same dependency-light constraint. It **inherited the flaw**, and the S1 diff review caught it in the new code. The new module was fixed to log-before-raise; this one was left alone for scope discipline, which means the pair is now **inconsistent** — the sibling written to match it no longer does.

Registered rather than fixed in #149 because it is pre-existing (shipped in þing-01 P1 / #134) and outside that story's boundary. It is a two-line change and the natural place to take it is **S8** (#156, epic closeout) or any PR that next touches `appwrite_env.py`.

Cross-refs: **C-19** (RESOLVED — the ADR-008 log-before-raise sweep whose convention this predates), **C-63** (the declaration-over-inference concern S1 closed), ADR-008, #134, #149, #156.

---

## Disagreements

### D-12: Post-Run-0 infrastructure & naming intents — repo rename, internal-store transport, compute co-location
Expand Down
87 changes: 0 additions & 87 deletions tests/test_append_metadata.py

This file was deleted.

5 changes: 4 additions & 1 deletion tests/test_env_declaration.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,10 @@ def test_declared_names_match_the_manager_reads():
appwrite_env.CONNECTION_ENV + appwrite_env.PROD_FORECASTS_ENV + appwrite_env.UNFAO_ENV
)
assert read_names == declared
assert text.count("appwrite_env.assert_env_declared(") == 3 # both stores + legacy save
# One validation per store construction. Was 3 until #149 retired the legacy
# `_save`, whose Appwrite config duplicated `_unfao_appwrite_config` verbatim;
# the two survivors are the production_forecasts read and the unfao_bucket write.
assert text.count("appwrite_env.assert_env_declared(") == 2


def test_secret_env_names_follow_the_platform_naming_rule():
Expand Down
2 changes: 1 addition & 1 deletion tests/test_falsification_fao_notes_relevance.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def test_falsify_fao_02_schema_country_identifier_m49_vs_iso_a3():
Probe 2 (Category H): FAO contract specifies UN M49; postprocessor uses ISO A3

Finding: Release Note 01 Topic C states "PRIO/VIEWS will use UN M49
as the standard country identifier." The postprocessor's _append_metadata
as the standard country identifier." The postprocessor's geography attachment
enriches with 'country_iso_a3' (ISO Alpha-3). If the postprocessor output
feeds into the API that FAO accesses, there is a coding system mismatch.

Expand Down
25 changes: 15 additions & 10 deletions tests/test_input_integrity_e2e.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,21 @@
story's primitives unit tests.

``UNFAOPostProcessorManager`` cannot be instantiated here (it needs views-pipeline-core +
Appwrite env — register C-40), so — exactly as ``test_append_metadata.py`` and
``test_validation.py`` do — each test **replicates the manager's own extraction→invariant
chain** on a synthetic delivery frame and proves the guard fires. Keep these in lockstep
with ``views_postprocessing/unfao/managers/unfao.py``:

S1 coverage _check_coverage cells_of -> assert_complete_coverage
S2 observed range _clip_observed_history months_of -> fabricated_months -> drop_months_above
S3 forecast identity _read_forecast_data file_metadata -> assert_forecast_identity
S4 land_gaul _check_coverage cells_of -> assert_no_excluded_cells
S5 provenance _delivery_description build_provenance(<- cells_of/unmapped_cell_count)
Appwrite env — register C-40), so — exactly as ``test_validation.py`` does — each test
**replicates the delivery's extraction→invariant chain** on a synthetic frame and proves
the guard fires.

What these prove is the **invariants in** ``views_postprocessing/delivery/``, which is
why they survived #149 untouched: the rules are representation-free, so retiring the
pandas delivery changed which module *calls* them, not whether they hold. The mapping to
manager methods below is therefore informational, and updated for the post-#149 tree:

S1 coverage _check_coverage cells_of -> assert_complete_coverage
S2 observed range _read_historical_frame months_of -> fabricated_months -> drop_months_above
S3 forecast identity (RETIRED with the legacy reader, #149 — selection is by run
manifest; `delivery/identity.py` itself is retired in #150)
S4 land_gaul _check_coverage cells_of -> assert_no_excluded_cells
S5 provenance _historical_frame_description build_provenance(<- cells_of/unmapped_cell_count)
"""

import json
Expand Down
124 changes: 124 additions & 0 deletions tests/test_launch_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
"""The launch declarations are asserted, not inferred (S1 / #149; register C-63).

Until #149 this package inferred its delivery mode from the *absence* of config
keys: `configs.get("wire_contract")` returning `None` because a launcher never
mentioned it was indistinguishable from a deliberate `False`, and either quietly
selected a pandas path kept "until run 0 proves the contract path live" and never
removed. Two independent axes gave four delivery modes; production used one; the
other three were reachable only by omission.

These tests pin the replacement: **one path, and a refusal that names what the
launcher left out.** They are the mechanical form of ADR-003 (authority of
declarations over inference) applied to the launch config, mirroring
`test_env_declaration.py`, which does the same job for the environment.

`launch_config` is dependency-light by design — no pipeline-core, no pandas, no
frames — so these run anywhere.
"""

import pytest

from views_postprocessing.unfao import launch_config
from views_postprocessing.unfao.launch_config import LaunchConfigError


# ── the forecast axis: wire_contract ────────────────────────────────────────


def test_missing_wire_contract_raises_naming_the_key():
"""The key's absence must be an error, not a silent route to a retired path."""
with pytest.raises(LaunchConfigError, match="wire_contract"):
launch_config.assert_contract_mode({})


def test_wire_contract_false_raises_too():
"""An explicit False is not a request for the old path — it no longer exists."""
with pytest.raises(LaunchConfigError, match="wire_contract"):
launch_config.assert_contract_mode({"wire_contract": False})


def test_the_refusal_says_there_is_no_fallback():
"""A launcher author must not read the error as 'set it False instead'."""
with pytest.raises(LaunchConfigError) as excinfo:
launch_config.assert_contract_mode({})
message = str(excinfo.value)
assert "no fallback" in message
assert "#149" in message # where the retired path went, for the reader


def test_the_refusal_is_logged_before_it_is_raised(caplog):
"""ADR-008: structural failures are logged persistently AND raised.

Not belt-and-braces — a refused launch must leave a record even if the raise is
swallowed further up the pipeline framework, because the message is the only
diagnostic a launcher author gets.
"""
with caplog.at_level("ERROR"):
with pytest.raises(LaunchConfigError):
launch_config.assert_contract_mode({})
assert any(r.levelname == "ERROR" for r in caplog.records), (
"the refusal was raised without being logged (ADR-008)"
)
assert "wire_contract" in caplog.text


def test_the_format_refusal_is_logged_too(caplog):
with caplog.at_level("ERROR"):
with pytest.raises(LaunchConfigError):
launch_config.assert_frame_native_historical("dataframe")
assert any(r.levelname == "ERROR" for r in caplog.records)


def test_declared_contract_mode_passes():
launch_config.assert_contract_mode({"wire_contract": True})


def test_other_keys_are_not_consulted():
"""Only the declared key decides. No inference from neighbouring config."""
with pytest.raises(LaunchConfigError):
launch_config.assert_contract_mode(
{"wire_upload_enabled": True, "region": "land_gaul", "ensemble": "rusty_bucket"}
)


# ── the historical axis: data_format ────────────────────────────────────────


def test_missing_data_format_raises_naming_the_expected_value():
with pytest.raises(LaunchConfigError, match="feature_frame"):
launch_config.assert_frame_native_historical(None)


def test_a_different_data_format_raises_and_reports_what_it_saw():
"""The error must name the offending value — a queryset author needs to find it."""
with pytest.raises(LaunchConfigError) as excinfo:
launch_config.assert_frame_native_historical("dataframe")
assert "'dataframe'" in str(excinfo.value)


def test_declared_feature_frame_passes():
launch_config.assert_frame_native_historical("feature_frame")


# ── the manager actually calls them ─────────────────────────────────────────


def test_the_manager_asserts_both_axes():
"""Source-scan: the manager pulls the full pipeline-core framework and cannot be
imported in every test environment (the repo's standing pattern — see
`test_env_declaration.py`). What matters is that neither axis is left to
`.get()` with a silent default."""
from pathlib import Path

source = (
Path(__file__).resolve().parent.parent
/ "views_postprocessing" / "unfao" / "managers" / "unfao.py"
).read_text()

assert "launch_config.assert_contract_mode(" in source
assert "launch_config.assert_frame_native_historical(" in source
# And the inference it replaced must not survive anywhere.
assert 'configs.get("wire_contract")' not in source, (
"the manager still branches on wire_contract — S1 replaced that fork with a "
"single refusal (C-63)."
)
Loading
Loading