Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/CICs/UNFAOPostProcessorManager.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,17 @@ Assumptions that are not met **must cause failure**, not fallback behavior.
- **Null values in required metadata columns:** Raises `ValueError` with null count and affected column name (C-01 resolved — validation active)
- **Dataset initialization failure:** Raises `ValueError` in `_save()` if datasets are None
- **Appwrite upload failure:** Propagates exception from `DatastoreModule`
- **Wrong forecast file selected:** Raises `ForecastIdentityError` in `_read_forecast_data()` if the newest `category="forecast"` file's identity (name/loa) does not match the configured ensemble (S3/C-25 — a stray upload cannot be silently shipped)
- **Region coverage mismatch:** Raises `CoverageError` in `_check_coverage()` (called from `_validate()`) if a pinned region's delivered cell count is wrong (S1/C-34) or a GAUL-uncovered excluded cell leaks into the delivery (S4/C-30)
- **Fabricated historical tail:** `_clip_observed_history()` drops months beyond the producer's `last_valid_month_id` so unobserved zero-padding is not shipped as observed history (S2/C-26); **degrades open** (skips the clip with a WARNING) if the boundary cannot be resolved
- **Upload provenance:** every upload's `description` carries structured provenance (lookup version, region, expected/actual cell counts, unmapped count) via `_delivery_description()` (S5/C-15)

The following **must never** fail silently:
- Missing or None environment variables for Appwrite
- Network failures during download or upload
- Schema validation failures (missing columns or null values)
- A forecast file whose identity does not match the configured ensemble (S3/C-25)
- Wrong region coverage or a leaked GAUL-uncovered cell (S1/C-34, S4/C-30)

---

Expand Down Expand Up @@ -140,6 +146,8 @@ manager._save()

Currently: the manager cannot be instantiated without `views-pipeline-core`, so its stage logic is covered by **replica tests** that mirror the real methods — `tests/test_validation.py` (`_validate`) and `tests/test_append_metadata.py` (`_append_metadata`). A full end-to-end test against the live manager (C-03) still requires a production-like environment.

The input-integrity guards (S0–S6, epic #51) are representation-free invariants in `views_postprocessing/delivery/` that the manager **calls** (never inherits). Each has primitives unit tests — `tests/test_delivery_coverage.py` (S1/S4), `tests/test_delivery_observed_range.py` (S2), `tests/test_identity.py` (S3), `tests/test_provenance.py` (S5), `tests/test_extraction.py` (the seam) — and `tests/test_input_integrity_e2e.py` replicates the manager's extract→invariant chain end-to-end. The design contract (representation-free, called-not-inherited) is pinned by `tests/test_input_integrity_design_contract.py`.

---

## 11. Evolution Notes
Expand Down
45 changes: 45 additions & 0 deletions docs/fao_excluded_cells.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Cells excluded from FAO global delivery

**Status:** disclosure note for the `land_gaul` global rollout (register C-30 / D-10, sprint S4 / #55).
**Source of truth:** views-datafactory v1.4.0 — `land_gaul` curated region (`land − land_gaul`).
**Pinned in code:** `views_postprocessing/delivery/coverage.py` (`EXCLUDED_GIDS_BY_REGION["land_gaul"]`).

## What FAO should know

The VIEWS global historical delivery covers the `land_gaul` region: **64,742** PRIO-GRID
cells — every land cell for which FAO's **GAUL 2024** boundaries provide an administrative
assignment.

**76 land cells are deliberately excluded.** They are remote sub-Antarctic islands that
fall outside GAUL 2024 coverage (e.g. Macquarie Island, the Auckland Islands, Prince
Edward Islands, South Sandwich Islands). They have **no** country/admin assignment in any
source, so they are dropped at the region level rather than shipped with a placeholder —
shipping them would attribute partner rows to a non-country (`-1`). Conflict activity in
these cells is negligible to nil.

This is the full, frozen exclusion list. If a future delivery's coverage changes, the
frozen list in code diverges loudly (it is asserted in `tests/test_delivery_coverage.py`
against the producer), so any drift is caught before delivery rather than silently
absorbed.

## History

The exclusion count was **82** when `land_gaul` was first curated (datafactory #159).
It dropped to **76** when datafactory #163 (ADR-043) supplemented **6 Azorean cells**
(gids 182470, 183190, 183909, 183910, 186058, 186778) into `land_gaul` — those are now
**covered and delivered**, not excluded.

## The 76 excluded gids

```
51078 51798 53979 54699 56852 56853 58318 62356
94776 99027 107733 107742 110367 112944 114769 116931
118753 121625 123748 124038 124425 124759 126561 126624
128012 129079 129387 129574 130919 131639 132525 146125
153966 157829 159987 160708 173423 179900 190724 201850
202249 203404 206592 208007 210596 212353 212774 221061
223941 225390 227961 229161 229429 229430 229866 233436
233743 234640 235586 235942 235943 235951 235954 236054
237351 238123 238231 238954 239705 240318 240319 240975
240977 241153 247862 248595
```
Loading
Loading