Epic: Migrate forecast reconciliation into views-postprocessing (frames-native, parity-proven)

[Polichinel]

Origin: executes #3 ("Broaden scope: accept reconciliation from views-reporting") and the views-reporting side views-platform/views-reporting#72 ("Reconciliation is not reporting — migrate to views-postprocessing").

Authored as a context-rich, solution-open brief. Approaches are proposals, not prescriptions.

Why

Forecast reconciliation makes PRIO-GRID-month (pgm) predictions sum to their country-month (cm) totals — a post-forecast operation called by ensemble managers before any reporting/evaluation/delivery. It lives in views-reporting by accident (extracted there alongside visualisation code). That repo's own roadmap, risk register (C-24: "torch leaves with it"), and frames-adoption plan all say it should move here, and it is cleanly separable (zero ties to reporting/plotting/Appwrite).

Moving it here gives it the right roof (post-processing, not reporting), removes views-reporting's torch dependency (reconciliation is its only consumer), and makes the dependency direction clean (pipeline-core → views-postprocessing).

Where we are

Slice 1 is done (PR #30): the leaf algorithm reconcile_proportional was ported to pure numpy and proven bit-for-bit against the untouched views-reporting torch oracle (tests/test_reconciliation_parity.py, max abs diff 0.0). C-37 records that this is the pragmatic per-draw method (FPP3 top-down forecast-proportions), to be upgraded to principled probabilistic reconciliation after the migration is wired.

This epic is everything after the leaf: orchestration, country↔grid grouping, end-to-end parity, the cross-repo repoint, and the phase-out.

Desired end state

Reconciliation runs end-to-end in views-postprocessing, frames-native, with no torch, no pandas, no viewser, no wandb in the path; its output is proven to reproduce the frozen views-reporting pipeline on a realistic fixture; pipeline-core's ensemble managers call it here; and the views-reporting copy (plus its torch dep) is removed.

Scope

✅ In: orchestration (ReconciliationModule equivalent), country↔grid grouping via views_frames.cross_level_align, frames-native validation, the end-to-end parity gate, the pipeline-core repoint, and the views-reporting phase-out.

❌ Out (future, not this epic):

• Changing the algorithm. Migration is parity-preserving; the upgrade to principled probabilistic reconciliation (the IJF paper, C-37) is a separate follow-on epic, gated on this one completing.
• Editing views-reporting before phase-out. It stays frozen as the parity oracle until the final story.
• Reconciling the GAUL vs VIEWS country-id systems. For parity we inject the oracle's VIEWS country_id mapping; whether to later source it from our GAUL lookup (admin1_gaul0_code) is a separate decision (see the first story).

The one decision to retire first

The oracle groups grid cells by VIEWS country_id fetched from viewser (network; views_reporting/metadata/entity_metadata.py). Our GAUL lookup uses admin1_gaul0_code — a different id system. For parity we must group by the same VIEWS country_id, injected (the leaf never embeds geography — views-frames ADR-014). Story S0 captures the oracle fixture offline and records this decision.

Stories

Execution order (full checklist in the tracking issue #41):

1. Capture the offline parity oracle + decide the country↔grid id system #32 — Capture offline parity oracle + decide country↔grid id system · planning needs-decision
2. CM/PGM → PredictionFrame adapters #33 — CM/PGM → PredictionFrame adapters · implementation (after Capture the offline parity oracle + decide the country↔grid id system #32)
3. Country↔grid grouping core via cross_level_align #34 — Country↔grid grouping core via cross_level_align · implementation (after CM/PGM → PredictionFrame adapters #33)
4. Frames-native reconciliation validation guards #35 — Frames-native validation guards · implementation (after CM/PGM → PredictionFrame adapters #33)
5. ReconciliationModule orchestration (numpy-native; drop wandb + multiprocessing) #36 — ReconciliationModule orchestration; drop wandb/multiprocessing · implementation (after Country↔grid grouping core via cross_level_align #34, Frames-native reconciliation validation guards #35)
6. End-to-end reconciliation parity gate vs frozen views-reporting #37 — End-to-end parity gate · testing (after ReconciliationModule orchestration (numpy-native; drop wandb + multiprocessing) #36)
7. CIC for ReconciliationModule + migration docs #38 — CIC + migration docs · documentation (after ReconciliationModule orchestration (numpy-native; drop wandb + multiprocessing) #36)
8. Wiring: expose reconciliation API + pipeline-core ensemble repoint #39 — Wiring / pipeline-core repoint · implementation blocked (after End-to-end reconciliation parity gate vs frozen views-reporting #37)
9. Phase out reconciliation in views-reporting (closes views-reporting#72) #40 — Phase out reconciliation in views-reporting (closes views-reporting#72) · implementation blocked (after Wiring: expose reconciliation API + pipeline-core ensemble repoint #39)

Epic acceptance criteria

• A frames-native reconciliation module reproduces the frozen views-reporting pipeline on a realistic fixture (end-to-end parity), offline (CI needs no torch / views-reporting / viewser).
• No torch, pandas, viewser, or wandb in the reconciliation path here.
• A CIC documents the orchestration class (repo convention, ADR-006).
• pipeline-core ensemble managers call reconciliation in views-postprocessing.
• Reconciliation (and torch) are removed from views-reporting; views-reporting#72 closed.
• C-37 (principled-upgrade) remains open and is handed to its follow-on epic.

[Polichinel]

Reframe 2026-06-24 — two tracks, and FAO is reconciliation-independent.

Cross-repo investigation this session re-aims this epic:

• FAO (rusty_bucket, views-models#143) needs NO reconciliation — it's a pure-grid ensemble aggregated up to admin-1/country, so it sums by construction. This migration is not on the FAO critical path.
• The real near-term consumer is an upcoming UN-agency deliverable — an FAO-like grid ensemble that does reconcile against a CM model, with full pooled draws (~1024) → probabilistic reconciliation on the frames path.
• The reconciler here is already probabilistic-ready (reconcile_proportional + grouping fully vectorized over samples; bit-exact at S=1000). No new algorithm to write.

Remaining cross-repo work splits into two tracks:

• B1 (cleanup): repoint the legacy point consumers (pipeline-core ensemble.py / dataframe_ensemble.py) → here, then delete reconciliation + torch from views-reporting (Wiring: expose reconciliation API + pipeline-core ensemble repoint #39/Phase out reconciliation in views-reporting (closes views-reporting#72) #40 → views-reporting#72). Frees views-reporting of torch; kills the late-crash.
• B2 (new capability — the UN driver): add probabilistic reconciliation to pipeline-core's PredictionFrameEnsembleManager (no reconciliation hooks today). Gated on the C-37 sample-alignment decision (country point total vs aligned country draws); must honor C-38 chunk-by-time. Tracked in a new pipeline-core issue under their epic S1 (#182) — appwrite_env logs before it raises; ADR-008 pinned across both entry validators #193.

[Polichinel]

Reframe (Epic 11): views-postprocessing was a way-station; the reconciler's final home is views-frames. This epic moved reconciliation OFF views-reporting into vpp, parity-proven (PR #30) — but vpp is the wrong final home: FAO delivery doesn't use reconciliation, and it's a frame→frame operation that belongs in the frames foundation (CRP/SDP). It's now moving to a views_frames_reconcile sibling: views-platform/views-frames#131 (Epic 11). This epic is superseded by that and closes when vpp's copy is deleted (#62, gated on the views-frames v1.7.0 release + the views-models repoint views-platform/views-models#191). The in-repo slices remain valid history.

[Polichinel]

Closing the migrate-reconciliation-into-vpp epic. Outcome: vpp was only ever a way-station — reconciliation's final home is the views-frames foundation (views_frames_reconcile, Epic 11 / views-platform/views-frames#131, ADR-023), shipped v1.7.0. vpp's copy was parity-proven (PR #30) then retired in PR #63 (#62, merged 2026-06-26).

Remaining cross-repo cutover continues independently (not vpp work): pipeline-core#221 (collapse the DIP port → direct-import views_frames_reconcile) and views-reporting retirement (#40 / views-reporting#72). Tracked via register C-42.