Repository navigation
[Tracking] Push pandas to the seams — execution checklist #93
Description
Tracking issue for the Push pandas to the seams epic (#85). Pick off one site at a time, each its own PR with a parity proof; check off as merged.
Execution order
- S1 S1 — frames.py: generalize to_prediction_frame to S>1 (pandas→frames) #86 —
frames.py→ S>1PredictionFrame[UNILATERAL](keystone; do first) - S2 S2 — extraction.py: frame-native sibling readers #87 —
extraction.pyframe-native sibling readers[UNILATERAL](dep S1) - S3 S3 — forecast convert-at-the-door (internal PredictionFrame) #88 — forecast "convert at the door"
[UNILATERAL](dep S1, S2) - S4 S4 — enrichment.py: numpy/pyarrow keyed gather (drop pandas merge) #89 —
enrichment.pynumpy/pyarrow gather[UNILATERAL, low pri](independent) - S5 S5 — build_gaul_lookup.py: pyarrow-native build #90 —
build_gaul_lookup.pypyarrow-native[UNILATERAL, lowest pri](independent) - S6 S6 — forecast outbound → arrow sample-frame wire (ships (N,S) to FAO) #91 — forecast outbound → arrow sample-frame
[GATED: faoapi #45 + C-40](dep S3) - S7 S7 — historical inbound read_dataframe/PGMDataset retirement #92 — historical inbound retirement
[GATED: pipeline-core C-40](do not move yet) - Close epic [EPIC] Push pandas to the seams: views-frames migration for sample-bearing predictions #85
Sequence
S1 ──► S2 ──► S3 ──► S6 (gated: faoapi #45)
S4, S5 (independent, low/lowest priority)
S7 (gated: pipeline-core C-40) — never blocks samples
Notes
- Unilateral arc = S1→S2→S3. Landing these makes vpp's interior carry
(N,S)with byte-identical S=1 deliveries — the samples-ready milestone behind frozen wires. - S6 ships samples to FAO but needs the coordinated faoapi wire change (FAO delivery path is point/DataFrame-based — won't carry rusty_bucket's pooled draws #45). S7 is pipeline-core-gated (C-40) and never blocks the sample work.
- Parity pattern for every step:
tests/test_views_frames_conformance.py(build pandas → convert → assert arrays equal) + head-to-head old-vs-new fixtures. - Each story = its own PR →
development, usual ritual (ruff/tests/CI; manual issue close ondevelopmentmerges). - Cross-refs: register C-40, FAO delivery path is point/DataFrame-based — won't carry rusty_bucket's pooled draws #45.
Activity
docs(risk-register): cross-link C-40 to the pandas-migration epic (#85)
Polichinel commentedon Jul 19, 2026
Checklist update: S3 (#88) and S6 (#91) are superseded by epic #105 (ADR-013 Hop-B sink leg — contract-mode inbound #111 and the full sink #112 carry their scope). S4 (#89), S5 (#90), S7 (#92) unchanged. The pandas-to-the-seams end state for the forecast path is now delivered through #105's fixture-proven wire.
🤖 Generated with Claude Code
Polichinel commentedon Jul 20, 2026
Epic #105 is complete (2026-07-20): the (N,S) forecast path is frame-native end to end behind the declared
wire_contractmode — the S3/S6 goals this checklist tracked are delivered (via #111/#112). Remaining here: S4 (#89), S5 (#90) — independent pandas removals — and S7 (#92, pipeline-core-gated).🤖 Generated with Claude Code
Polichinel commentedon Jul 27, 2026
Checklist update (2026-07-27): S7 (#92) delivered via #126/PR #129 (frame-native historical — the C-40 gate lifted; first production consumer of pipeline-core's frame path). S4 (#89) reduced to the legacy-forecast enrichment only (likely moot post-run-0). The pandas-to-the-seams end state is now reality on every live path except the legacy forecast branch, whose deletion is the named post-run-0 follow-up.
🤖 Generated with Claude Code
Polichinel commentedon Aug 4, 2026
Checklist complete — closing alongside epic #85.
Every story merged: S1–S7 across #86–#92, and S5 (#90, the arrow-native lookup builder) in PR #220. The residual this tracker carried in its comments — "delete the legacy forecast branch post-run-0" — was discharged in #149; the manager has carried no legacy path since, and register C-63 is resolved.
The final step was #221: retiring
GaulLookupEnricher, which held the package's last pandas reference.grepfor a pandas import acrossviews_postprocessing/andscripts/now returns nothing, and a test pins it.