Skip to content

S7 — historical inbound read_dataframe/PGMDataset retirement #92

Closed

Description

@Polichinel

Epic: #85 · S7 · [CROSS-REPO-GATED] · blocked · gate: pipeline-core C-40 · DO NOT MOVE YET

Background

The historical inbound is forced pandas: read_dataframe → PGMDataset (unfao.py:58-64), plus the pandas ops in _append_metadata (.reset_index/.set_index/.join, :142-149) and _validate (.isnull().sum(), :165,177). These are inherited from pipeline-core's PostprocessorManager base, whose _read/_transform/_validate contract is defined in terms of PGMDataset (pandas with object-dtype cells).

Why this is tracked but NOT actioned now

Work (eventual, post-gate)

  • Replace read_dataframe/PGMDataset with frame-native loaders; convert _append_metadata/_validate pandas ops to numpy/frame operations.

Acceptance criteria (eventual)

  • Historical path holds a frame, not a PGMDataset; _append_metadata/_validate are pandas-free.
  • Parity: identical delivered output to today.

Dependencies / gate

Blocked on pipeline-core C-40 (#186/#207/#161/#159/#211). Independent of S1–S6.

Files

views_postprocessing/unfao/managers/unfao.py (_read_historical_data, _append_metadata, _validate).

Activity

  1. added
    storyA single reviewable unit of an epic
    blockedBlocked on a dependency or decision
    on Jun 28, 2026
  2. Polichinel commented on Jul 27, 2026

    @Polichinel
    CollaboratorAuthor

    Delivered by #126 / PR #129 (2026-07-27) — closing as superseded-and-shipped. The historical inbound is frame-native: _read_historical_data dispatches on declared_data_format, fetches a views_frames.FeatureFrame via pipeline-core's get_feature_frame (this repo is that path's first production consumer — the C-40 gate this issue waited on has lifted), clips frame-natively, and builds the artifact pandas-free with reader-level parity proven against a legacy characterization golden. read_dataframe/PGMDataset remain only on the legacy-descriptor branch, which retires with the legacy forecast path post-run-0.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on a dependency or decisionimplementationCode implementation workstoryA single reviewable unit of an epic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions