Skip to content

Add extend_single_year_dataset for fast dataset year projection#7700

Draft
anth-volk wants to merge 1 commit intomainfrom
add-extend-single-year-dataset
Draft

Add extend_single_year_dataset for fast dataset year projection#7700
anth-volk wants to merge 1 commit intomainfrom
add-extend-single-year-dataset

Conversation

@anth-volk
Copy link
Contributor

Fixes #7699

Summary

  • Adds USSingleYearDataset and USMultiYearDataset schema classes (modeled after UK equivalents) for entity-level dataset handling
  • Adds extend_single_year_dataset() with multiplicative uprating derived from system.variables[var].uprating at runtime — no separate uprating list to maintain
  • Adds dual-path loading in Microsimulation.__init__: auto-detects entity-level HDFStore files, loads them as USSingleYearDataset, extends via uprating, and passes the multi-year result to policyengine-core
  • Legacy h5py variable-centric files continue to work via the existing code path

Depends on

Test plan

  • Load an HDFStore file: Microsimulation(dataset="path/to/STATE.hdfstore.h5") — verify it loads and extends correctly
  • Load a legacy h5py file: Microsimulation(dataset="path/to/STATE.h5") — verify existing path still works
  • Verify uprated variables (e.g. employment_income) grow year-over-year
  • Verify non-uprated variables (e.g. age) are carried forward unchanged
  • Verify _is_hdfstore_format() correctly distinguishes the two file formats

🤖 Generated with Claude Code

Adds USSingleYearDataset and USMultiYearDataset schema classes,
extend_single_year_dataset() with multiplicative uprating from the
parameter tree, and dual-path loading in Microsimulation that
auto-detects entity-level HDFStore files and extends them without
routing through the simulation engine.

Legacy h5py files continue to work via the existing code path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add extend_single_year_dataset for fast dataset year projection

1 participant