fix(profile): fold the sweep grid into the run identity#430
Open
vsbuffalo wants to merge 1 commit into
Open
Conversation
A profile's content-addressed base dir was keyed on the model + swept-param NAMES + config, but not the sweep VALUES. Re-running the same params over a different range therefore merged the two grids' cells into one base dir — a jagged union when the cell coordinates didn't line up (garki report). Fold the resolved grid (each focal axis + its values, canonicalized: axes sorted by name, values ascending) into ProfileBase, so a distinct grid is a distinct run. Re-running the SAME grid stays stable (still a cache hit); the method config stays in the `stage` level. Deliberate CAS re-key: every profile run_id changes, so profiles cached by an older binary recompute. No golden / expected / IR-schema impact — the profile CAS identity lives in the runid crate, not the IR. Cross-grid cell caching is traded away (an aligned refine recomputes shared cells); predictable separation beats a silent merge, and the shared-cell/manifest alternative is kept in reserve if the recompute cost ever bites. Red -> green: the old binary merged two different grids into one base dir (18 cells under one `sir-...`); the new binary gives them distinct dirs. profile_grid_identity.rs pins it — distinct grids get distinct base dirs, the same grid reuses its dir. Regression green: runid, profile_diagnostics, profile_pmmh, profile_priors, profile_greedy_order_invariant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A profile's content-addressed base dir was keyed on the model + swept-param names + config, but not the sweep values. So re-running the same params over a different range merged the two grids' cells into one base dir — a jagged union when the cell coordinates didn't line up (garki's report).
Fix
Fold the resolved grid (each focal axis + its values, canonicalized — axes sorted by name, values ascending) into
ProfileBase. A distinct grid is now a distinct run; re-running the same grid stays stable (a cache hit). Method config stays in thestagelevel.Red → green
61551324): grid Abeta=lin(0.1,0.5,3)…then grid Bbeta=lin(0.2,0.6,3)…→ one base dir, 18 cells piled together.profile_grid_identity.rspins it. Regression green:runid,profile_diagnostics,profile_pmmh,profile_priors,profile_greedy_order_invariant.Deliberate re-key
Every profile
run_idchanges → profiles cached by an older binary recompute. No golden / expected / IR-schema impact (the profile CAS identity lives inrunid, not the IR). Tradeoff: cross-grid cell caching is gone (an aligned refine recomputes shared cells). Predictable separation beats a silent merge; the shared-cell/manifest alternative (cells cache, a run-record lists its grid) is noted for later if the recompute cost bites.Branches off
main(which has the greedy work-queue), so this is greedy + the grid fix.