fix(egosuite): match saved labels by source provenance - #362
Conversation
kstonekuan
left a comment
There was a problem hiding this comment.
LGTM, merging.
Every guard bites, which for an eight-item definition of done is the thing I most wanted to check:
pipeline matches on the basename again -> matches_same_named_sources, rejects_missing_or_unrelated
pipeline stops requiring source_uri -> rejects_missing_or_unrelated_canonical_provenance
legacy one-basename-many-paths accepted -> rejects_one_basename_for_multiple_sources
duplicate key back to the episode name -> matches_same_named_sources_by_canonical_provenance
episode sampling ranks by basename again -> episode_sampling_ranks_same_named_sources_by_source_uri
report stops recording source_uri -> label_report_records_hflow_source_identity
DoD 4 is the one I went looking at hardest, because "not the sole durable identity" is easy to claim and hard to hold. Checked what the report actually records:
under the data root -> 'episodes-in/episode.mcap'
outside it -> '/tmp/.../run-a/episode.mcap'
two same-basename sources get distinct identities: True
same relative position, different root prefix: identical
So the report inherits HFlow's own portability envelope rather than inventing one: root-relative and mount-independent where source_identity normalizes, absolute only where HFlow itself would be absolute. That is DoD 1 and DoD 4 answered by the same decision, which is why it holds. Telling the reader to export HFLOW_DATA_ROOT='data/egosuite-evaluation' in the how-to is the part that turns the guarantee into something a user actually gets, and keeping DEFAULT_HFLOW_DATA_ROOT at <root>/hflow matches build_ai_evaluation, so I would not change it.
Bumping the check contract to -v2 is right and easy to forget: the matching semantics moved, so a cached result from -v1 must not satisfy the new path. That is the same discipline as DoD 6's schema note.
The legacy compatibility path is better than the issue asked for. Accepting old reports whose basenames are unambiguous, and refusing exactly the ones where a basename covers two paths, means nobody regenerates a report that was never ambiguous, and nobody keeps one that silently was. Detecting a mixed report (some records with source_uri, some without) rather than guessing per record closes the other half of that.
Gate clean: 47 passed on the example project, 1409 passed / 6 skipped at the root with current main merged in.
One observation, no action. _source_uri_by_path builds a fresh hflow.App each time it is called, and run_evaluation plus both argument paths call it separately. It is cheap and side-effect-free, and I confirmed it leaves nothing on disk, so this is only worth knowing if identity resolution ever grows teeth.
Thanks also for #365 — that is the corpus-testing invitation working exactly as intended, and it is accepted.
Summary
source_uriidentity in new EgoSuite label reportsWhy
Recordings in different directories can share a basename. Basename-keyed reports merged or rejected those labels and could apply them to an unrelated canonical episode. This keeps report matching aligned with
App.source_identity()and canonicalsource_urisemantics.Closes #310.
Validation
uv run --locked --project examples/egosuite_evaluation ruff check --fix examples/egosuite_evaluation— passeduv run --locked --project examples/egosuite_evaluation ruff format examples/egosuite_evaluation— 6 files unchangeduv run --locked --project examples/egosuite_evaluation ty check --project examples/egosuite_evaluation --extra-search-path . examples/egosuite_evaluation— passeduv run --locked --project examples/egosuite_evaluation pytest -q examples/egosuite_evaluation/tests— 47 passeduv run pytest -q— 1391 passed, 6 skippedChecklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.