Skip to content

fix(egosuite): match saved labels by source provenance - #362

Merged
kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
Galabavamsi:codex/310-egosuite-source-provenance
Sep 2, 2026
Merged

fix(egosuite): match saved labels by source provenance#362
kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
Galabavamsi:codex/310-egosuite-source-provenance

Conversation

@Galabavamsi

Copy link
Copy Markdown
Contributor

Summary

  • record HFlow source_uri identity in new EgoSuite label reports
  • match saved labels to canonical episode provenance instead of basenames
  • preserve unambiguous legacy reports and reject ambiguous ones with regeneration guidance
  • use source identity for deterministic episode, frame, and stratified sampling

Why

Recordings in different directories can share a basename. Basename-keyed reports merged or rejected those labels and could apply them to an unrelated canonical episode. This keeps report matching aligned with App.source_identity() and canonical source_uri semantics.

Closes #310.

Validation

  • uv run --locked --project examples/egosuite_evaluation ruff check --fix examples/egosuite_evaluation — passed
  • uv run --locked --project examples/egosuite_evaluation ruff format examples/egosuite_evaluation — 6 files unchanged
  • uv run --locked --project examples/egosuite_evaluation ty check --project examples/egosuite_evaluation --extra-search-path . examples/egosuite_evaluation — passed
  • uv run --locked --project examples/egosuite_evaluation pytest -q examples/egosuite_evaluation/tests — 47 passed
  • uv run pytest -q — 1391 passed, 6 skipped
  • fresh pre-reopen full suite — 1391 passed, 6 skipped

Checklist

  • I added or updated outcome-focused tests for changed business logic.
  • I updated documentation for changed behavior, flags, formats, or requirements.
  • I ran uv run ruff check --fix, uv run ruff format, and uv run ty check.
  • I ran the relevant pytest suite.
  • I did not add recordings, generated media, credentials, private URLs, or runtime artifacts.
  • I preserved stored-data compatibility or documented an explicit version change.

Copilot AI lite review requested due to automatic review settings September 2, 2026 14:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, merging.

Every guard bites, which for an eight-item definition of done is the thing I most wanted to check:

pipeline matches on the basename again      -> matches_same_named_sources, rejects_missing_or_unrelated
pipeline stops requiring source_uri         -> rejects_missing_or_unrelated_canonical_provenance
legacy one-basename-many-paths accepted     -> rejects_one_basename_for_multiple_sources
duplicate key back to the episode name      -> matches_same_named_sources_by_canonical_provenance
episode sampling ranks by basename again    -> episode_sampling_ranks_same_named_sources_by_source_uri
report stops recording source_uri           -> label_report_records_hflow_source_identity

DoD 4 is the one I went looking at hardest, because "not the sole durable identity" is easy to claim and hard to hold. Checked what the report actually records:

under the data root   -> 'episodes-in/episode.mcap'
outside it            -> '/tmp/.../run-a/episode.mcap'
two same-basename sources get distinct identities: True
same relative position, different root prefix: identical

So the report inherits HFlow's own portability envelope rather than inventing one: root-relative and mount-independent where source_identity normalizes, absolute only where HFlow itself would be absolute. That is DoD 1 and DoD 4 answered by the same decision, which is why it holds. Telling the reader to export HFLOW_DATA_ROOT='data/egosuite-evaluation' in the how-to is the part that turns the guarantee into something a user actually gets, and keeping DEFAULT_HFLOW_DATA_ROOT at <root>/hflow matches build_ai_evaluation, so I would not change it.

Bumping the check contract to -v2 is right and easy to forget: the matching semantics moved, so a cached result from -v1 must not satisfy the new path. That is the same discipline as DoD 6's schema note.

The legacy compatibility path is better than the issue asked for. Accepting old reports whose basenames are unambiguous, and refusing exactly the ones where a basename covers two paths, means nobody regenerates a report that was never ambiguous, and nobody keeps one that silently was. Detecting a mixed report (some records with source_uri, some without) rather than guessing per record closes the other half of that.

Gate clean: 47 passed on the example project, 1409 passed / 6 skipped at the root with current main merged in.

One observation, no action. _source_uri_by_path builds a fresh hflow.App each time it is called, and run_evaluation plus both argument paths call it separately. It is cheap and side-effect-free, and I confirmed it leaves nothing on disk, so this is only worth knowing if identity resolution ever grows teeth.

Thanks also for #365 — that is the corpus-testing invitation working exactly as intended, and it is accepted.

@kstonekuan
kstonekuan merged commit 050d145 into Hebbian-Robotics:main Sep 2, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Match EgoSuite saved labels through HFlow source provenance

3 participants