Skip to content

Add telemetry context propagation across search and load#28

Open
charles-turner-bot wants to merge 4 commits into
ACCESS-NRI:mainfrom
charles-turner-bot:ct/telemetry-context-lineage
Open

Add telemetry context propagation across search and load#28
charles-turner-bot wants to merge 4 commits into
ACCESS-NRI:mainfrom
charles-turner-bot:ct/telemetry-context-lineage

Conversation

@charles-turner-bot

Copy link
Copy Markdown
Contributor

Summary

This adds a small telemetry-lineage mechanism so catalog usage can be correlated across:

  • search()
  • selection of a result/catalog entry
  • to_xarray() materialisation

The implementation is deliberately lightweight:

  • adds a TelemetryContext dataclass anchored on a stable store_id
  • propagates that context through IcechunkCatalog.search() and downstream data-source creation
  • emits optional catalog.search, catalog.to_xarray.start, and catalog.to_xarray.end events via a callback hook
  • attaches lineage attrs to materialised xarray datasets for debugging/inspection

Why this shape

This avoids relying on Python object identity while still giving us a clean way to trace flows where the same backing icechunk store survives pre/post-search as different Python objects.

I kept it as context propagation + optional emitter hook rather than an accessor-only design, since the main problem seems to be lineage survival across object boundaries.

Testing

  • pixi run -e test-py313 lint
  • pixi run -e default pytest -q

Closes #27.

@codecov

codecov Bot commented Apr 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.70%. Comparing base (15a9c58) to head (fcbc9ad).
⚠️ Report is 30 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #28      +/-   ##
==========================================
+ Coverage   88.37%   89.70%   +1.33%     
==========================================
  Files           8        9       +1     
  Lines         456      515      +59     
==========================================
+ Hits          403      462      +59     
  Misses         53       53              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhancement: add telemetry/context propagation across search() -> to_xarray()

2 participants