[RyanPCo]

rldb: move MultiDataset and format-neutral pieces out of zarr/

MultiDataset, EvenStrideDataset, pinned_episode_subset, worker_context and
_IndexMap move to egomimic/rldb/multi.py; PinError and _ensure_dataset_filter
to rldb/filters.py. zarr_dataset_multi re-exports them so the Hydra
_target_s (MultiDataset._from_resolver) and older imports keep resolving.

Also moved, since both the zarr and Lance paths use them:

• zarr/action_chunk_transforms.py -> rldb/transforms.py
• zarr/norm_cache.py, zarr/episode_norm_samples.py -> rldb/norm/
• zarr/utils.py (set_global_seed) -> utils/seed.py

MultiDataset._from_resolver no longer type-checks for LocalEpisodeResolver:
it forwards sync_from_s3 only when set, so a resolver that cannot sync now
raises instead of silently ignoring it. The tutorials that passed it to an
S3EpisodeResolver (a no-op) drop it.

norm_cache hashes both rldb.multi and zarr_dataset_multi as reader modules.
code_hash covers module source, so every existing norm cache misses once.

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01KY4JqVTaiGrzTaprPEzwZo

rldb: EpisodeLeaf base for zarr and Lance leaves

LanceRecordingDataset subclassed ZarrDataset only for its keymap windowing,
and carried a hand copy of the end of ZarrDataset.getitem that had already
drifted (no multi-camera or 3x4 K handling). Both now subclass
egomimic/rldb/leaf.EpisodeLeaf, which owns:

• the windowing, padding and mask helpers (_read_windows, _history_indices,
  _action_pad_mask, ...), with _chunk_end_idx still the per-subclass hook;
• _finish: masks, transforms, tensors, embodiment, fps, intrinsics and
  episode_hash for every leaf. A flat 9-value K (how Lance stores it) is now
  reshaped instead of becoming the NaN sentinel;
• the batch contract (plan_sample / fetch_planned / assemble_sample) and a
  default cache_fingerprint.

MultiDataset.getitems batches leaves with batched_fetch set rather than
any leaf with a fetch_planned attribute, which every leaf now has.
LanceStreamingDataset and format_bench's FastZarrDataset set it.

test_leaf_equivalence reads one recording as a Lance leaf and as a zarr
episode written from the same rows and checks every sample matches.

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01KY4JqVTaiGrzTaprPEzwZo

rldb/lance: annotations table with review, commit and loader support

annotations.lance is one global table keyed by a content-addressed
annotation_id, with half-open [t_start_ns, t_end_ns) intervals on the
recording clock and in-place grading columns. apply_annotations adds and
deletes in one commit; the release tag covers it like every other table.
The loader optionally returns all covering texts per frame, with opt-in
action clamping and annotated-only sampling. task_instance_id is now
deterministic, and the pilot commit's validation is reusable as
load_deliveries.

The format bench writes its spans to annotations.lance too, since the tasks
annotations column and schema.ANNOTATION are gone.

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01AeFJsNDQrLsyrDyJ2xgbrW

aria_process: Ray pipeline from raw Aria recordings to Lance tables

Converts and chunks recordings as Ray tasks and commits them in batches
from the driver; reruns skip committed recordings.

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01AeFJsNDQrLsyrDyJ2xgbrW

https://claude.ai/code/session_01HjCQBqQCr2igFEMR7yUovw

[RyanPCo]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• aria_to_lance: half-open task and alias end #727
• trainHydra: record the split and Lance versions a run read #726
• rldb, algo, eval: leave an untracked hand out of losses, stats and bounds #725
• rldb/lance: require_hands_in_frame and per-hand sample masks #724
• rldb/lance: action_clamp and history_floor at segment bounds #723
• rldb/lance: one leaf per recording, sampled over its task segments #722
• rldb/mcap: vendor MCAP spec 0.1 and validator #718
• rldb: move MultiDataset and format-neutral pieces out of zarr/ #717 👈 (View in Graphite)
• lance migration #716
• pi: keep per-sample intrinsics so overlays use the episode camera #690 : 2 other dependent PRs (#691 , #707 )
• data: multi-frame image memory window with per-step mask; history stride in seconds #689
• norm stats: cache sampled rows per episode so splits share them #687 : 1 other dependent PR (#688 )
• data: exclude Mecka episodes with failed egomotion; repin val videos #686
• eval: log validation videos to wandb #684 : 1 other dependent PR (#685 )
• slurm: stop cancelled jobs from leaking whole GPUs #682 : 1 other dependent PR (#683 )
• algo: RDT (Robotics Diffusion Transformer) on the HPT chassis #680 : 1 other dependent PR (#681 )
• pi: initialize from PaliGemma with a scratch action expert #677 : 1 other dependent PR (#678 )
• perf: vectorized image augs and a torch.compile knob for HPT and Pi; Pi clips gradients like openpi #676
• loader: one zarr read per key, not one per keymap entry #674
• eval: build the pinned-episode video batches only where they are used #673
• checkpoints: last.ckpt as a symlink, not a second full copy #672
• data: the mecka freeform folding operator-scaling ladder (k = 1/2/4) #671
• transforms: per-step wrist framing for proprio history #670
• viz: one keypoint renderer, the horizon trace #669
• configs: the flagship keypoint HPT arm, on the seen/unseen operator split #668
• trainer: ddp_bf16_compress, DDP with a bf16 gradient all-reduce #667
• eval: overlay videos play at the source rate under DDP #666
• hpt: build the policy on CPU and let Lightning place it #664 : 1 other dependent PR (#665 )
• hpt: never re-initialise pretrained stems; hash-check them; mask text padding #645 : 1 other dependent PR (#646 )
• obs: proprio history (K proprio steps, one token each, learned time embedding) #650
• loss: mask the repeat-last action padding out of the HPT loss #649
• resnet: keep a frozen backbone in eval mode; give backbones their own LR group #652
• augs: per-sample image augmentation for HPT train-mode augs #651
• eval: deterministic validation (seeded prompts + flow sampling); log-only grad-norm MAD #644
• data: mecka_fold_flagship_top3_hpt_6d, the 3-operator flagship train split #643
• val: subsampled metric loaders + pinned contiguous video loaders #642
• infra: mecka 6D / keypoint recipes, 300M models, defaults, and the rollout revert #640
• eval: layout-aware frame-invariant metrics, and a train-split viz head #659
• data: stop letting one bad episode or a rotation channel take a run down #658
• norm: guard degenerate channels, horizon pooling, and no rot6d normalization #657
• keypoints: the 144-D wrist-first hand-keypoint action becomes the human default #656
• mecka: fix the double-mirrored LEFT wrist frame, and correct existing zarrs #655
• 6d: PI goes 6D-only; Eva and Human gain the continuous-rotation modes #654
• 6d: continuous rotation representation, bimanual layouts, shared norm helpers #653
• train: give each Slurm slurm job gets its own cache dir to avoid conflict #638 : 1 other dependent PR (#639 )
• write git version into the run dir for reproducibility #637
• train: don't load PI base weights on resume and verify that checkpoint path exists #636
• configs: declare proprio and actions dims once per HPT model config #634 : 1 other dependent PR (#635 )
• cache norm stats for data configs to minimize recomputation #633
• import cloud packages lazily to support local training #631 : 1 other dependent PR (#632 )
• cache dataset resolver for faster training initialization #630
• validate episode hashes in data config before training #628
• modify pi camera map to match data #629
• add tests to ensure training works between every model and data vendor #626
• run CI tests on every PR #625
• add env variables for user-specific paths #624
• build: set pyproject and uv as the dependency managers #623
• fix(configs): fix paths for data hydra yaml configs #622
• refactor(algo): delete ACT #627 : 1 other dependent PR (#621 )
• tests: create dedicated test folder with unit and integration tests #620
• add pyright type checker #619
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.