diff --git a/AGENTS.md b/AGENTS.md index 339e527988..9823f10894 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,7 +33,7 @@ datasets, normalizers, environments, recorded source, and unverified checkpoints Use bounded task directories; never discover files by recursively scanning a whole scratch/project root. Candidate inventory alone does not authorize deletion. -Publish source from the authenticated Skynet account with Graphite `gt submit`, +Publish source with Graphite `gt submit` using the user's authorized GitHub account, preserving branch parents. Do not merge without an explicit user request. ## Aidan stationery source diff --git a/README.md b/README.md index fa6cadf697..e16cededee 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ stack for training and evaluation. For cluster training and maintenance, start with [Pipeline and cluster operations](docs/cluster-pipeline.md). It distinguishes current source, historical run checkouts, and shared data/runtime dependencies. +For current task recipes and the data/model/evaluator configuration layout, +start with the [configuration guide](docs/CONFIGURATION.md). For the integration branch, start with the [migration guide](docs/integration/MIGRATION.md) for the locked environment, retained HPT/PI recipes, data tools, and checkpoint rules. [Integration status](docs/integration/PLAN.md) records the draft stack and @@ -146,14 +148,22 @@ python egomimic/scripts/data_download/sync_s3.py \ ``` ### Training -Select a complete Pipeline experiment; the root config deliberately has no -implicit model, data, or evaluator: -``` bash -python egomimic/trainHydra.py --config-name=train_zarr_cartesian \ - model= data= \ - evaluator= -``` -For full instructions on training see [``training.md``](./training.md) +Select a complete task recipe. Each public real experiment explicitly links its +model, data, evaluator, trainer, callbacks and logger. Inspect it without +starting training: + +```bash +python egomimic/trainHydra.py \ + +experiment=arc_real/stationary_velocity --cfg job --resolve +``` + +The [experiment menu](egomimic/hydra_configs/experiment/README.md) lists +`baseline`, `arc_vel` and `arc_dur` for each real task. A cluster launch adds +`-m hydra/launcher=` after resource/data preflight; +machine-specific paths and credentials must be configured. Advanced direct +`model=... data=... evaluator=...` composition remains supported; the root +config deliberately does not select those components implicitly. +See [training.md](./training.md) for the graph and launch workflow. ### Converting your own data See [``embodiment_tutorial.ipynb``](./egomimic/scripts/tutorials/embodiment_tutorial.ipynb) as reference to write a conversion script for your own data. diff --git a/docs/ARC_CHUNKING_MODES.md b/docs/ARC_CHUNKING_MODES.md index 4766380f87..a5dd9085eb 100644 --- a/docs/ARC_CHUNKING_MODES.md +++ b/docs/ARC_CHUNKING_MODES.md @@ -3,7 +3,8 @@ > Historical reference, not a current API or launch menu. Removed E1/legacy > codecs, configs and launchers described below require their recorded source. > Current usage: [clock contract](ARC_CLOCK_CONTRACT.md) and -> [experiment menu](../egomimic/hydra_configs/experiment/README.md). +> [experiment menu](../egomimic/hydra_configs/experiment/README.md), with the +> current folder layout in [CONFIGURATION.md](CONFIGURATION.md). Every retained `abc_arc/robot_bc` and `abc_arc/human_bc` visual experiment accepts `abc.arc_chunking_mode=race`, `multistream`, or `joint_distance`. diff --git a/docs/ARC_CLOCK_CONTRACT.md b/docs/ARC_CLOCK_CONTRACT.md index 4d26b24781..77c9228fd7 100644 --- a/docs/ARC_CLOCK_CONTRACT.md +++ b/docs/ARC_CLOCK_CONTRACT.md @@ -1,10 +1,10 @@ -# Cartesian ARC: one codec, two clocks, two stream layouts +# Cartesian ARC: one codec, two clocks, four stream layouts This contract replaces the historical Cartesian mean/wide/stacked/clock/E1 families. It builds on PR #263 and preserves the local first-row-delay implementation. It does not change the separate planar or Libero codecs. -New run presets are grouped by task under `hydra_configs/experiment/`: +New run presets are grouped by task under `hydra_configs/experiment/arc_real/`: `organize_stationary`, `fold_and_stack_towels`, `wireless_earbuds`, and `multitask`, each with `arc_vel`, `arc_dur`, and `baseline`. See the [experiment menu](../egomimic/hydra_configs/experiment/README.md) @@ -47,8 +47,8 @@ represent independent gripper movement. ## Timing rows Row zero of every timing column represents that stream's initial delay. -Rows i > 0 represent the incoming interval from waypoint i-1 to waypoint i. -There is no repeated final padding clock. +The meaning of later rows depends on `clock_mode`: incoming rates for velocity, +absolute waypoint seconds for duration. There is no repeated final padding clock. - velocity: row zero is seconds; later rows are the distance between emitted supports divided by the source interval's seconds. Units are m/s, rad/s, @@ -62,15 +62,17 @@ There is no repeated final padding clock. `first_delay_absolute_v1`; velocity keeps `first_delay_incoming_v1`. Translation distances are Euclidean, rotation distances are SO(3) geodesic, -and gripper distances are absolute value differences. No velocity smoothing, -physical speed clamp, or decoder-duration floor is introduced. +and gripper distances are absolute value differences. Velocity-mode rates are +not smoothed or physically speed-clamped. Duration-mode clock projection does +enforce its explicit 5 m/s translation bound. Neither mode adds an arbitrary +decoder-duration floor. An initial wait retains constant geometry before the first movement. Rotation and distance-gripper target times use first arrival, so a final stationary plateau does not stretch the last moving interval. Full-window holds retain constant waypoints. Velocity interval rates are zero. -Duration intervals span the configured hold_window_frames × dt. Constant +Duration absolute clocks span the configured hold_window_frames × dt. Constant streams do not terminate a chunk. Internal dwells are not explicitly allocated duplicate supports; zero speed alone cannot encode a zero-distance dwell time. @@ -82,7 +84,8 @@ Command-change supports and endpoint padding retain the prior local semantics. Both use the same codec and `control_steps()`: 1. Keep an exact integer waypoint prefix, e.g. 30 of M100. -2. Decode every stream using its own start delay and incoming intervals. +2. Decode every stream using its selected clock representation: start delay + plus distance/rate intervals, or absolute waypoint seconds. 3. Replan when the first moving stream reaches its prefix endpoint. 4. Ignore fully constant streams in this race. @@ -113,12 +116,14 @@ quantile widening (`norm_stats.widen_degenerate_quantiles: true`). The first-delay six-stream refactor was compared numerically against its preceding local implementation across 32 synthetic clock/gripper/hold cases, -including token, detokenizer, and termination parity. New regression tests cover -all four clock/stream combinations and validation/rollout agreement. +including token, detokenizer, and termination parity. Regression tests cover +clock/stream combinations and validation/rollout agreement. -Verification for this migration: 536 focused CPU tests passed, including +Historical verification for the initial two-layout migration: 536 focused CPU +tests passed, including normalization-cache rejection, baseline graph forward/backward, source adapters, -and planar ARC regressions. All 379 shipped Hydra YAML files composed. +and planar ARC regressions. All 379 Hydra YAML files shipped at that revision +composed; these are historical counts, not the current configuration inventory. The 32 six-stream comparisons used predecessor `f364923a3e675106882823da17f840a9505ab972`, with token tolerance 1e-10, decoded-trajectory tolerance 1e-9, and identical execution-step counts. diff --git a/docs/ARC_OAT.md b/docs/ARC_OAT.md index 42ad7a622c..6500a5df4b 100644 --- a/docs/ARC_OAT.md +++ b/docs/ARC_OAT.md @@ -450,18 +450,18 @@ examples use the measured LIBERO-10 selections in the [dated replay report](results/libero_arc_timed_replay_20260921.md): ```sh -python -m egomimic.trainHydra +experiment=sim/oat/libero_oattok \ +python -m egomimic.trainHydra +experiment=arc_sim/oat/libero_oattok \ benchmark.suite=libero_10 benchmark.dataset=/datasets/arc_oat/libero_10.zarr \ hydra.run.dir=/runs/tokenizer -python -m egomimic.trainHydra +experiment=sim/oat/libero_oatpolicy \ +python -m egomimic.trainHydra +experiment=arc_sim/oat/libero_oatpolicy \ benchmark.suite=libero_10 benchmark.dataset=/datasets/arc_oat/libero_10.zarr \ benchmark.tokenizer_checkpoint=/runs/tokenizer/checkpoints/last.ckpt \ hydra.run.dir=/runs/oat -python -m egomimic.trainHydra +experiment=sim/oat/libero_arc_dur_policy \ +python -m egomimic.trainHydra +experiment=arc_sim/oat/libero_arc_dur_policy \ benchmark.suite=libero_10 benchmark.dataset=/datasets/arc_oat/libero_10.zarr \ benchmark.arc_waypoints=32 benchmark.arc_max_translation=0.8 \ benchmark.arc_max_rotation_degrees=192 hydra.run.dir=/runs/arc_dur -python -m egomimic.trainHydra +experiment=sim/oat/libero_arc_stk_policy \ +python -m egomimic.trainHydra +experiment=arc_sim/oat/libero_arc_stk_policy \ benchmark.suite=libero_10 benchmark.dataset=/datasets/arc_oat/libero_10.zarr \ benchmark.arc_waypoints=32 benchmark.arc_max_translation=1.6 \ benchmark.arc_max_rotation_degrees=128 hydra.run.dir=/runs/arc_stk diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000000..a66442139b --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,139 @@ +# Configuration guide + +Start with a task experiment, then follow its Hydra `defaults` links. Folder +names organize recipes; Python `_target_` declarations identify implementations. +The generic pipeline does not choose behavior from a model or task name. + +```text +egomimic/hydra_configs/ +├── experiment/ +│ ├── arc_real/ +│ │ ├── stationary_{time,duration,velocity}.yaml +│ │ ├── towels_{time,duration,velocity}.yaml +│ │ └── earbuds_{time,duration,velocity}.yaml +│ ├── abc_arc/robot_bc/ retained ABC-lab recipes, including multitask +│ └── arc_sim/ PushT, LIBERO, OAT and synthetic recipes +├── data/ episode selection, transforms and loaders +├── model/ +│ ├── base/ shared model definitions and settings +│ ├── hpt/ HPT recipe leaves +│ ├── pi/ PI recipe leaves +│ └── … distinct non-HPT/PI model families +├── model_contract/ shared deployment and data contracts +├── evaluator/ +│ ├── eval_open_loop_sim.yaml +│ ├── eval_cartesian_*.yaml behavior/frame-specific metrics and videos +│ ├── _base/ inherited evaluator defaults +│ └── viz/ reusable renderers +└── hydra/launcher/ cluster allocation, separate from experiment +``` + +## Select and inspect one experiment + +The three real tasks each expose `_time`, `_velocity` and `_duration`: +nine standard entry points, preserving the current task pools and protocols. +Every leaf directly links model, data, +evaluator, trainer, callbacks and logger. The model owns the optimizer and +scheduler. Real recipes do not inherit other experiments: each contains its +task/run overrides and imports reusable definitions from the component groups. + +Activate the project environment, configure required data-cache/weight paths, +then inspect without running training: + +```bash +python egomimic/trainHydra.py \ + +experiment=arc_real/stationary_velocity --cfg job --resolve +``` + +The [experiment menu](../egomimic/hydra_configs/experiment/README.md) records +task pools and numeric defaults. The top-level experiment directories `abc/` +and `abc_arc/human_bc/`, and `arc_real/arcdur_abstime/`, are removed. Their old +commands are historical, not aliases for the retained task recipes. The +removed absolute-time recipe menu does not remove duration mode: +`arc_real/_duration` uses absolute waypoint seconds. + +For an authorized cluster launch, use that same experiment with `-m` and an +explicit approved `hydra/launcher=...` selection. Verify the resolved resource +request, effective global batch, pinned data and normalization before launch. +See [training.md](../training.md) and [cluster operations](cluster-pipeline.md). + +## What belongs in each layer + +| Layer | Owns | Does not imply | +| --- | --- | --- | +| `experiment/arc_real/_` | Complete component selection, task values and run schedule | Another task-specific Python runtime | +| `data/` | SQL filters or pinned episode lists, splits, transforms, loader behavior | The model architecture | +| `model/hpt/`, `model/pi/` | Model graph and family-specific recipe overrides | A model-specific evaluator implementation | +| `model/base/` | Identical inherited graphs/settings factored out of leaves | An experiment with data and evaluation selected | +| `model_contract/` | Reusable inference/deployment and preprocessing declarations | Permission to reinterpret a checkpoint | +| `evaluator/` | Metrics, frame reversion, execution protocol and artifacts | Training-time tokenization choices inferred from a model name | + +`model/base/` contains shared HPT flow, cotrain, keypoint and Qwen definitions, +PI defaults, and YAM HPT/diffusion settings. Distinct architectures and +contracts remain distinct; only equivalent definitions should be shared. +The Python networks stay in `egomimic/models/`, and configured graph stages +stay in `egomimic/pipeline/`. Moving YAML does not move or resize a network. + +### Direct component links; no shared experiment parents + +`experiment/_shared/` and the real experiment-parent chain are removed. Every +one of the nine `arc_real/` recipes selects its six components directly. Model +dimensions live in `model/base/real_*_settings`, training defaults in +`trainer/real_*`, and the checkpoint schedule in +`callbacks/real_yam`. Task identity, dataset selection, caps and run +overrides remain visible in the experiment itself. + +Canonical clock dimensions/provenance live in +`model_contract/arc_clock_settings`. YAM ARC experiments explicitly +select `model/hpt/yam_300m_arc` and the matching `data/abc_arc/*_arc` +adapter; the data adapter retains the original pinned pool and binds the +model-declared transforms through `data/_base/yam_arc`. + +Component inheritance still deduplicates model graphs and dataset lists. +There is no earbuds-to-towels-to-stationery experiment inheritance, and no +extra experiment file is needed to launch any real recipe. Simulation recipes +retain their existing parent structure under the renamed `arc_sim/` folder. + +## Evaluator selection + +| Selection | Purpose | +| --- | --- | +| `eval_open_loop_sim` | Recorded-episode execution simulation for real baseline and ARC recipes; metrics, DTW and executed-chunk videos | +| `eval_bimanual_cartesian` | Time-indexed chunk errors and overlays, without sequential execution/replanning | +| `eval_cartesian_chunk_metrics` | Metrics-only chunk comparison using the configured YAM batch budget; no renderer | +| `eval_cartesian_camera`, `eval_cartesian_wrist` | Cartesian pose metrics with the chosen frame reversion | +| `eval_cartesian_*_prompt` | Corresponding metrics with sampled-prompt overlays; `6d` variants use continuous-6D rotations | + +For example, the old `eval_hpt` selector becomes `eval_cartesian_camera`, and +the old root `eval_pi` becomes `eval_cartesian_camera_prompt`. A model name is +not sufficient to choose the right evaluator: frame, rotation encoding, +ground-truth key and execution protocol must match the data/model contract. +Renderer-only defaults live in `evaluator/viz/`; common evaluator definitions +live in `evaluator/_base/`. + +`eval_open_loop_sim.yaml` groups settings by execution, canonical codec, +complete-episode scoring and artifacts. Real baselines predict 100 actions, +execute the first 25 at 30 Hz, then replan against the next recorded +observation. ARC keeps a waypoint prefix and ends when the first moving stream +finishes. This is open-loop replay of recorded observations, not closed-loop +physics or robot evaluation. Ordered complete episodes and matching video/GT +coverage are part of its contract, not optional batch-sampling details. + +The [Cartesian clock contract](ARC_CLOCK_CONTRACT.md) defines velocity versus +absolute-seconds duration, four stream layouts and start-delay semantics. +Planar and LIBERO codecs have their own contracts; the removed Cartesian +legacy options must not be reintroduced into the open-loop YAML. + +## Provenance and migration + +Use the new selectors for new compositions. Keep the original source commit, +resolved config, normalization and checkpoint for an existing run. Do not +resume a different action representation under the same run identity or infer +decoder compatibility from M×20/M×18/M×16 shape alone. YAML organization does +not migrate trained weights. See [checkpoint rules](integration/CHECKPOINTS.md) +and [current real-task recipes](STATTEMPO.md). + +The exact old-to-new model, evaluator and data selector map is in +[`config_cleanup_migration.json`](config_cleanup_migration.json). Removed +experiment menus have no automatic replacement: choosing another task recipe +could change its data pool or training protocol. diff --git a/docs/CURRENT_FAMILY_PROFILES.md b/docs/CURRENT_FAMILY_PROFILES.md index ae131ea474..12a7a1d0ba 100644 --- a/docs/CURRENT_FAMILY_PROFILES.md +++ b/docs/CURRENT_FAMILY_PROFILES.md @@ -61,7 +61,7 @@ selected emitted configs remain separate immutable task artifacts. ## PushShapes: explicit multiplier versus historical physical speed -AF selects `sim/pusht/action_flow_cotrain_uc_multiplier_interpolation`; its typed +AF selects `arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation`; its typed route requires fresh initialization, scalar raw dimensionless multiplier, no physical-speed reference, Euler50, batches32+32 and fixed EMA0.9978. The latest explicit PushT preference uses normal native per-source image processing, not diff --git a/docs/LIBERO_ARC_OAT_DP.md b/docs/LIBERO_ARC_OAT_DP.md index 858a28260c..593b74aeed 100644 --- a/docs/LIBERO_ARC_OAT_DP.md +++ b/docs/LIBERO_ARC_OAT_DP.md @@ -47,7 +47,7 @@ only the camera/state encoder receives the observation LR. Use `--arc-backbone oat_dp` with a standalone `--arc-profile` and one ARC mode in `scripts/benchmarks/launch_libero_osmo.py`. The new experiment is -`sim/oat/libero_arc_oat_dp_policy`; the existing default remains the U-Net. +`arc_sim/oat/libero_arc_oat_dp_policy`; the existing default remains the U-Net. A fresh policy may reuse `--arc-replay-runs-file` without `--resume-from-run`. The worker still checks the complete replay confirmation, chosen R/D/M, diff --git a/docs/LIBERO_ARC_OAT_HYBRID.md b/docs/LIBERO_ARC_OAT_HYBRID.md index c6a9acb0cf..0058513a6b 100644 --- a/docs/LIBERO_ARC_OAT_HYBRID.md +++ b/docs/LIBERO_ARC_OAT_HYBRID.md @@ -42,8 +42,8 @@ episodes, two observations, a 32-action horizon and 16 executed actions per replan. These are separate experiment runs; previous ARC/OAT weights are not used to initialize the models. -The YAML recipes are `sim/oat/libero_arc_oattok` and -`sim/oat/libero_arc_oatpolicy`. They compose the existing `LiberoArcStage`, +The YAML recipes are `arc_sim/oat/libero_arc_oattok` and +`arc_sim/oat/libero_arc_oatpolicy`. They compose the existing `LiberoArcStage`, `OATTokenizerStage`/`OATPolicyStage`, and ARC decoder in `PipelineAlgo`. The shared trainer, decoded replay loader, normalizer and evaluator are used. Validation reports dense controller-action errors after decoding, while the diff --git a/docs/LIBERO_ARC_REPLAY.md b/docs/LIBERO_ARC_REPLAY.md index a675f12b2a..f652470ac9 100644 --- a/docs/LIBERO_ARC_REPLAY.md +++ b/docs/LIBERO_ARC_REPLAY.md @@ -60,7 +60,7 @@ selection 2–9, and final testing 35–49. Each mode freezes its own R/D/M choi These specs use 16 workers, recycling after four episodes, and 128 GiB memory. Use `--replay-spec libero_arc_replay_stk` or `libero_arc_replay_dur` when rendering an OSMO workflow. Their corresponding policy recipes are -`sim/oat/libero_arc_stk_policy` and `sim/oat/libero_arc_dur_policy`. +`arc_sim/oat/libero_arc_stk_policy` and `arc_sim/oat/libero_arc_dur_policy`. The earlier LIBERO codec/results are explicitly **`joint_dur`**, a shared-clock M×11 baseline. They cannot be reused as STK or independent-clock DUR evidence. diff --git a/docs/PI05_GRAPH.md b/docs/PI05_GRAPH.md index 52d2057b2a..bff4e543cd 100644 --- a/docs/PI05_GRAPH.md +++ b/docs/PI05_GRAPH.md @@ -38,8 +38,14 @@ accept `WT`, `PY`, `PI05_WEIGHTS`, and their existing norm/output/resume options Their lab filesystem defaults and Slurm partitions still need to match the cluster. Checkpoints are retained. No launcher was submitted during this port. -Data, model and evaluator groups are prefixed with `pi05/`, for example -`data=pi05/mecka_abc_pi_6d`. The older `aria_bimanual` configuration slot maps +Data groups retain `pi05/`, for example `data=pi05/mecka_abc_pi_6d`. Model +recipes now live under `model/pi/` with their shared implementation defaults +under `model/base/`; for example `model=pi/bc_mecka_abc_6d`. +Evaluators are named by behavior and frame, such as +`evaluator=eval_cartesian_wrist_6d_prompt`, not by model family. See the +[configuration guide](CONFIGURATION.md). This config reorganization does not +move the Python backend in `models/pi05/` or its compatibility imports. +The older `aria_bimanual` configuration slot maps to `human_bimanual`; the SQL filters and episode exclusions retain their source values. Some historical filters intentionally describe older episode-table snapshots and may require a matching snapshot to reproduce their experiments. @@ -91,7 +97,7 @@ dataset access and robot actuation remain cluster validation work. ## Attention and latent analysis -Root `pi0.5_*` recipes declare `PI05AttentionProvider` under +The retained `model/pi/*_legacy` recipes declare `PI05AttentionProvider` under `model.diagnostic_provider`. Select `data=cotrain_pi_latent evaluator=eval_latent` with a cotraining model to capture attention keys through the normal graph. Set `data.selection.mode=pairs` for the source-pinned pairs, or `custom` with diff --git a/docs/STATTEMPO.md b/docs/STATTEMPO.md index 7964eaf815..f85109bd79 100644 --- a/docs/STATTEMPO.md +++ b/docs/STATTEMPO.md @@ -1,26 +1,39 @@ # Current real-task recipes The current ARC cleanup stack is based on PR #263. Its public experiment -entry points live under `egomimic/hydra_configs/experiment/real/`: +entry points live under `egomimic/hydra_configs/experiment/arc_real/`: -- `organize_stationary/{baseline,arc_vel,arc_dur}` -- `fold_and_stack_towels/{baseline,arc_vel,arc_dur}` -- `wireless_earbuds/{baseline,arc_vel,arc_dur}` -- `multitask/{baseline,arc_vel,arc_dur}` +- `stationary_{time,velocity,duration}` — 417 training / 24 validation episodes +- `towels_{time,velocity,duration}` — 462 training / 17 validation episodes +- `earbuds_{time,velocity,duration}` — 339 training / 15 validation episodes + +ABC-lab multitask recipes remain under `experiment/abc_arc/robot_bc/multitask/`, +separate from the nine rl2 real-task recipes introduced in PR #277. Each YAML selects a complete model/data/evaluator/training/logging configuration. See the [experiment menu](../egomimic/hydra_configs/experiment/README.md) for -defaults, frozen dataset pools, variants and launch selection. +defaults, pinned dataset pools, variants and launch selection, and the +[configuration guide](CONFIGURATION.md) for the shared-component layout. + +The canonical [clock contract](ARC_CLOCK_CONTRACT.md) has incoming waypoint +velocity or absolute waypoint seconds, with start delay in row zero. The +`full_separation`, `translation_rotation`, `translation` and `projection` +layouts use six, four, two and two clocks respectively. `_duration` selects the +absolute-seconds duration representation; the separate `arc_real/arcdur_abstime` +experiment menu is removed, not required to enable it. Retired codecs are not +selectable. -The canonical [clock contract](ARC_CLOCK_CONTRACT.md) has velocity or log -interval duration, first-row delay, and four or six independent clocks. -Neither retired codecs nor absolute-seconds layouts are selectable here. +All nine use HPT300, 240k optimizer steps and 25% open-loop evaluation. ARC +defaults to M200, a 100-frame source window, one translation clock per arm and +D=0.22 m for stationery/earbuds or D=0.33 m for towels. The rollout contract +executes 50% of the ARC waypoints; these distinct evaluation/deployment prefixes +are preserved from PR #277. -Frozen towel/earbud recipes retain their HPT300 architecture and episode -membership. The shared definitions are `model/abc_arc/frozen_yam_hpt300_time`, -`model/abc_arc/frozen_yam_hpt_flow`, `model_contract/frozen_yam_wrists`, -and `experiment/_shared/frozen_yam/`. These are dependencies, not extra -public run menus. ARC leaves override the baseline codec and evaluator. +YAM task recipes retain their HPT300 architecture and episode +membership. They link directly to `model/hpt/yam_300m` (baseline) or +`model/hpt/yam_300m_arc` (ARC), their matching data adapter, and the +open-loop evaluator. Reusable settings live in `model/base/`, `trainer/` and +`model_contract/`; no real recipe imports another experiment or `_shared/`. Older experiments, removed launchers and checkpoint layouts remain recoverable from their original Git commits. Preserve every running job's recorded source, diff --git a/docs/YAM_RUNTIME.md b/docs/YAM_RUNTIME.md index a2d875f0d4..e495b78ce0 100644 --- a/docs/YAM_RUNTIME.md +++ b/docs/YAM_RUNTIME.md @@ -384,13 +384,17 @@ EEF pose at inference time. `model_frame` instead uses the configured calibratio Predictions are unnormalized once before decoding/frame reversion. A generated ARC artifact contains the schema-2 model declaration and immutable binding hashes; it is generated, not hand-written. Select the matching -`experiment/real//arc_vel` or `arc_dur` recipe. Its model contract declares +`experiment/arc_real//arc_vel` or `arc_dur` recipe. Its model contract declares the decoder, clock mode, stream separation, waypoint cap and runtime controls. Changing a YAML cannot migrate weights trained with another representation. -The current codec supports per-waypoint velocity or log interval duration, -with full separation (M×20) or translation/rotation separation (M×18). -Row zero encodes the leading delay. Retired codecs require their original +The current codec supports incoming per-waypoint velocity or absolute waypoint +seconds. Stream layouts are `full_separation` (M×20), `translation_rotation` +(M×18), `translation` (M×16) and `projection` (M×16). In the last two, rotation +and gripper use their arm's translation clock; projection measures progress +at an offset point on the end effector. Row zero encodes the leading delay. +See the [clock contract](ARC_CLOCK_CONTRACT.md) and +[configuration guide](CONFIGURATION.md). Retired codecs require their original source revision; they are not runtime alternatives in this stack. A `decoder_attribute` control binds only to an explicitly configured decoder. @@ -408,7 +412,7 @@ codec's own clocks; no checkpoint-era tokenizer copies are carried. Use the exported model contract's `first_stream`, `execute_percent` and sampling budget rather than imposing a station-wide default. The frozen time -contract (`model_contract/frozen_yam_wrists.yaml`) declares `TimeChunkRetimer` +contract (`model_contract/yam_wrists.yaml`) declares `TimeChunkRetimer` and replay/hold tempo controls; 100% preserves the original chunk. Replay tempo and termination use decode-only helpers and do not alter training targets. diff --git a/docs/action-flow-usocket-candidates.md b/docs/action-flow-usocket-candidates.md index 0bac8af85d..90d8b9e352 100644 --- a/docs/action-flow-usocket-candidates.md +++ b/docs/action-flow-usocket-candidates.md @@ -11,7 +11,7 @@ reconstruction-only phase. The generic pipeline runner is unchanged. ### FM-only clean stop-gradient (method 2) -Config: `sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42`. +Config: `arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42`. - Reconstruction: `A → E → g → MSE(A)`; updates E and g, weight1. - Latent FM: `A → E → stop-gradient → latent bridge → v(z,t,O) → latent MSE`; @@ -28,7 +28,7 @@ Inference: `Gaussian latent → reverse Euler16(v,O) → g → action`. ### Learned Gaussian-bridge likelihood (method 3) -Config: `sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42`. +Config: `arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42`. Private reference mean `mu(A,t)=(1-t)E_raw(A,t)` ends exactly at zero. The reference stage, Gaussian noising stage, conditional reverse-mean transformer, @@ -64,7 +64,7 @@ errors and fixed-bank stochastic EnergyScore@32 remain required. ### Exact graph section (method 4, restricted diagnostic) -Config: `sim/pusht/action_flow_bc_usocket_graph_section_s42`. +Config: `arc_sim/pusht/action_flow_bc_usocket_graph_section_s42`. `E(A)=(A,f(A))`; split each latent token into x4,h4 and decode `g(x,h)=x + [R(x,h)-R(x,f(x))]`. The subtraction is grouped before adding x. diff --git a/docs/cluster-pipeline.md b/docs/cluster-pipeline.md index 81cbcbf7f2..5942487be9 100644 --- a/docs/cluster-pipeline.md +++ b/docs/cluster-pipeline.md @@ -34,8 +34,8 @@ The paired raw-multiplier recipes are: | Family | Experiment selector | | --- | --- | -| Action Flow | `sim/pusht/action_flow_cotrain_uc_multiplier_interpolation` | -| Matched Standard DP | `sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier` | +| Action Flow | `arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation` | +| Matched Standard DP | `arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier` | Use normal per-source image processing, retaining 32 U-Socket plus 32 ChainGripper samples per optimizer update. ChainGripper manual4919 contains diff --git a/docs/config_cleanup_migration.json b/docs/config_cleanup_migration.json new file mode 100644 index 0000000000..6dd8d523b1 --- /dev/null +++ b/docs/config_cleanup_migration.json @@ -0,0 +1,56 @@ +{ + "note": "Selectors only. Preserve original source/configs for existing checkpoints; removed experiments have no automatic replacement.", + "renamed": { + "data/cotrain_pi_pickplace_qwen": "data/cotrain_pickplace_qwen", + "evaluator/eval_hpt": "evaluator/eval_cartesian_camera", + "evaluator/eval_hpt_wrist": "evaluator/eval_cartesian_wrist", + "evaluator/eval_pi": "evaluator/eval_cartesian_camera_prompt", + "evaluator/pi05/eval_pi": "evaluator/eval_cartesian_wrist_prompt", + "evaluator/pi05/eval_pi_camframe_6d": "evaluator/eval_cartesian_camera_6d_prompt", + "evaluator/pi05/eval_pi_wristframe_6d": "evaluator/eval_cartesian_wrist_6d_prompt", + "evaluator/pi05/eval_pi_wristframe_6d_grip": "evaluator/eval_cartesian_wrist_6d_gripper_prompt", + "evaluator/viz/cartesian_wrist": "evaluator/viz/cartesian", + "evaluator/viz/cartesian_wrist_lang": "evaluator/viz/cartesian_annotations", + "evaluator/viz/cotrain_lang": "evaluator/viz/cartesian_annotations", + "evaluator/viz/pi_cartesian": "evaluator/viz/cartesian", + "evaluator/viz/pi_cartesian_lang": "evaluator/viz/cartesian_prompt", + "evaluator/viz/pi_cartesian_lang_wrist": "evaluator/viz/cartesian_prompt", + "model/abc/yam_bimanual_hpt": "model/hpt/yam_bimanual", + "model/abc_arc/hpt_bimanual_visual": "model/hpt/bimanual_visual", + "model/abc_arc/hpt_yam_visual": "model/hpt/yam_visual", + "model/egobridge": "model/hpt/egobridge", + "model/hpt_bc_flow_aria": "model/hpt/bc_flow_human", + "model/hpt_bc_flow_eva": "model/hpt/bc_flow_eva", + "model/hpt_bc_flow_human": "model/hpt/bc_flow_human", + "model/hpt_bc_flow_mecka": "model/hpt/bc_flow_mecka", + "model/hpt_bc_flow_scale": "model/hpt/bc_flow_scale", + "model/hpt_bc_keypoints_base": "model/base/hpt_keypoints", + "model/hpt_bc_keypoints_wrist": "model/hpt/bc_keypoints_wrist", + "model/hpt_bc_pickplace_qwen_pertoken": "model/hpt/bc_pickplace_qwen_pertoken", + "model/hpt_bc_pickplace_qwen_pooled": "model/hpt/bc_pickplace_qwen_pooled", + "model/hpt_cotrain_enc_dec_base": "model/base/hpt_cotrain_enc_dec", + "model/hpt_cotrain_flow_seperate_head": "model/hpt/cotrain_flow_seperate_head", + "model/hpt_cotrain_flow_shared_head": "model/hpt/cotrain_flow_shared_head", + "model/hpt_cotrain_mecka_flow_shared_head": "model/hpt/cotrain_mecka_flow_shared_head", + "model/hpt_cotrain_scale_flow_shared_head": "model/hpt/cotrain_mecka_flow_shared_head", + "model/pi0.5_base": "model/base/pi05", + "model/pi0.5_bc_aria": "model/pi/bc_aria_legacy", + "model/pi0.5_bc_eva": "model/pi/bc_eva_legacy", + "model/pi0.5_bc_mecka": "model/pi/bc_mecka_legacy", + "model/pi0.5_bc_scale": "model/pi/bc_mecka_legacy", + "model/pi0.5_cotrain_eva_aria": "model/pi/cotrain_eva_aria_legacy", + "model/pi0.5_cotrain_mecka_scale": "model/pi/bc_mecka_legacy", + "model/pi05/pi0.5_bc_abc_eva_6d": "model/pi/bc_abc_eva_6d", + "model/pi05/pi0.5_bc_eva": "model/pi/bc_eva", + "model/pi05/pi0.5_bc_mecka_6d": "model/pi/bc_mecka_6d", + "model/pi05/pi0.5_bc_mecka_abc_6d": "model/pi/bc_mecka_abc_6d", + "model/pi05/pi0.5_cotrain_eva_aria_6d": "model/pi/cotrain_eva_aria_6d", + "model/pi05/pi0.5_ft_abc_eva_6d": "model/pi/bc_abc_eva_6d", + "model/pi05/pi0.5_ft_aria_fold_grip": "model/pi/ft_aria_fold_grip" + }, + "removed_experiment_menus": [ + "abc", + "abc_arc/human_bc", + "arc_real/arcdur_abstime" + ] +} diff --git a/docs/experiment_config_migration.json b/docs/experiment_config_migration.json index fbd8117969..e8a4411ee8 100644 --- a/docs/experiment_config_migration.json +++ b/docs/experiment_config_migration.json @@ -1,14 +1,14 @@ { - "abc_arc/cotrain/organize_clock_duration_full_separation": "real/stationary_duration", - "abc_arc/cotrain/organize_clock_duration_translation_rotation": "real/stationary_duration", - "abc_arc/cotrain/organize_clock_velocity_full_separation": "real/stationary_velocity", - "abc_arc/cotrain/organize_clock_velocity_translation_rotation": "real/stationary_velocity", + "abc_arc/cotrain/organize_clock_duration_full_separation": "arc_real/stationary_duration", + "abc_arc/cotrain/organize_clock_duration_translation_rotation": "arc_real/stationary_duration", + "abc_arc/cotrain/organize_clock_velocity_full_separation": "arc_real/stationary_velocity", + "abc_arc/cotrain/organize_clock_velocity_translation_rotation": "arc_real/stationary_velocity", "abc_arc/cotrain/organize_rl2_elmo_hpt180_cotrain_baseline": null, "abc_arc/cotrain/organize_rl2_elmo_hpt180_cotrain_multistream": null, - "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_baseline": "real/stationary_time", - "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream": "real/stationary_velocity", - "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream_logclock": "real/stationary_duration", - "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream_perwaypoint": "real/stationary_velocity", + "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_baseline": "arc_real/stationary_time", + "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream": "arc_real/stationary_velocity", + "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream_logclock": "arc_real/stationary_duration", + "abc_arc/cotrain/organize_rl2_elmo_hpt300_cotrain_multistream_perwaypoint": "arc_real/stationary_velocity", "abc_arc/robot_bc/abc_multitask4_abcdit_baseline_visual_openloop": "abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop", "abc_arc/robot_bc/abc_multitask4_abcdit_hybrid_visual_openloop": "abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop", "abc_arc/robot_bc/abc_multitask4_hpt300_baseline_visual_openloop": "abc_arc/robot_bc/multitask/baseline", @@ -19,12 +19,12 @@ "abc_arc/robot_bc/abc_rl2_towels_hpt180_multistream_visual_openloop": null, "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", "abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop": "abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop", - "abc_arc/robot_bc/abc_visual_hpt300_base": "_shared/visual_hpt300", + "abc_arc/robot_bc/abc_visual_hpt300_base": null, "abc_arc/robot_bc/stationery_rl2_hpt300_visual_baseline_openloop": null, "abc_arc/robot_bc/stationery_rl2_hpt300_visual_hybrid_openloop": null, - "fold_and_stack_towels/arc_dur": "real/towels_duration", - "fold_and_stack_towels/arc_vel": "real/towels_velocity", - "fold_and_stack_towels/baseline": "real/towels_time", + "fold_and_stack_towels/arc_dur": "arc_real/towels_duration", + "fold_and_stack_towels/arc_vel": "arc_real/towels_velocity", + "fold_and_stack_towels/baseline": "arc_real/towels_time", "fold_and_stack_towels/variants/abc_rl2_towels_abcdit_baseline_visual_openloop": null, "fold_and_stack_towels/variants/abc_rl2_towels_abcdit_multistream_logclock_visual_openloop": null, "fold_and_stack_towels/variants/abc_rl2_towels_abcdit_multistream_visual_openloop": null, @@ -32,122 +32,122 @@ "fold_and_stack_towels/variants/abc_towels_hpt180_baseline_visual_openloop": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", "fold_and_stack_towels/variants/abc_towels_hpt180_hybrid_visual_openloop": "abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop", "fold_and_stack_towels/variants/new290plus74_baseline": null, - "libero/action_flow_libero10_oat_dp_matched_s42": "sim/libero/action_flow_libero10_oat_dp_matched_s42", - "libero/action_flow_libero_goal_oat_dp_matched_s42": "sim/libero/action_flow_libero_goal_oat_dp_matched_s42", - "libero/action_flow_libero_object_oat_dp_matched_s42": "sim/libero/action_flow_libero_object_oat_dp_matched_s42", - "libero/action_flow_libero_spatial_oat_dp_matched_s42": "sim/libero/action_flow_libero_spatial_oat_dp_matched_s42", - "libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42": "sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", - "libero/action_flow_libero_spatial_sharpa_mot_s42": "sim/libero/action_flow_libero_spatial_sharpa_mot_s42", - "libero/dp_libero10_oat_batch16_keep_steps_s42": "sim/libero/dp_libero10_oat_batch16_keep_steps_s42", - "libero/dp_libero_goal_oat_batch16_keep_steps_s42": "sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42", - "libero/dp_libero_object_oat_batch16_keep_steps_s42": "sim/libero/dp_libero_object_oat_batch16_keep_steps_s42", - "libero/dp_libero_spatial_oat_batch16_keep_steps_s42": "sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42", - "libero_historical/action_flow_libero10_avln_80k_s42": "sim/libero_historical/action_flow_libero10_avln_80k_s42", - "libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42": "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", - "libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42": "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", - "libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42": "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", - "libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42": "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", - "libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42": "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", - "libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42": "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", - "libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42": "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", - "libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42": "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", - "libero_historical/dp_libero10_oat_dp_matched_s42": "sim/libero_historical/dp_libero10_oat_dp_matched_s42", - "libero_historical/dp_libero_goal_oat_dp_matched_s42": "sim/libero_historical/dp_libero_goal_oat_dp_matched_s42", - "libero_historical/dp_libero_object_oat_dp_matched_s42": "sim/libero_historical/dp_libero_object_oat_dp_matched_s42", - "libero_historical/dp_libero_spatial_oat_dp_matched_s42": "sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42", + "libero/action_flow_libero10_oat_dp_matched_s42": "arc_sim/libero/action_flow_libero10_oat_dp_matched_s42", + "libero/action_flow_libero_goal_oat_dp_matched_s42": "arc_sim/libero/action_flow_libero_goal_oat_dp_matched_s42", + "libero/action_flow_libero_object_oat_dp_matched_s42": "arc_sim/libero/action_flow_libero_object_oat_dp_matched_s42", + "libero/action_flow_libero_spatial_oat_dp_matched_s42": "arc_sim/libero/action_flow_libero_spatial_oat_dp_matched_s42", + "libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42": "arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", + "libero/action_flow_libero_spatial_sharpa_mot_s42": "arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42", + "libero/dp_libero10_oat_batch16_keep_steps_s42": "arc_sim/libero/dp_libero10_oat_batch16_keep_steps_s42", + "libero/dp_libero_goal_oat_batch16_keep_steps_s42": "arc_sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42", + "libero/dp_libero_object_oat_batch16_keep_steps_s42": "arc_sim/libero/dp_libero_object_oat_batch16_keep_steps_s42", + "libero/dp_libero_spatial_oat_batch16_keep_steps_s42": "arc_sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42", + "libero_historical/action_flow_libero10_avln_80k_s42": "arc_sim/libero_historical/action_flow_libero10_avln_80k_s42", + "libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42": "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", + "libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42": "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", + "libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42": "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", + "libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42": "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", + "libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42": "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", + "libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42": "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", + "libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42": "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", + "libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42": "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", + "libero_historical/dp_libero10_oat_dp_matched_s42": "arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42", + "libero_historical/dp_libero_goal_oat_dp_matched_s42": "arc_sim/libero_historical/dp_libero_goal_oat_dp_matched_s42", + "libero_historical/dp_libero_object_oat_dp_matched_s42": "arc_sim/libero_historical/dp_libero_object_oat_dp_matched_s42", + "libero_historical/dp_libero_spatial_oat_dp_matched_s42": "arc_sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42", "multitask/arc_dur": "abc_arc/robot_bc/multitask/arc_dur", "multitask/arc_vel": "abc_arc/robot_bc/multitask/arc_vel", "multitask/baseline": "abc_arc/robot_bc/multitask/baseline", "multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop": "abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop", "multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop": "abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop", - "oat/libero_arc_dur_1_r192_d1p6_m24": "sim/oat/libero_arc_dur_1_r192_d1p6_m24", - "oat/libero_arc_dur_2_r384_d0p8_m32": "sim/oat/libero_arc_dur_2_r384_d0p8_m32", - "oat/libero_arc_dur_policy": "sim/oat/libero_arc_dur_policy", - "oat/libero_arc_global_chebyshev": "sim/oat/libero_arc_global_chebyshev", - "oat/libero_arc_global_common": "sim/oat/libero_arc_global_common", - "oat/libero_arc_global_fourier": "sim/oat/libero_arc_global_fourier", - "oat/libero_arc_global_pilot": "sim/oat/libero_arc_global_pilot", - "oat/libero_arc_global_uniform": "sim/oat/libero_arc_global_uniform", - "oat/libero_arc_oat_base": "sim/oat/libero_arc_oat_base", - "oat/libero_arc_oat_dp_policy": "sim/oat/libero_arc_oat_dp_policy", - "oat/libero_arc_oatpolicy": "sim/oat/libero_arc_oatpolicy", - "oat/libero_arc_oattok": "sim/oat/libero_arc_oattok", - "oat/libero_arc_policy": "sim/oat/libero_arc_policy", - "oat/libero_arc_shared_r384_d1p6_m36": "sim/oat/libero_arc_shared_r384_d1p6_m36", - "oat/libero_arc_stk_1_r192_d1p6_m36": "sim/oat/libero_arc_stk_1_r192_d1p6_m36", - "oat/libero_arc_stk_2_r192_d0p8_m32": "sim/oat/libero_arc_stk_2_r192_d0p8_m32", - "oat/libero_arc_stk_policy": "sim/oat/libero_arc_stk_policy", - "oat/libero_base": "sim/oat/libero_base", - "oat/libero_oatpolicy": "sim/oat/libero_oatpolicy", - "oat/libero_oattok": "sim/oat/libero_oattok", - "organize_stationary/_base/arc": "real/stationary_velocity", - "organize_stationary/arc_dur": "real/stationary_duration", - "organize_stationary/arc_vel": "real/stationary_velocity", - "organize_stationary/baseline": "real/stationary_time", + "oat/libero_arc_dur_1_r192_d1p6_m24": "arc_sim/oat/libero_arc_dur_1_r192_d1p6_m24", + "oat/libero_arc_dur_2_r384_d0p8_m32": "arc_sim/oat/libero_arc_dur_2_r384_d0p8_m32", + "oat/libero_arc_dur_policy": "arc_sim/oat/libero_arc_dur_policy", + "oat/libero_arc_global_chebyshev": "arc_sim/oat/libero_arc_global_chebyshev", + "oat/libero_arc_global_common": "arc_sim/oat/libero_arc_global_common", + "oat/libero_arc_global_fourier": "arc_sim/oat/libero_arc_global_fourier", + "oat/libero_arc_global_pilot": "arc_sim/oat/libero_arc_global_pilot", + "oat/libero_arc_global_uniform": "arc_sim/oat/libero_arc_global_uniform", + "oat/libero_arc_oat_base": "arc_sim/oat/libero_arc_oat_base", + "oat/libero_arc_oat_dp_policy": "arc_sim/oat/libero_arc_oat_dp_policy", + "oat/libero_arc_oatpolicy": "arc_sim/oat/libero_arc_oatpolicy", + "oat/libero_arc_oattok": "arc_sim/oat/libero_arc_oattok", + "oat/libero_arc_policy": "arc_sim/oat/libero_arc_policy", + "oat/libero_arc_shared_r384_d1p6_m36": "arc_sim/oat/libero_arc_shared_r384_d1p6_m36", + "oat/libero_arc_stk_1_r192_d1p6_m36": "arc_sim/oat/libero_arc_stk_1_r192_d1p6_m36", + "oat/libero_arc_stk_2_r192_d0p8_m32": "arc_sim/oat/libero_arc_stk_2_r192_d0p8_m32", + "oat/libero_arc_stk_policy": "arc_sim/oat/libero_arc_stk_policy", + "oat/libero_base": "arc_sim/oat/libero_base", + "oat/libero_oatpolicy": "arc_sim/oat/libero_oatpolicy", + "oat/libero_oattok": "arc_sim/oat/libero_oattok", + "organize_stationary/_base/arc": "arc_real/stationary_velocity", + "organize_stationary/arc_dur": "arc_real/stationary_duration", + "organize_stationary/arc_vel": "arc_real/stationary_velocity", + "organize_stationary/baseline": "arc_real/stationary_time", "organize_stationary/variants/organize_rl2_elmo_hpt180_cotrain_baseline": null, "organize_stationary/variants/organize_rl2_elmo_hpt180_cotrain_multistream": null, "organize_stationary/variants/stationery_rl2_hpt300_visual_baseline_openloop": null, "organize_stationary/variants/stationery_rl2_hpt300_visual_hybrid_openloop": null, - "pusht/action_flow_bc_usocket_bridge_likelihood_s42": "sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42", - "pusht/action_flow_bc_usocket_graph_section_s42": "sim/pusht/action_flow_bc_usocket_graph_section_s42", - "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42": "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42", - "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42": "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42", - "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42", - "pusht/action_flow_bc_usocket_recon100_s42": "sim/pusht/action_flow_bc_usocket_recon100_s42", - "pusht/action_flow_bc_usocket_recon10_s42": "sim/pusht/action_flow_bc_usocket_recon10_s42", - "pusht/action_flow_bc_usocket_recon10_warmup10k_s42": "sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42", - "pusht/action_flow_bc_usocket_recon1_s42": "sim/pusht/action_flow_bc_usocket_recon1_s42", - "pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42": "sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42", - "pusht/action_flow_cotrain_uc_multiplier_interpolation": "sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", - "pusht/action_flow_usocket_candidate_common": "sim/pusht/action_flow_usocket_candidate_common", - "pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42": "sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42", - "pusht/action_latent_vfm_usocket_val01_h16": "sim/pusht/action_latent_vfm_usocket_val01_h16", - "pusht/parents/action_flow_cotrain_uc_manual4919": "sim/pusht/parents/action_flow_cotrain_uc_manual4919", - "pusht/parents/action_flow_usocket_h384": "sim/pusht/parents/action_flow_usocket_h384", - "pusht/parents/action_flow_usocket_h384_sum14": "sim/pusht/parents/action_flow_usocket_h384_sum14", - "pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier": "sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier", - "pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked": "sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", - "pusht/planar_v2_base": "sim/pusht/planar_v2_base", - "pusht/planar_v2_chain_arc_bc": "sim/pusht/planar_v2_chain_arc_bc", - "pusht/planar_v2_chain_base": "sim/pusht/planar_v2_chain_base", - "pusht/planar_v2_chain_direct_bc": "sim/pusht/planar_v2_chain_direct_bc", - "pusht/planar_v2_chain_dp_paper": "sim/pusht/planar_v2_chain_dp_paper", - "pusht/planar_v2_chain_dp_standard": "sim/pusht/planar_v2_chain_dp_standard", - "pusht/planar_v2_cotrain_clean_dp_standard": "sim/pusht/planar_v2_cotrain_clean_dp_standard", - "pusht/planar_v2_usocket_arc_bc": "sim/pusht/planar_v2_usocket_arc_bc", - "pusht/planar_v2_usocket_arc_graph_tok": "sim/pusht/planar_v2_usocket_arc_graph_tok", - "pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc": "sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc", - "pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg": "sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg", - "pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg": "sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", - "pusht/planar_v2_usocket_base": "sim/pusht/planar_v2_usocket_base", - "pusht/planar_v2_usocket_direct_bc": "sim/pusht/planar_v2_usocket_direct_bc", - "pusht/planar_v2_usocket_dp_paper": "sim/pusht/planar_v2_usocket_dp_paper", - "pusht/planar_v2_usocket_dp_standard": "sim/pusht/planar_v2_usocket_dp_standard", - "pusht/unite_cotrain_usocket_chain_val01_h16": "sim/pusht/unite_cotrain_usocket_chain_val01_h16", - "pusht/unite_usocket_register_sweep_val01_h16": "sim/pusht/unite_usocket_register_sweep_val01_h16", - "synthetic/direct_fm3d_gaussian_paraboloid": "sim/synthetic/direct_fm3d_gaussian_paraboloid", - "synthetic/direct_fm3d_gaussian_torus": "sim/synthetic/direct_fm3d_gaussian_torus", - "synthetic/unite_gaussian_torus": "sim/synthetic/unite_gaussian_torus", - "synthetic/unite_gaussian_torus_latent2_codec10k": "sim/synthetic/unite_gaussian_torus_latent2_codec10k", - "synthetic/unite_gaussian_torus_latent4": "sim/synthetic/unite_gaussian_torus_latent4", - "synthetic/unite_gaussian_torus_latent4_codec10k": "sim/synthetic/unite_gaussian_torus_latent4_codec10k", - "synthetic/vfm_gaussian_torus": "sim/synthetic/vfm_gaussian_torus", - "synthetic/vfm_gaussian_torus_latent2_codec10k": "sim/synthetic/vfm_gaussian_torus_latent2_codec10k", - "synthetic/vfm_gaussian_torus_latent4": "sim/synthetic/vfm_gaussian_torus_latent4", - "synthetic/vfm_gaussian_torus_latent4_codec10k": "sim/synthetic/vfm_gaussian_torus_latent4_codec10k", - "wireless_earbuds/arc_dur": "real/earbuds_duration", - "wireless_earbuds/arc_vel": "real/earbuds_velocity", - "wireless_earbuds/baseline": "real/earbuds_time", - "yam_arc_grid/_base/earbuds_updated_hpt300": "real/earbuds_time", - "yam_arc_grid/_base/stattempo": "_shared/frozen_yam/stattempo", - "yam_arc_grid/_base/stattempo_openloop": "_shared/frozen_yam/stattempo_openloop", - "yam_arc_grid/_base/towels_hpt300": "real/towels_time", - "yam_arc_grid/bc_rl2_stattempo_slowpace366_arcdur_abstime_m200_d033": "real/stationary_duration", - "yam_arc_grid/bc_rl2_stattempo_sp366e51_arcdur_abstime_m200_d022": "real/stationary_duration", - "yam_arc_grid/earbuds_plus49_arcdur_abstime_m200_d022_hpt300": "real/earbuds_duration", - "yam_arc_grid/earbuds_updated_arcdur_abstime_m200_d033_hpt300": "real/earbuds_duration", - "yam_arc_grid/earbuds_updated_time_hpt300": "real/earbuds_time", + "pusht/action_flow_bc_usocket_bridge_likelihood_s42": "arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42", + "pusht/action_flow_bc_usocket_graph_section_s42": "arc_sim/pusht/action_flow_bc_usocket_graph_section_s42", + "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42": "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42", + "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42": "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42", + "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42", + "pusht/action_flow_bc_usocket_recon100_s42": "arc_sim/pusht/action_flow_bc_usocket_recon100_s42", + "pusht/action_flow_bc_usocket_recon10_s42": "arc_sim/pusht/action_flow_bc_usocket_recon10_s42", + "pusht/action_flow_bc_usocket_recon10_warmup10k_s42": "arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42", + "pusht/action_flow_bc_usocket_recon1_s42": "arc_sim/pusht/action_flow_bc_usocket_recon1_s42", + "pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42": "arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42", + "pusht/action_flow_cotrain_uc_multiplier_interpolation": "arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", + "pusht/action_flow_usocket_candidate_common": "arc_sim/pusht/action_flow_usocket_candidate_common", + "pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42": "arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42", + "pusht/action_latent_vfm_usocket_val01_h16": "arc_sim/pusht/action_latent_vfm_usocket_val01_h16", + "pusht/parents/action_flow_cotrain_uc_manual4919": "arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919", + "pusht/parents/action_flow_usocket_h384": "arc_sim/pusht/parents/action_flow_usocket_h384", + "pusht/parents/action_flow_usocket_h384_sum14": "arc_sim/pusht/parents/action_flow_usocket_h384_sum14", + "pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier": "arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier", + "pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked": "arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", + "pusht/planar_v2_base": "arc_sim/pusht/planar_v2_base", + "pusht/planar_v2_chain_arc_bc": "arc_sim/pusht/planar_v2_chain_arc_bc", + "pusht/planar_v2_chain_base": "arc_sim/pusht/planar_v2_chain_base", + "pusht/planar_v2_chain_direct_bc": "arc_sim/pusht/planar_v2_chain_direct_bc", + "pusht/planar_v2_chain_dp_paper": "arc_sim/pusht/planar_v2_chain_dp_paper", + "pusht/planar_v2_chain_dp_standard": "arc_sim/pusht/planar_v2_chain_dp_standard", + "pusht/planar_v2_cotrain_clean_dp_standard": "arc_sim/pusht/planar_v2_cotrain_clean_dp_standard", + "pusht/planar_v2_usocket_arc_bc": "arc_sim/pusht/planar_v2_usocket_arc_bc", + "pusht/planar_v2_usocket_arc_graph_tok": "arc_sim/pusht/planar_v2_usocket_arc_graph_tok", + "pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc": "arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc", + "pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg": "arc_sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg", + "pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg": "arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", + "pusht/planar_v2_usocket_base": "arc_sim/pusht/planar_v2_usocket_base", + "pusht/planar_v2_usocket_direct_bc": "arc_sim/pusht/planar_v2_usocket_direct_bc", + "pusht/planar_v2_usocket_dp_paper": "arc_sim/pusht/planar_v2_usocket_dp_paper", + "pusht/planar_v2_usocket_dp_standard": "arc_sim/pusht/planar_v2_usocket_dp_standard", + "pusht/unite_cotrain_usocket_chain_val01_h16": "arc_sim/pusht/unite_cotrain_usocket_chain_val01_h16", + "pusht/unite_usocket_register_sweep_val01_h16": "arc_sim/pusht/unite_usocket_register_sweep_val01_h16", + "synthetic/direct_fm3d_gaussian_paraboloid": "arc_sim/synthetic/direct_fm3d_gaussian_paraboloid", + "synthetic/direct_fm3d_gaussian_torus": "arc_sim/synthetic/direct_fm3d_gaussian_torus", + "synthetic/unite_gaussian_torus": "arc_sim/synthetic/unite_gaussian_torus", + "synthetic/unite_gaussian_torus_latent2_codec10k": "arc_sim/synthetic/unite_gaussian_torus_latent2_codec10k", + "synthetic/unite_gaussian_torus_latent4": "arc_sim/synthetic/unite_gaussian_torus_latent4", + "synthetic/unite_gaussian_torus_latent4_codec10k": "arc_sim/synthetic/unite_gaussian_torus_latent4_codec10k", + "synthetic/vfm_gaussian_torus": "arc_sim/synthetic/vfm_gaussian_torus", + "synthetic/vfm_gaussian_torus_latent2_codec10k": "arc_sim/synthetic/vfm_gaussian_torus_latent2_codec10k", + "synthetic/vfm_gaussian_torus_latent4": "arc_sim/synthetic/vfm_gaussian_torus_latent4", + "synthetic/vfm_gaussian_torus_latent4_codec10k": "arc_sim/synthetic/vfm_gaussian_torus_latent4_codec10k", + "wireless_earbuds/arc_dur": "arc_real/earbuds_duration", + "wireless_earbuds/arc_vel": "arc_real/earbuds_velocity", + "wireless_earbuds/baseline": "arc_real/earbuds_time", + "yam_arc_grid/_base/earbuds_updated_hpt300": "arc_real/earbuds_time", + "yam_arc_grid/_base/stattempo": null, + "yam_arc_grid/_base/stattempo_openloop": null, + "yam_arc_grid/_base/towels_hpt300": "arc_real/towels_time", + "yam_arc_grid/bc_rl2_stattempo_slowpace366_arcdur_abstime_m200_d033": "arc_real/stationary_duration", + "yam_arc_grid/bc_rl2_stattempo_sp366e51_arcdur_abstime_m200_d022": "arc_real/stationary_duration", + "yam_arc_grid/earbuds_plus49_arcdur_abstime_m200_d022_hpt300": "arc_real/earbuds_duration", + "yam_arc_grid/earbuds_updated_arcdur_abstime_m200_d033_hpt300": "arc_real/earbuds_duration", + "yam_arc_grid/earbuds_updated_time_hpt300": "arc_real/earbuds_time", "yam_arc_grid/towels_new290plus74_time_hpt300": null, - "yam_arc_grid/towels_since0924plus74_arcdur_m200_d033_hpt300": "real/towels_duration", - "yam_arc_grid/towels_since0924plus74_time_hpt300": "real/towels_time" + "yam_arc_grid/towels_since0924plus74_arcdur_m200_d033_hpt300": "arc_real/towels_duration", + "yam_arc_grid/towels_since0924plus74_time_hpt300": "arc_real/towels_time" } diff --git a/docs/experiments/action-flow-merge-readiness.md b/docs/experiments/action-flow-merge-readiness.md index 332b85ed35..439333bb70 100644 --- a/docs/experiments/action-flow-merge-readiness.md +++ b/docs/experiments/action-flow-merge-readiness.md @@ -44,8 +44,9 @@ within an arbitrary grace period. The old failed ICE smoke remains failed; run a readiness-aware scheduled smoke before relying on preemption recovery for a new cluster run. No training jobs or merge into main were requested here. -GitHub access uses Skynet's authenticated `ElmoPA` identity through `sky2`, the -working fallback after `sky1` stalled. Publication includes the previously +For that historical publication, GitHub access used the `ElmoPA` identity through +`sky2` after `sky1` stalled; this is provenance, not an account requirement for +future submissions. Publication includes the previously unpublished gradient audit, sphere/cube experiment, stop-gradient ablation, and requeue commits. Existing experimental checkpoints remain bound to their original source revisions. diff --git a/docs/integration/CONFIG_DEDUPLICATION.md b/docs/integration/CONFIG_DEDUPLICATION.md index 32f1f29047..00b6e44c41 100644 --- a/docs/integration/CONFIG_DEDUPLICATION.md +++ b/docs/integration/CONFIG_DEDUPLICATION.md @@ -1,5 +1,11 @@ # Model configuration consolidation: first local layer +> Historical October 1 audit, not the current config inventory. Its alias +> names, counts and review status describe that source snapshot. Current +> family folders, shared bases and task entry points are documented in +> [CONFIGURATION.md](../CONFIGURATION.md). Preserve this report as evidence; +> do not use its old selectors for new runs. + This is a partial implementation of the October 1 consolidation handoff. The complete A/#197, B/#159 and C/#160 integration has **not** been assembled, reviewed, pushed or merged. Main and the published source branches are unchanged. @@ -28,11 +34,11 @@ available, while 1,356 duplicated YAML lines are removed. | Compatibility name (`model=`) | Canonical definition (`model=`) | | --- | --- | -| `hpt_bc_flow_aria` | `hpt_bc_flow_human` | -| `hpt_cotrain_scale_flow_shared_head` | `hpt_cotrain_mecka_flow_shared_head` | -| `pi0.5_bc_scale` | `pi0.5_bc_mecka` | -| `pi0.5_cotrain_mecka_scale` | `pi0.5_bc_mecka` | -| `pi05/pi0.5_ft_abc_eva_6d` | `pi05/pi0.5_bc_abc_eva_6d` | +| `hpt/bc_flow_human` | `hpt/bc_flow_human` | +| `hpt/cotrain_mecka_flow_shared_head` | `hpt/cotrain_mecka_flow_shared_head` | +| `pi/bc_mecka_legacy` | `pi/bc_mecka_legacy` | +| `pi/bc_mecka_legacy` | `pi/bc_mecka_legacy` | +| `pi/bc_abc_eva_6d` | `pi/bc_abc_eva_6d` | Live launch and normalization scripts use the canonical names. Dataset selection stays explicit; choosing a shared model does not change the selected diff --git a/docs/integration/DATA_TOOLS.md b/docs/integration/DATA_TOOLS.md index 90688f3d5d..6ad85b30e4 100644 --- a/docs/integration/DATA_TOOLS.md +++ b/docs/integration/DATA_TOOLS.md @@ -35,8 +35,9 @@ The old scripts depended on absent `rldb.utils` and `egomimicUtils` imports at the audit baseline; copying them would not preserve a runnable workflow. For wrist-frame targets, configure the renderer's `transform_list` explicitly. -Prediction-renderer selections such as `pi_cartesian_lang_wrist` remain usable -with `evaluator=eval_hpt_wrist`; keypoints use `eval_keypoints_wrist`. These +Prediction-renderer selections such as `evaluator/viz/cartesian_prompt` remain +usable with `evaluator=eval_cartesian_wrist`; keypoints use +`eval_keypoints_wrist`. These renderer selections expect the evaluator to perform coordinate reversion, as in the original runtime. They do not independently apply it a second time. diff --git a/docs/integration/MIGRATION.md b/docs/integration/MIGRATION.md index 5871735b5e..0f9c18f8c7 100644 --- a/docs/integration/MIGRATION.md +++ b/docs/integration/MIGRATION.md @@ -24,22 +24,32 @@ verification and separate Yam hardware environment. ## Select a complete recipe -The general `train_zarr_cartesian` entry point requires explicit model, data and -evaluator choices. For example, after configuring credentials and dataset paths: +Prefer a complete task entry point; its defaults explicitly select model, data, +evaluator, trainer, callbacks and logging. Inspect without launching: + +```bash +python egomimic/trainHydra.py \ + +experiment=arc_real/stationary_velocity --cfg job --resolve +``` + +See [the configuration guide](../CONFIGURATION.md) for the arc_real/arc_sim menus, +family-organized model configs and behavior-named evaluators. The general +`train_zarr_cartesian` entry point also supports explicit component selection. +For example, after configuring credentials and dataset paths: ```bash python -m egomimic.trainHydra --config-name=train_zarr_cartesian \ - model=hpt_bc_flow_eva data=eva evaluator=eval_hpt + model=hpt/bc_flow_eva data=eva evaluator=eval_cartesian_camera ``` The PI convenience entry point retains its concrete ABC/6D/wrist-frame default -and requires an explicit pretrained weight path. Root retained recipes can be +and requires an explicit pretrained weight path. Retained legacy recipes can be selected on the general entry point too: ```bash export EGOVERSE_PI05_WEIGHTS=/absolute/path/to/pi05_base_pytorch python -m egomimic.trainHydra --config-name=train_zarr_cartesian \ - model=pi0.5_bc_eva data=eva_pi evaluator=eval_pi + model=pi/bc_eva_legacy data=eva_pi evaluator=eval_cartesian_camera_prompt ``` PI weights/tokenizers must be available at the selected model's configured @@ -52,8 +62,9 @@ it contradicts that contract. The nested PI recipes expose `model.coordinate_frame=camframe|eef_frame`; it must match the data selection and the evaluator's frame reversion. The model declaration also chooses the deployment frame, so a station cannot silently reinterpret wrist-relative -actions as camera-frame actions. E1's `e1.policy_domain` is explicit experiment -metadata and binds its source identity and data contract. +actions as camera-frame actions. Contracts bind the selected model and data +semantics without inferring them from folder names. Removed E1 recipes must +use their original source; they are not current launch choices. The keypoint wrist entry point shares the retained keypoint architecture but selects a wrist-frame model contract. Nested Yam/ARC models declare padded diff --git a/docs/integration/RECIPE_PARITY.md b/docs/integration/RECIPE_PARITY.md index c0e6e12fca..f8d86654ac 100644 --- a/docs/integration/RECIPE_PARITY.md +++ b/docs/integration/RECIPE_PARITY.md @@ -26,7 +26,7 @@ Two stale source settings are corrected explicitly: `actions_joints`, while their shipped keymaps/transforms only produced Cartesian state/actions. The retained Cartesian recipes bind those actual keys. This is not a joint-action checkpoint converter. -- `hpt_cotrain_enc_dec_base` declared eight decoder context tokens against +- `base/hpt_cotrain_enc_dec` declared eight decoder context tokens against a 64-token trunk. Its deterministic decoder now declares 64 context tokens; its smooth-L1 loss retains beta 0.05. It rejects mismatched target shapes. diff --git a/docs/integration/THREE_SOURCE_MERGE.md b/docs/integration/THREE_SOURCE_MERGE.md index 534b954081..a0bf8a1b49 100644 --- a/docs/integration/THREE_SOURCE_MERGE.md +++ b/docs/integration/THREE_SOURCE_MERGE.md @@ -73,7 +73,7 @@ parity retest passes seven cases. The next full run reached a terminal result: ongoing repairs and is not a final-head gate. Its failures remain recorded. This runner does not change cluster execution or establish exact-runtime or historical-run parity. -Publication through authenticated Skynet Graphite and the final GitHub merge are +Publication through Graphite and the final GitHub merge are still incomplete. No final integration, deployment, training or GPU parity claim is made by this progress record. diff --git a/egomimic/benchmarks/libero/campaign.py b/egomimic/benchmarks/libero/campaign.py index c1d93e543b..7e858f22c9 100644 --- a/egomimic/benchmarks/libero/campaign.py +++ b/egomimic/benchmarks/libero/campaign.py @@ -28,7 +28,7 @@ def campaign_plan(data_root, output_root, python=sys.executable): python, "-m", "egomimic.trainHydra", - f"+experiment=sim/oat/{experiment}", + f"+experiment=arc_sim/oat/{experiment}", f"benchmark.suite={suite}", f"benchmark.dataset={dataset}", f"hydra.run.dir={outputs[name]}", diff --git a/egomimic/benchmarks/libero/cluster.py b/egomimic/benchmarks/libero/cluster.py index 46312d3695..913857be99 100644 --- a/egomimic/benchmarks/libero/cluster.py +++ b/egomimic/benchmarks/libero/cluster.py @@ -284,7 +284,7 @@ def training_arguments( sys.executable, "-m", "egomimic.trainHydra", - f"+experiment=sim/oat/{experiment}", + f"+experiment=arc_sim/oat/{experiment}", "hydra/launcher=basic", f"benchmark.suite={suite}", f"benchmark.dataset={dataset}", @@ -354,7 +354,7 @@ def global_basis_training_arguments(basis, dataset, evidence, mode, run_id, *, g cadence = 8 if mode == "smoke" else 2500 options.update( { - "+experiment": "sim/oat/libero_arc_global_pilot", + "+experiment": "arc_sim/oat/libero_arc_global_pilot", "benchmark.arc_basis": basis, "hydra.run.dir": str(run), "paths.output_dir": str(run), diff --git a/egomimic/benchmarks/libero/native_launch_profiles.py b/egomimic/benchmarks/libero/native_launch_profiles.py index 039b660a0e..bd784146aa 100644 --- a/egomimic/benchmarks/libero/native_launch_profiles.py +++ b/egomimic/benchmarks/libero/native_launch_profiles.py @@ -16,7 +16,7 @@ class NativeLaunchProfile: @property def experiment(self): - return "sim/libero_historical/" + self.name + return "arc_sim/libero_historical/" + self.name HISTORICAL_NATIVE_PROFILES = { @@ -73,7 +73,9 @@ def profile_for_experiment(experiment): def profile_for_config(config): - profile = profile_for_experiment("sim/libero_historical/" + str(config.get("name"))) + profile = profile_for_experiment( + "arc_sim/libero_historical/" + str(config.get("name")) + ) if config["benchmark"]["suite"] != profile.suite: raise ValueError("native suite/recipe identity mismatch") expected_ablation = ( @@ -128,7 +130,7 @@ class JointNativeLaunchProfile: @property def experiment(self): - return "sim/libero/" + self.name + return "arc_sim/libero/" + self.name SPATIAL_HAND_JOINT_PROFILE = JointNativeLaunchProfile( diff --git a/egomimic/hydra_configs/callbacks/real_yam.yaml b/egomimic/hydra_configs/callbacks/real_yam.yaml new file mode 100644 index 0000000000..bb8735a536 --- /dev/null +++ b/egomimic/hydra_configs/callbacks/real_yam.yaml @@ -0,0 +1,9 @@ +defaults: +- /callbacks/checkpoints@_here_ +- _self_ +model_checkpoint: + every_n_epochs: null + save_top_k: -1 + filename: epoch-{epoch:04d}-step-{step:08d} + every_n_train_steps: 30000 + auto_insert_metric_name: false diff --git a/egomimic/hydra_configs/data/_base/yam_arc.yaml b/egomimic/hydra_configs/data/_base/yam_arc.yaml new file mode 100644 index 0000000000..32a694d1d8 --- /dev/null +++ b/egomimic/hydra_configs/data/_base/yam_arc.yaml @@ -0,0 +1,10 @@ +train_datasets: + yam_bimanual: + resolver: + key_map: ${model.data_requirements.preprocessing.7.key_map} + transform_list: ${model.data_requirements.preprocessing.7.transforms} +valid_datasets: + yam_bimanual: + resolver: + key_map: ${model.data_requirements.preprocessing.7.key_map} + transform_list: ${model.data_requirements.preprocessing.7.transforms} diff --git a/egomimic/hydra_configs/data/abc_arc/abc_fstshirt_bc.yaml b/egomimic/hydra_configs/data/abc_arc/abc_fstshirt_bc.yaml index e275bd9cee..810a2b7819 100644 --- a/egomimic/hydra_configs/data/abc_arc/abc_fstshirt_bc.yaml +++ b/egomimic/hydra_configs/data/abc_arc/abc_fstshirt_bc.yaml @@ -6,9 +6,9 @@ # ABC-ONLY (no cotraining) time-indexed BC data: fold_and_stack_the_t_shirts, # yam_bimanual. # -# Retained for experiment/abc/yam_fstshirt_{dp,hpt}. Historical cotrain/ARC -# ablation templates were retired; the supported ARC experiments now live in -# experiment/abc_arc/{robot_bc,human_bc}. Keep this older baseline's 20-episode +# Reusable by the YAM diffusion and HPT models. The old experiment/abc menu +# is retired; supported run entrypoints now live in experiment/arc_real/. +# Keep this older baseline's 20-episode # sampling contract unchanged for its existing consumers. _target_: egomimic.rldb.zarr.data_module.ZarrDataModule _recursive_: false diff --git a/egomimic/hydra_configs/data/abc_arc/earbuds.yaml b/egomimic/hydra_configs/data/abc_arc/earbuds.yaml index df440d0c47..36af5e8569 100644 --- a/egomimic/hydra_configs/data/abc_arc/earbuds.yaml +++ b/egomimic/hydra_configs/data/abc_arc/earbuds.yaml @@ -9,7 +9,7 @@ train_datasets: yam_bimanual: resolver: transform_list: - variant: ${frozen_yam.variant} + variant: ${yam.variant} filters: filter_lambdas: - 'lambda row, V=frozenset({''2026-10-11-16-08-23-846000'',''2026-10-11-16-44-43-228000'',''2026-10-11-16-45-21-385000'',''2026-10-11-17-01-14-429000'',''2026-10-13-00-15-32-623000'',''2026-10-13-00-37-11-807000'',''2026-10-13-00-44-29-976000'',''2026-10-13-20-25-12-391000'',''2026-10-13-20-45-26-035000'',''2026-10-13-20-59-59-437000'',''2026-10-13-21-08-07-480000'',''2026-10-14-00-52-22-131000'',''2026-10-14-01-24-05-208000'',''2026-10-14-04-30-07-925000'',''2026-10-14-04-50-28-570000''}): @@ -25,7 +25,7 @@ valid_datasets: yam_bimanual: resolver: transform_list: - variant: ${frozen_yam.variant} + variant: ${yam.variant} filters: filter_lambdas: - 'lambda row, V=frozenset({''2026-10-11-16-08-23-846000'',''2026-10-11-16-44-43-228000'',''2026-10-11-16-45-21-385000'',''2026-10-11-17-01-14-429000'',''2026-10-13-00-15-32-623000'',''2026-10-13-00-37-11-807000'',''2026-10-13-00-44-29-976000'',''2026-10-13-20-25-12-391000'',''2026-10-13-20-45-26-035000'',''2026-10-13-20-59-59-437000'',''2026-10-13-21-08-07-480000'',''2026-10-14-00-52-22-131000'',''2026-10-14-01-24-05-208000'',''2026-10-14-04-30-07-925000'',''2026-10-14-04-50-28-570000''}): diff --git a/egomimic/hydra_configs/data/abc_arc/earbuds_arc.yaml b/egomimic/hydra_configs/data/abc_arc/earbuds_arc.yaml new file mode 100644 index 0000000000..e699791aab --- /dev/null +++ b/egomimic/hydra_configs/data/abc_arc/earbuds_arc.yaml @@ -0,0 +1,4 @@ +defaults: +- /data/abc_arc/earbuds@_here_ +- /data/_base/yam_arc@_here_ +- _self_ diff --git a/egomimic/hydra_configs/data/abc_arc/stationary_arc.yaml b/egomimic/hydra_configs/data/abc_arc/stationary_arc.yaml new file mode 100644 index 0000000000..4ce8871a2e --- /dev/null +++ b/egomimic/hydra_configs/data/abc_arc/stationary_arc.yaml @@ -0,0 +1,4 @@ +defaults: +- /data/abc_arc/stationary@_here_ +- /data/_base/yam_arc@_here_ +- _self_ diff --git a/egomimic/hydra_configs/data/abc_arc/towels.yaml b/egomimic/hydra_configs/data/abc_arc/towels.yaml index d5a6a4b699..f5da2ae9ec 100644 --- a/egomimic/hydra_configs/data/abc_arc/towels.yaml +++ b/egomimic/hydra_configs/data/abc_arc/towels.yaml @@ -9,7 +9,7 @@ train_datasets: yam_bimanual: resolver: transform_list: - variant: ${frozen_yam.variant} + variant: ${yam.variant} filters: filter_lambdas: - 'lambda row, V=frozenset({''2026-09-24-23-51-03-634000'',''2026-09-24-23-57-00-028000'',''2026-09-25-01-06-37-195000'',''2026-10-06-23-40-47-843000'',''2026-10-07-02-32-49-094000'',''2026-10-07-03-51-16-530000'',''2026-10-07-04-13-01-063000'',''2026-10-07-04-20-33-929000'',''2026-10-07-04-22-32-444000'',''2026-10-07-04-47-47-787000'',''2026-10-07-23-17-42-659000'',''2026-10-08-19-55-54-515000'',''2026-10-08-20-13-56-804000'',''2026-10-08-20-28-07-741000'',''2026-10-08-20-39-24-864000'',''2026-10-08-20-51-59-155000'',''2026-10-08-20-57-27-696000''}): @@ -25,7 +25,7 @@ valid_datasets: yam_bimanual: resolver: transform_list: - variant: ${frozen_yam.variant} + variant: ${yam.variant} filters: filter_lambdas: - 'lambda row, V=frozenset({''2026-09-24-23-51-03-634000'',''2026-09-24-23-57-00-028000'',''2026-09-25-01-06-37-195000'',''2026-10-06-23-40-47-843000'',''2026-10-07-02-32-49-094000'',''2026-10-07-03-51-16-530000'',''2026-10-07-04-13-01-063000'',''2026-10-07-04-20-33-929000'',''2026-10-07-04-22-32-444000'',''2026-10-07-04-47-47-787000'',''2026-10-07-23-17-42-659000'',''2026-10-08-19-55-54-515000'',''2026-10-08-20-13-56-804000'',''2026-10-08-20-28-07-741000'',''2026-10-08-20-39-24-864000'',''2026-10-08-20-51-59-155000'',''2026-10-08-20-57-27-696000''}): diff --git a/egomimic/hydra_configs/data/abc_arc/towels_arc.yaml b/egomimic/hydra_configs/data/abc_arc/towels_arc.yaml new file mode 100644 index 0000000000..7c070fc7c2 --- /dev/null +++ b/egomimic/hydra_configs/data/abc_arc/towels_arc.yaml @@ -0,0 +1,4 @@ +defaults: +- /data/abc_arc/towels@_here_ +- /data/_base/yam_arc@_here_ +- _self_ diff --git a/egomimic/hydra_configs/data/cotrain_pi_pickplace_qwen.yaml b/egomimic/hydra_configs/data/cotrain_pi_pickplace_qwen.yaml deleted file mode 100644 index d4182332c9..0000000000 --- a/egomimic/hydra_configs/data/cotrain_pi_pickplace_qwen.yaml +++ /dev/null @@ -1,86 +0,0 @@ -# Retained selection and split from EgoVerse ec5c903c; graph data adapter. -_target_: egomimic.rldb.zarr.data_module.ZarrDataModule -train_datasets: - eva_bimanual: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.MultiDataset._from_resolver - resolver: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.S3AnnotationCutoffEpisodeResolver - folder_path: /storage/project/r-dxu345-0/paphiwetsa3/datasets/pickplace_qwen - key_map: - _target_: egomimic.rldb.embodiment.eva.Eva.get_keymap - keymap_mode: cartesian - annotation_key: annotations - transform_list: - _target_: egomimic.rldb.embodiment.eva.Eva.get_transform_list - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - filters: - _target_: egomimic.rldb.filters.ScaleAnnotationDatasetFilter - project_name: dense-language - filter_lambdas: - - 'lambda row: row.get(''task'') == ''pick_place'' and row.get(''embodiment'') - == ''eva_bimanual'' and (row.get(''zarr_processed_path'') or '''') != '''' - and ''alignment'' not in ((row.get(''task_description'') or '''').lower())' - mode: total - human_bimanual: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.MultiDataset._from_resolver - resolver: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.S3AnnotationCutoffEpisodeResolver - folder_path: /storage/project/r-dxu345-0/paphiwetsa3/datasets/pickplace_qwen - key_map: - _target_: egomimic.rldb.embodiment.human.Human.get_keymap - keymap_mode: cartesian - annotation_key: annotations - transform_list: - _target_: egomimic.rldb.embodiment.human.Human.get_transform_list - stride: 3 - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - filters: - _target_: egomimic.rldb.filters.ScaleAnnotationDatasetFilter - project_name: dense-language - filter_lambdas: - - 'lambda row: row.get(''task'') == ''pick_place'' and row.get(''embodiment'') - == ''human_bimanual'' and (row.get(''zarr_processed_path'') or '''') != '''' - and any(s in ((row.get(''task_description'') or '''').lower()) for s in (''base'', - ''object'')) and (row.get(''episode_hash'') or '''') not in (''2026-04-26-00-17-53-000000'', - ''2026-04-26-00-27-26-000000'', ''2026-05-01-02-52-58-000000'')' - mode: total -valid_datasets: - eva_bimanual: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.MultiDataset._from_resolver - resolver: - _target_: egomimic.rldb.zarr.zarr_dataset_multi.S3AnnotationCutoffEpisodeResolver - folder_path: /storage/project/r-dxu345-0/paphiwetsa3/datasets/pickplace_qwen - key_map: - _target_: egomimic.rldb.embodiment.eva.Eva.get_keymap - keymap_mode: cartesian - annotation_key: annotations - transform_list: - _target_: egomimic.rldb.embodiment.eva.Eva.get_transform_list - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - filters: - _target_: egomimic.rldb.filters.ScaleAnnotationDatasetFilter - project_name: dense-language - filter_lambdas: - - 'lambda row: row.get(''task'') == ''pick_place'' and row.get(''embodiment'') - == ''eva_bimanual'' and (row.get(''zarr_processed_path'') or '''') != '''' - and ''alignment'' in ((row.get(''task_description'') or '''').lower())' - mode: total -train_dataloader_params: - eva_bimanual: - batch_size: 32 - num_workers: 6 - human_bimanual: - batch_size: 32 - num_workers: 6 -valid_dataloader_params: - eva_bimanual: - batch_size: 32 - num_workers: 6 -_recursive_: false -source_fps: 30.0 diff --git a/egomimic/hydra_configs/data/pi05/abc_eva_pi_6d.yaml b/egomimic/hydra_configs/data/pi05/abc_eva_pi_6d.yaml index 30e27f4ea7..e220ead2af 100644 --- a/egomimic/hydra_configs/data/pi05/abc_eva_pi_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/abc_eva_pi_6d.yaml @@ -5,7 +5,7 @@ # WRIST-frame cartesian actions with the continuous 6D rotation # representation (20D = xyz+6D+gripper per arm); the head/cam-frame proprio # ee_pose is 6D-encoded too. Eva's default stride applies. Pairs with -# model=pi0.5_bc_abc_eva_6d and evaluator=eval_pi_wristframe_6d. +# model=pi/bc_abc_eva_6d and evaluator=eval_pi_wristframe_6d. # # LEGACY EXTRINSICS NOTE: this config predates the extrinsics_key plumbing # and bakes the default x5Dec13_2 (rl2 lab rig) cam frame into the proprio — diff --git a/egomimic/hydra_configs/data/pi05/abc_stationery_pi_6d.yaml b/egomimic/hydra_configs/data/pi05/abc_stationery_pi_6d.yaml index c01bbd73d7..1bc4bc323a 100644 --- a/egomimic/hydra_configs/data/pi05/abc_stationery_pi_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/abc_stationery_pi_6d.yaml @@ -34,7 +34,7 @@ # the eval-viz overlay is not geometrically meaningful on ABC images (it # projects rl2yam-frame poses with the ABC camera's K); the arc-matched and # MSE metrics are unaffected, both sides go through the same transform. -# Pairs with model=pi0.5_bc_abc_eva_6d and evaluator=eval_pi_wristframe_6d. +# Pairs with model=pi/bc_abc_eva_6d and evaluator=eval_pi_wristframe_6d. # Train-set spot check for the TrainVizEvalVideo head (train_viz/ metrics, # videos_train_viz/). No shuffle -> same leading slice every val. # Graph port: train_viz is a nested validation group; episode filters retained. diff --git a/egomimic/hydra_configs/data/pi05/aria_fold_pi_6d_grip.yaml b/egomimic/hydra_configs/data/pi05/aria_fold_pi_6d_grip.yaml index 9fef3ada4d..696c3a4d7f 100644 --- a/egomimic/hydra_configs/data/pi05/aria_fold_pi_6d_grip.yaml +++ b/egomimic/hydra_configs/data/pi05/aria_fold_pi_6d_grip.yaml @@ -5,7 +5,7 @@ # grip channel derived from the MANO hand keypoints (pinch/curl heuristic, # KeypointsToGripper) inserted at the robot-layout slots 9/19 -> actions # AND proprio are 20D, layout-identical to eva robot data. Use the ROBOT -# converter (model=pi0.5_ft_aria_fold_grip). +# converter (model=pi/ft_aria_fold_grip). # - NO fix_mecka_left_wrist: verified on-data that raw aria already has the # target left-wrist convention (aria was the reference mecka was fixed # toward; applying the Rz(180) here would flip it wrong). diff --git a/egomimic/hydra_configs/data/pi05/cotrain_pi_lang_6d.yaml b/egomimic/hydra_configs/data/pi05/cotrain_pi_lang_6d.yaml index b5702934c7..6e7ed72e88 100644 --- a/egomimic/hydra_configs/data/pi05/cotrain_pi_lang_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/cotrain_pi_lang_6d.yaml @@ -1,7 +1,7 @@ # Same dense-language cotrain data as `cotrain_pi_lang`, but the cartesian action # chunk is expressed with the continuous 6D rotation representation # (xyz+6D per arm, +gripper for eva) instead of xyz+ypr. Pairs with the -# `cartesian_normalized_rot6d` action_encoding (model=pi0.5_cotrain_eva_aria_6d). +# `cartesian_normalized_rot6d` action_encoding (model=pi/cotrain_eva_aria_6d). # Valid datasets inherit the train resolver via `${...}` interpolation in # cotrain_pi_base, so overriding the train transform mode is sufficient. # Graph port: train_viz is a nested validation group; episode filters retained. diff --git a/egomimic/hydra_configs/data/pi05/mecka_all_pi_6d.yaml b/egomimic/hydra_configs/data/pi05/mecka_all_pi_6d.yaml index 0315ffa4d0..16787c03cc 100644 --- a/egomimic/hydra_configs/data/pi05/mecka_all_pi_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/mecka_all_pi_6d.yaml @@ -10,8 +10,8 @@ # mecka actions are recorded at the target rate (Aria uses 3). Language # prompts are sampled from the `annotations` track; prompt assembly # (proprio-in-prompt on, embodiment label off) lives in -# model=pi0.5_bc_mecka_6d, which this config pairs with, alongside -# evaluator=eval_pi_wristframe_6d (reverts wrist -> head frame via the +# model=pi/bc_mecka_6d, which this config pairs with, alongside +# evaluator=eval_cartesian_wrist_6d_prompt (reverts wrist -> head frame via the # proprio, then 6D -> ypr, for cam-frame MSE + viz). # # Validation/checkpoint cadence stays the epoch-based trainer default — no diff --git a/egomimic/hydra_configs/data/pi05/mecka_fold_flagship_pi_6d.yaml b/egomimic/hydra_configs/data/pi05/mecka_fold_flagship_pi_6d.yaml index 991d698385..42e1697cc5 100644 --- a/egomimic/hydra_configs/data/pi05/mecka_fold_flagship_pi_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/mecka_fold_flagship_pi_6d.yaml @@ -4,8 +4,8 @@ # distinct task label 'folding_clothes' and is excluded). # # Same wristframe-6D pipeline as mecka_all_pi_6d (left-wrist fix, padded -# 20-dim proprio, stride 1); pairs with model=pi0.5_bc_mecka_6d and -# evaluator=eval_pi_wristframe_6d (+ the train_viz second head). +# 20-dim proprio, stride 1); pairs with model=pi/bc_mecka_6d and +# evaluator=eval_cartesian_wrist_6d_prompt (+ the train_viz second head). # Train-set spot check for the TrainVizEvalVideo head (train_viz/ metrics, # videos_train_viz/). No shuffle -> same leading slice every val. # Graph port: train_viz is a nested validation group; episode filters retained. diff --git a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_6d.yaml b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_6d.yaml index 478d99beb8..7fcf7b3eaa 100644 --- a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_6d.yaml @@ -1,7 +1,7 @@ # 6D-rotation variant of the motion-generalization cotrain (obj_gen_pi_lang): # - cartesian action chunk expressed with the continuous 6D rotation # representation (xyz+6D per arm, +gripper for eva) via the `cartesian_6d` -# transform mode. Pairs with model=pi0.5_cotrain_eva_aria_6d +# transform mode. Pairs with model=pi/cotrain_eva_aria_6d # (action_encoding=cartesian_normalized_rot6d) and evaluator=eval_pi_camframe_6d. # - dataset switched from the plain Scale-annotation resolver to the # AnnotationCutoff resolver, which clamps each action chunk at the end of the diff --git a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe.yaml b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe.yaml index ece3d7f369..85e77b559d 100644 --- a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe.yaml +++ b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe.yaml @@ -1,7 +1,7 @@ # Wristframe variant of motion_gen_pi_lang: annotated-only eva + annotated-only # aria (base/object descriptions). Actions are expressed in each wrist's own # frame (cartesian_wristframe_ypr) instead of the head/camera frame. Pair with -# evaluator=eval_pi so the revert transform projects predictions back to cam +# evaluator=eval_cartesian_wrist_prompt so the revert transform projects predictions back to cam # frame for the viz video. # Graph port: train_viz is a nested validation group; episode filters retained. _target_: egomimic.rldb.zarr.data_module.ZarrDataModule diff --git a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe_6d.yaml b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe_6d.yaml index 8b1208c253..cefbe0189a 100644 --- a/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe_6d.yaml +++ b/egomimic/hydra_configs/data/pi05/obj_gen_pi_lang_wristframe_6d.yaml @@ -2,7 +2,7 @@ # cotrain (obj_gen_pi_lang_wristframe): # - actions in each wrist's own frame, rotation expressed as the continuous 6D # representation via the `cartesian_wristframe_6d` transform mode. Pairs with -# model=pi0.5_cotrain_eva_aria_6d and evaluator=eval_pi_wristframe_6d (which +# model=pi/cotrain_eva_aria_6d and evaluator=eval_cartesian_wrist_6d_prompt (which # un-6Ds then projects wrist-frame preds back to cam frame for viz/MSE). # - dataset switched to the AnnotationCutoff resolver (clamps each action chunk # at the end of its enclosing language-annotation span). diff --git a/egomimic/hydra_configs/evaluator/_base/cartesian_prompt.yaml b/egomimic/hydra_configs/evaluator/_base/cartesian_prompt.yaml new file mode 100644 index 0000000000..1d277df441 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/_base/cartesian_prompt.yaml @@ -0,0 +1,20 @@ +# Shared sampling/overlay settings for prompt-conditioned Cartesian outputs. +# Frame/rotation reversion is selected by the public evaluator preset. +defaults: + - /evaluator/viz@viz_func: cartesian_prompt + - _self_ +_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval +viz_func: + eva_bimanual: + fallback_intrinsics: + - [436.26, 0.0, 310.1, 0.0] + - [0.0, 435.13, 241.93, 0.0] + - [0.0, 0.0, 1.0, 0.0] +viz_every_n_epochs: 100 +image_key: observations.images.front_img_1 +pose_metrics: true +rkl_samples: ${model.pipeline.stages.0.policy.config.reverse_kl_samples} +group_options: + train_viz: + limit_batches: 50 + rkl_samples: 1 diff --git a/egomimic/hydra_configs/evaluator/_base/yam_human_cartesian.yaml b/egomimic/hydra_configs/evaluator/_base/yam_human_cartesian.yaml new file mode 100644 index 0000000000..d76f9cdcc5 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/_base/yam_human_cartesian.yaml @@ -0,0 +1,19 @@ +# Shared native-pose bindings; both metrics and videos use the same reversion. +defaults: + - /evaluator/viz@viz_func: cartesian/base + - _self_ +_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval +action_key: actions_cartesian +obs_pose_key: observations.state.ee_pose +image_key: observations.images.front_img_1 +revert_transforms: + yam_bimanual: + _target_: egomimic.rldb.embodiment.yam._build_yam_bimanual_revert_eef_frame_transform_list + is_quat: false + to_camera_frame: true + human_bimanual: + _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_eef_frame_transform_list + is_quat: false + gripper_padded: true +video_chunk_frames: 1000 +max_episode_frames: 6000 diff --git a/egomimic/hydra_configs/evaluator/eval_bimanual_cartesian.yaml b/egomimic/hydra_configs/evaluator/eval_bimanual_cartesian.yaml index 07a15b4f02..81e0617fa4 100644 --- a/egomimic/hydra_configs/evaluator/eval_bimanual_cartesian.yaml +++ b/egomimic/hydra_configs/evaluator/eval_bimanual_cartesian.yaml @@ -1,60 +1,8 @@ -# Cartesian bimanual validation for baseline (time-indexed) cotrain runs. -# -# Logs deterministic normalized MSE, native (metres/radians) MSE, and h264 mp4 -# overlay videos of predicted vs GT trajectories, buffered across the whole val -# epoch on rank 0. One mp4 per episode when the loader stamps ``episode_hash`` -# (it does for zarr sources); batches without it fall back to fixed-size -# ``validation_video_{i}.mp4`` chunks. Each finished mp4 is also logged to -# WandB via ``wandb.Video`` under ``Val_video/{embodiment}`` (or -# ``Val_video_{group}/{embodiment}`` off the default group). -# -# Valid/MSE macro mean of per-embodiment normalized MSE -# Valid/MSE/{embodiment} per-embodiment normalized MSE -# Valid/Native_MSE macro mean, unnormalized action-chunk MSE -# Valid/Native_MSE/{embodiment} per-embodiment native MSE -# -# This is the TIME-INDEXED (baseline) evaluator. ARC recipes select -# eval_open_loop_sim instead: an arc token's last row is a velocity, and -# reverting it to camera frame yields a position. - +# Time-indexed chunk MSE in normalized/native units, with Cartesian overlays. +# This does not simulate sequential execution or ARC replanning; real-task +# baselines and ARC experiments use eval_open_loop_sim for that evaluation. defaults: - - viz@viz_func: cartesian/base + - /evaluator/_base/yam_human_cartesian@_here_ - _self_ - -_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval - -action_key: actions_cartesian -obs_pose_key: observations.state.ee_pose -image_key: observations.images.front_img_1 - -# Per-embodiment revert transform lists. The evaluator applies these to both GT -# and predicted eef-frame chunks BEFORE handing them to ``viz_func``, so overlays -# are drawn in the front-camera frame that ``viz_gt_preds`` projects into. Each -# entry instantiates to the already-constructed list of transforms. -revert_transforms: - yam_bimanual: - _target_: egomimic.rldb.embodiment.yam._build_yam_bimanual_revert_eef_frame_transform_list - is_quat: false - to_camera_frame: true - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_eef_frame_transform_list - is_quat: false - gripper_padded: true - -# Base directory for mp4s. Left null so writes land under the trainer's -# ``default_root_dir/videos`` (matches the main EgoVerse layout). Override to -# ``${paths.output_dir}/val_videos`` if the run needs Hydra-rooted paths. video_output_dir: null - -# Fixed-size fallback: only hit when a batch has no ``episode_hash``. Zarr -# sources stamp it, so this path is mostly a safety net. -video_chunk_frames: 1000 - -# Per-episode safety cap: also the peak RAM bound on rank 0, roughly -# n_embodiments * max_episode_frames * H * W * 3 bytes -# (~5.5 GB per embodiment at 480x640 and 6000 frames). Episodes past this are -# written and truncated, not split. -max_episode_frames: 6000 - -# Pin the sampler for reproducibility across resumed runs. deterministic_seed: 420042 diff --git a/egomimic/hydra_configs/evaluator/eval_hpt.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_camera.yaml similarity index 83% rename from egomimic/hydra_configs/evaluator/eval_hpt.yaml rename to egomimic/hydra_configs/evaluator/eval_cartesian_camera.yaml index cf6d6d1848..5d06521335 100644 --- a/egomimic/hydra_configs/evaluator/eval_hpt.yaml +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_camera.yaml @@ -1,4 +1,4 @@ -# Retained command name; all models use the shared graph evaluator. +# Camera-frame Cartesian metrics, with ordinary elementwise angle MSE. defaults: - viz@viz_func: cartesian - _self_ diff --git a/egomimic/hydra_configs/evaluator/eval_cartesian_camera_6d_prompt.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_camera_6d_prompt.yaml new file mode 100644 index 0000000000..f0b53a4ad5 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_camera_6d_prompt.yaml @@ -0,0 +1,8 @@ +defaults: + - /evaluator/_base/cartesian_prompt@_here_ + - _self_ +revert_transforms: + eva_bimanual: + _target_: egomimic.rldb.embodiment.eva._build_eva_cartesian_revert_6d_transform_list + human_bimanual: + _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_transform_list diff --git a/egomimic/hydra_configs/evaluator/eval_pi.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_camera_prompt.yaml similarity index 62% rename from egomimic/hydra_configs/evaluator/eval_pi.yaml rename to egomimic/hydra_configs/evaluator/eval_cartesian_camera_prompt.yaml index ae72a3e758..5c11961549 100644 --- a/egomimic/hydra_configs/evaluator/eval_pi.yaml +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_camera_prompt.yaml @@ -1,6 +1,6 @@ -# Retained root PI recipes emit legacy normalized Cartesian actions. +# Camera-frame Cartesian metrics with sampled-prompt overlays. defaults: - - eval_hpt + - eval_cartesian_camera - _self_ rkl_samples: 8 viz_func: diff --git a/egomimic/hydra_configs/evaluator/frozen_yam_time.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_chunk_metrics.yaml similarity index 55% rename from egomimic/hydra_configs/evaluator/frozen_yam_time.yaml rename to egomimic/hydra_configs/evaluator/eval_cartesian_chunk_metrics.yaml index 9ef8d78d8c..77a4c0483b 100644 --- a/egomimic/hydra_configs/evaluator/frozen_yam_time.yaml +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_chunk_metrics.yaml @@ -1,5 +1,6 @@ +# Metrics-only Cartesian chunks; the selected recipe supplies its batch budget. _target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval action_key: actions_cartesian untokenized_action_key: actions_time -limit_val_batches: ${frozen_yam.limit_val_batches} +limit_val_batches: ${yam.limit_val_batches} viz_func: null diff --git a/egomimic/hydra_configs/evaluator/eval_hpt_wrist.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist.yaml similarity index 92% rename from egomimic/hydra_configs/evaluator/eval_hpt_wrist.yaml rename to egomimic/hydra_configs/evaluator/eval_cartesian_wrist.yaml index 74ebf1dc86..34c5179092 100644 --- a/egomimic/hydra_configs/evaluator/eval_hpt_wrist.yaml +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist.yaml @@ -1,5 +1,5 @@ defaults: - - eval_hpt + - eval_cartesian_camera - _self_ revert_transforms: eva_bimanual: diff --git a/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_gripper_prompt.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_gripper_prompt.yaml new file mode 100644 index 0000000000..10b4d6ede1 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_gripper_prompt.yaml @@ -0,0 +1,6 @@ +defaults: + - eval_cartesian_wrist_6d_prompt + - _self_ +revert_transforms: + human_bimanual: + _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_wristframe_grip_transform_list diff --git a/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_prompt.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_prompt.yaml new file mode 100644 index 0000000000..cd4d74b079 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_6d_prompt.yaml @@ -0,0 +1,9 @@ +defaults: + - /evaluator/_base/cartesian_prompt@_here_ + - _self_ +viz_max_batches: 10 +revert_transforms: + eva_bimanual: + _target_: egomimic.rldb.embodiment.eva._build_eva_cartesian_revert_6d_wristframe_transform_list + human_bimanual: + _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_wristframe_transform_list diff --git a/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_prompt.yaml b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_prompt.yaml new file mode 100644 index 0000000000..4682d4f4c5 --- /dev/null +++ b/egomimic/hydra_configs/evaluator/eval_cartesian_wrist_prompt.yaml @@ -0,0 +1,10 @@ +defaults: + - /evaluator/_base/cartesian_prompt@_here_ + - _self_ +revert_transforms: + eva_bimanual: + _target_: egomimic.rldb.embodiment.eva._build_eva_bimanual_revert_eef_frame_transform_list + is_quat: false + human_bimanual: + _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_eef_frame_transform_list + is_quat: false diff --git a/egomimic/hydra_configs/evaluator/eval_open_loop_sim.yaml b/egomimic/hydra_configs/evaluator/eval_open_loop_sim.yaml index a6bde937b3..9b29326df1 100644 --- a/egomimic/hydra_configs/evaluator/eval_open_loop_sim.yaml +++ b/egomimic/hydra_configs/evaluator/eval_open_loop_sim.yaml @@ -1,48 +1,44 @@ +# Recorded-episode execution evaluation; not closed-loop physics simulation. defaults: -- viz@viz_func: cartesian/base -- _self_ + - /evaluator/_base/yam_human_cartesian@_here_ + - _self_ _target_: egomimic.eval.open_loop_sim.OpenLoopSimEval -action_key: actions_cartesian + +# Execution: baseline control steps or a prefix of canonical ARC waypoints. +action_mode: auto ground_truth_action_key: actions_cartesian_untokenized execute_fraction: 0.3 control_horizon: 100 control_dt: 0.03333333333333333 -action_mode: auto -arc_chunking_mode: multistream -arc_execution_cap_mode: waypoints -arc_video_trajectory_cap_mode: ${oc.select:abc.arc_chunking_mode,multistream} + +# Canonical codec: independent streams, ending at the first moving stream. +# duration means absolute waypoint seconds; velocity means incoming rates. +clock_mode: ${oc.select:abc.arc_clock_mode,velocity} +streams: ${oc.select:abc.arc_streams,full_separation} min_distance_unit: ${oc.select:abc.arc_distance,0.40} rotation_distance_unit: ${oc.select:abc.arc_rotation_distance,1.7453292519943295} resampled_vector_length: ${oc.select:abc.arc_waypoints,100} -clock_mode: ${oc.select:abc.arc_clock_mode,velocity} -streams: ${oc.select:abc.arc_streams,full_separation} gripper_distance_unit: ${oc.select:abc.arc_gripper_distance,null} -log_step: null -results_path: ${paths.output_dir}/open_loop_sim.json -records_dump_path: null -trajectory_snapshot_path: null -video_only: false +arc_chunking_mode: multistream +arc_execution_cap_mode: waypoints +arc_video_trajectory_cap_mode: ${oc.select:abc.arc_chunking_mode,multistream} + +# Complete-episode scoring and bounded DTW work. +require_episode_start: true +requires_ordered_validation: true +limit_val_episodes: null distance_dtw_enabled: true dtw_max_cells: 50000000 dtw_max_prediction_steps: 30000 dtw_max_segment_steps: null -require_episode_start: true -limit_val_episodes: null -requires_ordered_validation: true -obs_pose_key: observations.state.ee_pose -image_key: observations.images.front_img_1 -revert_transforms: - yam_bimanual: - _target_: egomimic.rldb.embodiment.yam._build_yam_bimanual_revert_eef_frame_transform_list - is_quat: false - to_camera_frame: true - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_eef_frame_transform_list - is_quat: false - gripper_padded: true + +# Artifacts and overlays. Null dump paths disable optional diagnostic exports. +results_path: ${paths.output_dir}/open_loop_sim.json video_output_dir: ${paths.output_dir}/val_videos video_overlay_mode: executed_chunk -video_chunk_frames: 1000 -max_episode_frames: 6000 +video_only: false viz_every_n_epochs: 1 viz_max_batches: null +log_step: null +records_dump_path: null +trajectory_snapshot_path: null diff --git a/egomimic/hydra_configs/evaluator/pi05/eval_pi.yaml b/egomimic/hydra_configs/evaluator/pi05/eval_pi.yaml deleted file mode 100644 index 8ab24290b1..0000000000 --- a/egomimic/hydra_configs/evaluator/pi05/eval_pi.yaml +++ /dev/null @@ -1,44 +0,0 @@ -viz_func: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva.Eva.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt - fallback_intrinsics: - - - 436.26 - - 0.0 - - 310.1 - - 0.0 - - - 0.0 - - 435.13 - - 241.93 - - 0.0 - - - 0.0 - - 0.0 - - 1.0 - - 0.0 - human_bimanual: - _target_: egomimic.rldb.embodiment.human.Human.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt -_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval -viz_every_n_epochs: 100 -revert_transforms: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva._build_eva_bimanual_revert_eef_frame_transform_list - is_quat: false - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_eef_frame_transform_list - is_quat: false -image_key: observations.images.front_img_1 -pose_metrics: true -rkl_samples: ${model.pipeline.stages.0.policy.config.reverse_kl_samples} -group_options: - train_viz: - limit_batches: 50 - rkl_samples: 1 diff --git a/egomimic/hydra_configs/evaluator/pi05/eval_pi_camframe_6d.yaml b/egomimic/hydra_configs/evaluator/pi05/eval_pi_camframe_6d.yaml deleted file mode 100644 index a714a4b4ff..0000000000 --- a/egomimic/hydra_configs/evaluator/pi05/eval_pi_camframe_6d.yaml +++ /dev/null @@ -1,42 +0,0 @@ -viz_func: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva.Eva.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt - fallback_intrinsics: - - - 436.26 - - 0.0 - - 310.1 - - 0.0 - - - 0.0 - - 435.13 - - 241.93 - - 0.0 - - - 0.0 - - 0.0 - - 1.0 - - 0.0 - human_bimanual: - _target_: egomimic.rldb.embodiment.human.Human.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt -_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval -viz_every_n_epochs: 100 -revert_transforms: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva._build_eva_cartesian_revert_6d_transform_list - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_transform_list -image_key: observations.images.front_img_1 -pose_metrics: true -rkl_samples: ${model.pipeline.stages.0.policy.config.reverse_kl_samples} -group_options: - train_viz: - limit_batches: 50 - rkl_samples: 1 diff --git a/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d.yaml b/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d.yaml deleted file mode 100644 index e867f72eda..0000000000 --- a/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d.yaml +++ /dev/null @@ -1,43 +0,0 @@ -viz_func: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva.Eva.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt - fallback_intrinsics: - - - 436.26 - - 0.0 - - 310.1 - - 0.0 - - - 0.0 - - 435.13 - - 241.93 - - 0.0 - - - 0.0 - - 0.0 - - 1.0 - - 0.0 - human_bimanual: - _target_: egomimic.rldb.embodiment.human.Human.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt -_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval -viz_every_n_epochs: 100 -viz_max_batches: 10 -revert_transforms: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva._build_eva_cartesian_revert_6d_wristframe_transform_list - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_wristframe_transform_list -image_key: observations.images.front_img_1 -pose_metrics: true -rkl_samples: ${model.pipeline.stages.0.policy.config.reverse_kl_samples} -group_options: - train_viz: - limit_batches: 50 - rkl_samples: 1 diff --git a/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d_grip.yaml b/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d_grip.yaml deleted file mode 100644 index 09242c90ba..0000000000 --- a/egomimic/hydra_configs/evaluator/pi05/eval_pi_wristframe_6d_grip.yaml +++ /dev/null @@ -1,43 +0,0 @@ -viz_func: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva.Eva.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt - fallback_intrinsics: - - - 436.26 - - 0.0 - - 310.1 - - 0.0 - - - 0.0 - - 435.13 - - 241.93 - - 0.0 - - - 0.0 - - 0.0 - - 1.0 - - 0.0 - human_bimanual: - _target_: egomimic.rldb.embodiment.human.Human.viz_gt_preds - _partial_: true - image_key: observations.images.front_img_1 - action_key: actions_cartesian - mode: traj - annotation_key: sampled_prompt -_target_: egomimic.eval.bimanual_cartesian_eval.BimanualCartesianEval -viz_every_n_epochs: 100 -viz_max_batches: 10 -revert_transforms: - eva_bimanual: - _target_: egomimic.rldb.embodiment.eva._build_eva_cartesian_revert_6d_wristframe_transform_list - human_bimanual: - _target_: egomimic.rldb.embodiment.human._build_human_cartesian_revert_6d_wristframe_grip_transform_list -image_key: observations.images.front_img_1 -pose_metrics: true -rkl_samples: ${model.pipeline.stages.0.policy.config.reverse_kl_samples} -group_options: - train_viz: - limit_batches: 50 - rkl_samples: 1 diff --git a/egomimic/hydra_configs/evaluator/viz/cotrain_lang.yaml b/egomimic/hydra_configs/evaluator/viz/cartesian_annotations.yaml similarity index 71% rename from egomimic/hydra_configs/evaluator/viz/cotrain_lang.yaml rename to egomimic/hydra_configs/evaluator/viz/cartesian_annotations.yaml index 85f0a2e6cc..bfe2425de7 100644 --- a/egomimic/hydra_configs/evaluator/viz/cotrain_lang.yaml +++ b/egomimic/hydra_configs/evaluator/viz/cartesian_annotations.yaml @@ -1,3 +1,4 @@ +# Recorded annotations, rather than model-sampled prompts. defaults: - cartesian - _self_ diff --git a/egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang.yaml b/egomimic/hydra_configs/evaluator/viz/cartesian_prompt.yaml similarity index 65% rename from egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang.yaml rename to egomimic/hydra_configs/evaluator/viz/cartesian_prompt.yaml index 4f44303949..a8aba8c1bc 100644 --- a/egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang.yaml +++ b/egomimic/hydra_configs/evaluator/viz/cartesian_prompt.yaml @@ -1,3 +1,4 @@ +# Frame-independent drawing; the evaluator owns coordinate reversion. defaults: - cartesian - _self_ diff --git a/egomimic/hydra_configs/evaluator/viz/cartesian_wrist.yaml b/egomimic/hydra_configs/evaluator/viz/cartesian_wrist.yaml deleted file mode 100644 index 386b79e34a..0000000000 --- a/egomimic/hydra_configs/evaluator/viz/cartesian_wrist.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# Frame reversion belongs to evaluator=eval_hpt_wrist; renderer receives camera poses. -defaults: - - cartesian - - _self_ diff --git a/egomimic/hydra_configs/evaluator/viz/cartesian_wrist_lang.yaml b/egomimic/hydra_configs/evaluator/viz/cartesian_wrist_lang.yaml deleted file mode 100644 index f83e986a06..0000000000 --- a/egomimic/hydra_configs/evaluator/viz/cartesian_wrist_lang.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# Pair with evaluator=eval_hpt_wrist for native wrist-frame predictions. -defaults: - - cotrain_lang - - _self_ diff --git a/egomimic/hydra_configs/evaluator/viz/pi_cartesian.yaml b/egomimic/hydra_configs/evaluator/viz/pi_cartesian.yaml deleted file mode 100644 index a0a098b143..0000000000 --- a/egomimic/hydra_configs/evaluator/viz/pi_cartesian.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# Retained renderer selection; delegates to the same Cartesian renderer as HPT. -defaults: - - cartesian - - _self_ diff --git a/egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang_wrist.yaml b/egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang_wrist.yaml deleted file mode 100644 index 7bfaec4e97..0000000000 --- a/egomimic/hydra_configs/evaluator/viz/pi_cartesian_lang_wrist.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# Pair with evaluator=eval_hpt_wrist for native wrist-frame predictions. -defaults: - - pi_cartesian_lang - - _self_ diff --git a/egomimic/hydra_configs/experiment/README.md b/egomimic/hydra_configs/experiment/README.md index ee75537b6d..468d8d19a8 100644 --- a/egomimic/hydra_configs/experiment/README.md +++ b/egomimic/hydra_configs/experiment/README.md @@ -4,30 +4,29 @@ The Cartesian YAM/ABC experiment menu is: ```text experiment/ - real/ the nine rl2 YAM task leaves (Aidan's frozen pools) + arc_real/ the nine rl2 YAM task leaves (Aidan's frozen pools) stationary_{time,duration,velocity}.yaml earbuds_{time,duration,velocity}.yaml towels_{time,duration,velocity}.yaml abc_arc/ ABC-lab visual experiments robot_bc/ ABC towels HPT180, ABC multitask4 HPT300 and ABC-DiT - human_bc/ MECKA human-only folding - sim/ + arc_sim/ libero/ libero_historical/ oat/ pusht/ synthetic/ - _shared/ shared defaults, not standalone runs ``` -Select one with `+experiment=real/stationary_duration` (or `_velocity` or +Select one with `+experiment=arc_real/stationary_duration` (or `_velocity` or `_time`). Cluster resources still come from the explicit `hydra/launcher=...` selection and require a separate launch preflight. Each leaf is a **complete experiment entry point**: its defaults select the model graph, data filters/splits/transforms, evaluator, optimizer, trainer, callbacks and logging. No additional `model=` or `data=` selection is needed. -Shared parents avoid copying those definitions into every leaf. Supply +Reusable definitions live in their component groups; real leaves do not inherit +other experiments. Supply machine-specific data/weight paths, credentials, a fresh W&B identity (`logger.wandb.id` defaults to `_s`) and the reviewed cluster allocation at launch. @@ -36,7 +35,7 @@ Example (submits a run; execute only after launch preflight): ```bash python egomimic/trainHydra.py -m \ - +experiment=real/stationary_duration \ + +experiment=arc_real/stationary_duration \ hydra/launcher=submitit_pace_h100 ``` @@ -56,7 +55,8 @@ its pool. A pool YAML lists only its frozen validation hashes; the train rule is the task minus that list, and the count / SHA-256 pins guard the membership. To add data, add it to the task and update the pins. -All nine share one graph and schedule (`_shared/frozen_yam/hpt300_openloop`: +All nine share one graph and schedule (`model/hpt/yam_300m`, +`model/base/real_yam_settings`, `trainer/real_yam` and `callbacks/real_yam`: HPT-300M, embed 840, trunk 19 x 10 heads, flow 6 x 320, 240k steps, cosine with 10k warmup) and one scorer (the complete-episode open-loop evaluator on the 25 % prefix; rank on `Valid/open_loop_sim/{XYZ,YPR,Grip}_MSE`). The three @@ -65,7 +65,7 @@ leaves of a task select exactly the same episodes. - `_time`: the baseline, 100 x 14 time-indexed actions, deployed through `TimeChunkRetimer`. - `_duration`: absolute-duration single-clock ARC - (`_shared/arc_single_clock`): M 200, one clock per arm + (`model/hpt/yam_300m_arc` and `model_contract/arc_clock_settings`): M 200, one clock per arm (`abc.arc_streams=translation`, M x 16; rotation and gripper ride the translation supports), absolute waypoint seconds decoded by speed-bounded isotonic regression, 100-frame native source window, deployed 50 % waypoint @@ -91,27 +91,26 @@ No tokenizer or detokenizer algorithm is changed by this YAML reorganization. `abc_arc/robot_bc/` keeps the ABC-lab visual experiments on the `abc_visual` pool (ABC towels HPT180 baseline / hybrid, ABC four-task HPT300 `multitask/{baseline,arc_vel,arc_dur}` and the ABC-DiT `multitask/variants`). -`abc_arc/human_bc/` keeps the MECKA human-only folding experiments. They read -400 native frames with the full-separation clock contract and are not +The ABC recipes read 400 native frames with the full-separation clock contract and are not interchangeable with the real task leaves. ## Retired on 2026-10-22 -The per-task folders (`real/{organize_stationary,wireless_earbuds, +The per-task folders (`arc_real/{organize_stationary,wireless_earbuds, fold_and_stack_towels}/{baseline,arc_vel,arc_dur}` with their `variants/`), -`real/arcdur_abstime/`, `real/multitask/` (moved to `abc_arc/robot_bc/`), the -`_shared/frozen_yam/{towels_hpt300,earbuds_updated_hpt300}` recipes and the +`arc_real/arcdur_abstime/`, `arc_real/multitask/` (moved to `abc_arc/robot_bc/`), the +`_shared/yam/{towels_hpt300,earbuds_updated_hpt300}` recipes and the dated `data/abc_arc` pools they selected are removed in favour of the nine leaves above. `docs/experiment_config_migration.json` maps every old path to its replacement, or to `null` when it was retired (Elmo's organize robot + human cotrain leaves, the HPT180 / ABC-DiT rl2 towel variants and the `new290plus74` pool). All of it is recoverable from Git history (`8f5eeb7a`). -PushT, LIBERO, OAT/LIBERO and synthetic experiments live under `sim/`. +PushT, LIBERO, OAT/LIBERO and synthetic experiments live under `arc_sim/`. The `e1`, `ravens` and `stationary_recipe` configuration menus are removed. E1 scripts and runtime compatibility entry points are also removed. Shared components required by current real recipes have neutral names: -`model/abc_arc/frozen_yam_*`, `model_contract/frozen_yam_wrists`, +`model/hpt/yam_*`, `model_contract/yam_wrists`, `embodiment_resolvers`, and `progress_sampler`. Their behavior is unchanged. Old CLI paths under `abc_arc/{robot_bc,cotrain}` and `yam_arc_grid` are removed, diff --git a/egomimic/hydra_configs/experiment/_shared/arc_single_clock.yaml b/egomimic/hydra_configs/experiment/_shared/arc_single_clock.yaml deleted file mode 100644 index 7cfeaf2ff1..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/arc_single_clock.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# @package _global_ -# Aidan's single-clock ARC protocol (vault note "ARC Abstime And Interval Arcvel Run Protocol 10-20") on the -# shared frozen-YAM ARC plumbing: one clock per arm (rotation and gripper ride the translation supports), -# absolute waypoint seconds decoded by speed-bounded isotonic regression, M 200 waypoints, a 100-frame native -# source window (not the 400-frame default), 25 % open-loop scoring prefix and a 50 % deployed waypoint prefix. -# The *_duration leaves use this as is; the *_velocity leaves switch abc.arc_clock_mode to velocity (interval -# arcvel). Task leaves set D (abc.arc_distance). -defaults: - - /experiment/_shared/frozen_yam_arc - - _self_ -abc: - arc_clock_mode: duration - arc_streams: translation - arc_waypoints: 200 - arc_source_buffer_frames: 100 -evaluator: - execute_fraction: 0.25 -model: - inference_contracts: - hybrid_arc_tokenizer_cartesian: - profiles: - default: - adapter: - decoder: - execute_percent: 50 diff --git a/egomimic/hydra_configs/experiment/_shared/frozen_yam/base.yaml b/egomimic/hydra_configs/experiment/_shared/frozen_yam/base.yaml deleted file mode 100644 index ee800ddd09..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/frozen_yam/base.yaml +++ /dev/null @@ -1,56 +0,0 @@ -# @package _global_ -defaults: -- override /evaluator: frozen_yam_time -- _self_ -frozen_yam: - policy_domain: human - variant: ??? - spread: ??? - train_root: ??? - valid_root: ??? - train_mode: total - valid_mode: total - D: 0.4 - M: 100 - time_rows: 100 - chunk_length: 100 - h_match_frames: 40 - limit_val_batches: 40 - prog_horizon_m: 0.19 - anchor_sampler: null - arcmatch_gt_span_m: null - action_dim: 14 - action_horizon: 100 - image_weights: DEFAULT -name: frozen_yam_${frozen_yam.variant} -description: ${frozen_yam.spread}_s${seed} -arc_tokenizer: - min_distance_unit: ${frozen_yam.D} - resampled_vector_length: ${frozen_yam.M} -trainer: - strategy: auto - sync_batchnorm: false - check_val_every_n_epoch: 10 - num_sanity_val_steps: 0 - limit_val_batches: ${frozen_yam.limit_val_batches} -callbacks: - model_checkpoint: - every_n_epochs: 50 - save_top_k: -1 -norm_stats: - sample_frac: 0.25 - num_workers: 8 -hpt: - embed_dim: 256 - num_blocks: 16 - num_heads: 8 - stem_specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: ${hpt.embed_dim} - action_dim: ${frozen_yam.action_dim} - action_horizon: ${frozen_yam.action_horizon} diff --git a/egomimic/hydra_configs/experiment/_shared/frozen_yam/hpt300_openloop.yaml b/egomimic/hydra_configs/experiment/_shared/frozen_yam/hpt300_openloop.yaml deleted file mode 100644 index fb54ea446b..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/frozen_yam/hpt300_openloop.yaml +++ /dev/null @@ -1,28 +0,0 @@ -# @package _global_ -# HPT-300M frozen-YAM recipe scored by the complete-episode open-loop evaluator (baseline mode, 25 % prefix): -# the training schedule of _shared/frozen_yam/stattempo, the open-loop validation of stattempo_openloop, and the -# 2026-10-21 campaign widths (embed 840, trunk 19 blocks x 10 heads x 84, flow CrossTransformer 6 x 320 from -# model abc_arc/frozen_yam_hpt300_time). Shared by the nine real task leaves; a leaf picks its frozen pool -# (override /data), its spread and its name. The ARC leaves add _shared/arc_single_clock on top. -defaults: - - /experiment/_shared/frozen_yam/stattempo_openloop - - override /model: abc_arc/frozen_yam_hpt300_time - - _self_ -description: h840t19_d320x6_cos10k_240k_${name} -frozen_yam: - policy_domain: yam - variant: time - action_dim: 14 -hpt: - action_dim: ${frozen_yam.action_dim} - embed_dim: 840 - num_blocks: 19 - num_heads: 10 - stem_specs: - cross_attn: - crossattn_heads: 10 - crossattn_dim_head: 84 - modality_embed_dim: 840 -logger: - wandb: - id: ${name}_s${seed} diff --git a/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo.yaml b/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo.yaml deleted file mode 100644 index bb4a325716..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo.yaml +++ /dev/null @@ -1,68 +0,0 @@ -# @package _global_ -defaults: -- /experiment/_shared/frozen_yam/base -- override /model: abc_arc/frozen_yam_hpt_flow -- override /data: abc_arc/stationary -- override /trainer: ddp -- override /logger: wandb -- override /callbacks: checkpoints -- _self_ -name: bc_rl2_stattempo_slowpace366_${frozen_yam.variant} -description: h640t8_d384x10_cos10k_240k_s42_${name} -ft: - max_steps: 240000 - warmup_steps: 10000 -init_weights_from: null -paths: - dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets - work_dir: null -trainer: - default_root_dir: null - max_epochs: -1 - min_epochs: 0 - devices: 1 - limit_val_batches: 80 - check_val_every_n_epoch: 200 - num_nodes: 1 - num_sanity_val_steps: 2 - max_steps: ${ft.max_steps} -callbacks: - model_checkpoint: - filename: epoch-{epoch:04d}-step-{step:08d} - every_n_epochs: null - every_n_train_steps: 30000 - auto_insert_metric_name: false -logger: - wandb: - save_dir: null - id: ${name}_s42 - project: arc - tags: - - rl2 - - stationery - - stattempo - - slowpace366 - - bc - - ${frozen_yam.variant} - resume: never -norm_stats: - sample_frac: 0.2 - save_cache_dir: null -frozen_yam: - policy_domain: yam - variant: time - spread: stationery - train_root: unused-s3-resolver - valid_root: unused-s3-resolver -arc_tokenizer: - min_distance_unit: 0.4 - resampled_vector_length: 100 -hpt: - embed_dim: 640 - num_blocks: 8 - stem_specs: - cross_attn: - crossattn_dim_head: 80 - modality_embed_dim: 640 - action_dim: 14 - action_horizon: 100 diff --git a/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo_openloop.yaml b/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo_openloop.yaml deleted file mode 100644 index 897dac1669..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/frozen_yam/stattempo_openloop.yaml +++ /dev/null @@ -1,22 +0,0 @@ -# @package _global_ -# Shared frozen YAM stationery recipe with robot-only complete-episode open-loop -# validation, for robot-only BC and (via _base/stattempo_aria) Aria cotrain. -# Keep evaluator selection in defaults so teacher-forced kwargs cannot leak in. -defaults: -- /experiment/_shared/frozen_yam/stattempo -- override /evaluator: eval_open_loop_sim -- _self_ -trainer: - limit_val_batches: 1.0 -evaluator: - # Recorded runs capped overlay trajectories by distance; the shared evaluator - # now defaults to the chunking mode, which these recipes do not set. - arc_video_trajectory_cap_mode: distance - limit_val_episodes: null - execute_fraction: 0.25 - distance_dtw_enabled: false - viz_every_n_epochs: 0 - ground_truth_action_key: actions_time - action_mode: baseline - min_distance_unit: 0.4 - resampled_vector_length: 100 diff --git a/egomimic/hydra_configs/experiment/_shared/frozen_yam_arc.yaml b/egomimic/hydra_configs/experiment/_shared/frozen_yam_arc.yaml deleted file mode 100644 index 0a82b16137..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/frozen_yam_arc.yaml +++ /dev/null @@ -1,93 +0,0 @@ -# @package _global_ -# Canonical ARC over the retained frozen YAM baseline graph and episode lists. -defaults: - - /experiment/_shared/clock_settings - - /model_contract/abc_visual@model - - override /evaluator: eval_open_loop_sim - - _self_ -abc: - action_mode: hybrid_arc_tokenizer_cartesian - arc_chunking_mode: multistream - arc_distance: 0.33 - arc_rotation_distance: 1.7453292519943295 - arc_waypoints: 200 - arc_source_buffer_frames: 400 - arc_gripper_stream_mode: events - arc_gripper_distance: null -frozen_yam: - variant: arc - yam_source_frames: ${abc.arc_source_buffer_frames} - M: ${abc.arc_waypoints} - D: ${abc.arc_distance} -arc_tokenizer: - min_distance_unit: ${abc.arc_distance} - resampled_vector_length: ${abc.arc_waypoints} -hpt: - action_horizon: ${abc.arc_token_rows} - token_dim: ${abc.arc_token_dim} -model: - action_token_dim: ${abc.arc_token_dim} - action_horizon: ${abc.arc_token_rows} - native_action_horizon: ${abc.arc_token_rows} - native_action_dim: ${abc.arc_token_dim} - inference: ${model.inference_contracts.hybrid_arc_tokenizer_cartesian} - data_requirements: - preprocessing: - '7': - key_map: - _target_: egomimic.rldb.embodiment.bimanual_arc.get_keymap - horizon: 200 - embodiment: yam - drop_wrist_images: false - yam_source_frames: ${abc.arc_source_buffer_frames} - transforms: - _target_: egomimic.rldb.embodiment.bimanual_arc.get_transform_list - variant: arc - chunk_length: 200 - time_rows: ${abc.arc_source_buffer_frames} - min_distance_unit: ${abc.arc_distance} - rotation_distance_unit: ${abc.arc_rotation_distance} - resampled_vector_length: ${abc.arc_waypoints} - stride: 1 - rotation_mode: euler - embodiment: yam - yam_source_frames: ${abc.arc_source_buffer_frames} - clock_mode: ${abc.arc_clock_mode} - streams: ${abc.arc_streams} - gripper_stream_mode: ${abc.arc_gripper_stream_mode} - source_fps: 30.0 - graph: - noising: - action_horizon: ${abc.arc_token_rows} - sampler: - action_horizon: ${abc.arc_token_rows} - model: - act_seq: ${abc.arc_token_rows} -data: - train_datasets: - yam_bimanual: - resolver: - key_map: ${model.data_requirements.preprocessing.7.key_map} - transform_list: ${model.data_requirements.preprocessing.7.transforms} - valid_datasets: - yam_bimanual: - resolver: - key_map: ${model.data_requirements.preprocessing.7.key_map} - transform_list: ${model.data_requirements.preprocessing.7.transforms} -trainer: - limit_val_batches: 1.0 -evaluator: - action_mode: arc - ground_truth_action_key: actions_time - execute_fraction: 0.3 - arc_execution_cap_mode: waypoints - arc_chunking_mode: ${abc.arc_chunking_mode} - min_distance_unit: ${abc.arc_distance} - rotation_distance_unit: ${abc.arc_rotation_distance} - resampled_vector_length: ${abc.arc_waypoints} - clock_mode: ${abc.arc_clock_mode} - streams: ${abc.arc_streams} -logger: - wandb: - id: ${name}_s${seed} - tags: [rl2, hpt300, "${abc.arc_clock_mode}", "${abc.arc_streams}", open_loop, video] diff --git a/egomimic/hydra_configs/experiment/_shared/visual_hpt300.yaml b/egomimic/hydra_configs/experiment/_shared/visual_hpt300.yaml deleted file mode 100644 index 126054dfbb..0000000000 --- a/egomimic/hydra_configs/experiment/_shared/visual_hpt300.yaml +++ /dev/null @@ -1,72 +0,0 @@ -# @package _global_ -defaults: -- /experiment/_shared/clock_settings -- override /model: abc_arc/hpt_yam_visual -- override /data: abc_arc/abc_visual -- override /evaluator: eval_open_loop_sim -- override /logger: wandb -- _self_ -launch_params: - gpus_per_node: 2 - nodes: 1 -hpt: - embed_dim: 840 - num_blocks: 19 - num_heads: 10 - action_dim: 14 - action_horizon: 100 - flow_blocks: 6 - flow_hidden_dim: 320 - stem_specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: ${hpt.num_heads} - crossattn_dim_head: 84 - crossattn_modality_dropout: 0.1 - modality_embed_dim: ${hpt.embed_dim} -abc: - action_mode: cartesian - action_horizon: 100 - arc_distance: 0.4 - arc_rotation_distance: null - arc_waypoints: 100 - arc_chunking_mode: multistream - task_predicate: ??? -trainer: - max_steps: 240000 - max_epochs: -1 - min_epochs: null - val_check_interval: 10000 - check_val_every_n_epoch: null - strategy: ddp_find_unused_parameters_true - precision: 32-true - num_sanity_val_steps: 0 - accumulate_grad_batches: 1 - log_every_n_steps: 10 -norm_stats: - sample_frac: 0.1 -evaluator: - action_mode: baseline - ground_truth_action_key: actions_cartesian - control_horizon: 100 - control_dt: 0.03333333333333333 - execute_fraction: 0.3 - arc_execution_cap_mode: waypoints - arc_chunking_mode: ${abc.arc_chunking_mode} - min_distance_unit: ${abc.arc_distance} - rotation_distance_unit: ${abc.arc_rotation_distance} - resampled_vector_length: ${abc.arc_waypoints} - clock_mode: ${abc.arc_clock_mode} - limit_val_episodes: 16 - requires_ordered_validation: true -logger: - wandb: - project: abc-yam - tags: - - hpt300m - - abc - - visual_only - - no_language - - open_loop - - video diff --git a/egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml b/egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml deleted file mode 100644 index 1fc547b5ae..0000000000 --- a/egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml +++ /dev/null @@ -1,70 +0,0 @@ -# @package _global_ -# ABC-130k single-domain BC: fold and stack the t-shirts, yam_bimanual only. -# -# The ABC arm of the cotrain ablation. Robot data volume is pinned at the same -# 20 episodes the cotrain config uses, so a delta against that run is -# attributable to the human data rather than to seeing more robot data. -defaults: - - override /model: abc/yam_bimanual_dp - - override /data: abc_arc/abc_fstshirt_bc - - override /evaluator: null - - override /hydra/launcher: submitit_pace - - _self_ - -name: abc_yam_fstshirt_dp -description: yam_bimanual_eef_euler_DP_DDIM100_obs1 - -launch_params: {gpus_per_node: 2, nodes: 1} -trainer: - max_steps: 240000 - max_epochs: -1 - min_epochs: null - val_check_interval: 10000 - check_val_every_n_epoch: null - num_sanity_val_steps: 0 - accumulate_grad_batches: 1 - log_every_n_steps: 10 - -norm_stats: - norm_mode: quantile - sample_frac: 0.05 - -logger: - wandb: - project: abc-yam - -abc: - # eef_frame + euler => [L xyz ypr g, R xyz ypr g]. - action_dim: 14 - # Yam's cartesian transform list interpolates every action chunk to - # `chunk_length` (default 100), and the data config does not override it, so - # the loader hands the graph a (100, 14) target. The diffusion stages - # validate the target's width, so this MUST match -- 16 aborts on the first - # batch. Matches the sibling cotrain baseline model, which hardcodes 100. - action_horizon: 100 - # FusedObsEncoder output: 64 image features + the 14D proprio pose. - condition_dim: 78 - encoder: - _target_: egomimic.pipeline.stages_sampler.DPStyleObsEncoder - obs_specs: - ee_pose: {input_dim: 14, input_slice: [0, 14]} - img_encoders: - front_img_1: - _target_: egomimic.models.stems.visual_core.VisualCore - in_channels: 3 - # ABC frames are 480x640 (the data config pins image_hw to [480, 640]), - # so height and width are given separately -- a square image_size sizes - # the SpatialSoftmax grid wrong for a non-square frame. - image_size: 480 - image_width: 640 - num_kp: 32 - feature_dimension: 64 - pretrained: false - crop_aug: true - crop_height: 432 - crop_width: 576 - crop_eval_mode: center - crop_sample_mode: v02 - crop_scope: frame - norm_layer: group - pool_type: spatial_softmax diff --git a/egomimic/hydra_configs/experiment/abc/yam_fstshirt_hpt.yaml b/egomimic/hydra_configs/experiment/abc/yam_fstshirt_hpt.yaml deleted file mode 100644 index 7c55817dea..0000000000 --- a/egomimic/hydra_configs/experiment/abc/yam_fstshirt_hpt.yaml +++ /dev/null @@ -1,69 +0,0 @@ -# @package _global_ -# ABC-130k single-domain HPT: fold and stack the t-shirts, yam_bimanual. -# -# The HPT arm of the architecture comparison. Same 20 episodes, same action -# space and same dataloader params as experiment/abc/yam_fstshirt_dp -- the -# only difference is the model, so a delta between the two is attributable to -# the architecture rather than the data. -# -# Launch: -# python egomimic/trainHydra.py -m \ -# +experiment=abc/yam_fstshirt_hpt -defaults: - - override /model: abc/yam_bimanual_hpt - - override /data: abc_arc/abc_fstshirt_bc - - override /evaluator: null - - override /hydra/launcher: submitit_pace - - _self_ - -name: abc_yam_fstshirt_hpt -description: hpt_flow_trunk16_embed256 - -paths: - dataset_dir: /storage/project/r-dxu345-0/shared/arc_abc - -launch_params: - gpus_per_node: 1 - nodes: 1 - -trainer: - max_steps: 240000 - max_epochs: -1 - min_epochs: null - val_check_interval: 10000 - check_val_every_n_epoch: null - num_sanity_val_steps: 0 - accumulate_grad_batches: 1 - log_every_n_steps: 10 - -norm_stats: - norm_mode: quantile - sample_frac: 0.05 - -logger: - wandb: - project: abc-yam - tags: - - abc - - hpt - - flow - -hpt: - # Shared bus width. Every stem output, cross_attn modality_embed_dim, the - # trunk and the head's cond_dim read this one value. - embed_dim: 256 - num_blocks: 16 - num_heads: 8 - # Yam's cartesian transform list interpolates to chunk_length (default 100), - # so the loader hands over 100-row chunks; the flow stages validate that. - action_horizon: 100 - action_dim: 14 - # Cross-attention latent pooling, applied per modality by each stem. - stem_specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: ${hpt.embed_dim} diff --git a/egomimic/hydra_configs/experiment/abc_arc/README.md b/egomimic/hydra_configs/experiment/abc_arc/README.md index 415f46af3b..26a7c5c5cc 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/README.md +++ b/egomimic/hydra_configs/experiment/abc_arc/README.md @@ -1,6 +1,6 @@ # ABC-lab visual experiments -The rl2 YAM task leaves live in `real/` (see [the experiment menu](../README.md)). +The rl2 YAM task leaves live in `arc_real/` (see [the experiment menu](../README.md)). This folder keeps the ABC-lab visual experiments on the `abc_visual` pool: `robot_bc/abc_towels_hpt180_{baseline,hybrid}_visual_openloop`, the ABC four-task `robot_bc/multitask/{baseline,arc_vel,arc_dur}` and the ABC-DiT diff --git a/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop.yaml deleted file mode 100644 index ebc0c7aaba..0000000000 --- a/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop.yaml +++ /dev/null @@ -1,17 +0,0 @@ -# @package _global_ -defaults: - - /experiment/_shared/visual_hpt300 - - override /model: abc_arc/hpt_bimanual_visual - - override /data: abc_arc/mecka_fold_clothes_40h_human_baseline - - _self_ -name: mecka_fold_clothes_40h_human_visual_baseline_openloop -description: HPT300M human-bimanual MECKA fold_clothes visual-only baseline -abc: - task_predicate: "row['task'] == 'fold_clothes'" - # Human-only pool: selects the model's data requirements. - population: human -evaluator: - limit_val_episodes: 5 -logger: - wandb: - tags: [hpt300m, mecka, human_bimanual, fold_clothes, visual_only, no_language, baseline, open_loop, video] diff --git a/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_hybrid_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_hybrid_openloop.yaml deleted file mode 100644 index 21ca0d4026..0000000000 --- a/egomimic/hydra_configs/experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_hybrid_openloop.yaml +++ /dev/null @@ -1,22 +0,0 @@ -# @package _global_ -defaults: - - /experiment/abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop - - override /data: abc_arc/mecka_fold_clothes_40h_human_hybrid_D40_M100_R24deg - - _self_ -name: mecka_fold_clothes_40h_human_visual_hybrid_openloop -description: HPT300M human-bimanual MECKA fold_clothes visual-only hybrid D40 M100 R24deg -abc: - action_mode: hybrid_arc_tokenizer_cartesian - arc_rotation_distance: 0.4188790204786391 -run_provenance: - action_contract: - source_sampling: native_30hz_v1 -hpt: - action_horizon: ${abc.arc_token_rows} - token_dim: ${abc.arc_token_dim} -evaluator: - action_mode: arc - ground_truth_action_key: actions_cartesian_untokenized -logger: - wandb: - tags: [hpt300m, mecka, human_bimanual, fold_clothes, visual_only, no_language, hybrid, D40, M100, R24deg, per_waypoint, open_loop, video] diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop.yaml index a6f1eb4b69..c744e60145 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop.yaml @@ -1,32 +1,60 @@ # @package _global_ defaults: - - /experiment/_shared/visual_hpt300 - - _self_ - -name: abc_towels_hpt180_baseline_visual_openloop -description: HPT180 ABC towels visual-only baseline 100-frame chunks - +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: - task_predicate: "row['task'] == 'fold and stack the towels'" action_mode: cartesian - arc_distance: 0.40 + action_horizon: 100 + arc_distance: 0.4 arc_rotation_distance: 0.4188790204786391 - + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] == 'fold and stack the towels' +launch_params: + gpus_per_node: 2 + nodes: 1 hpt: embed_dim: 640 num_heads: 8 - action_horizon: 100 flow_blocks: 10 flow_hidden_dim: 384 stem_specs: cross_attn: crossattn_dim_head: 80 - +norm_stats: + sample_frac: 0.1 evaluator: action_mode: baseline ground_truth_action_key: actions_cartesian + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} limit_val_episodes: 4 - + requires_ordered_validation: true logger: wandb: - tags: [hpt180, abc, towels, visual_only, no_language, baseline, open_loop, video] + project: abc-yam + tags: + - hpt180 + - abc + - towels + - visual_only + - no_language + - baseline + - open_loop + - video +name: abc_towels_hpt180_baseline_visual_openloop +description: HPT180 ABC towels visual-only baseline 100-frame chunks diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop.yaml index 176ba2e78d..fae7ec8aae 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop.yaml @@ -1,28 +1,66 @@ # @package _global_ defaults: - - /experiment/abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop - - _self_ -name: abc_towels_hpt180_hybrid_visual_openloop -description: HPT180 ABC towels visual-only hybrid D40 M100 R24deg +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: - task_predicate: "row['task'] == 'fold and stack the towels'" action_mode: hybrid_arc_tokenizer_cartesian - arc_distance: 0.40 + action_horizon: 100 + arc_distance: 0.4 arc_rotation_distance: 0.4188790204786391 + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] == 'fold and stack the towels' +launch_params: + gpus_per_node: 2 + nodes: 1 hpt: embed_dim: 640 num_heads: 8 action_horizon: ${abc.arc_token_rows} - token_dim: ${abc.arc_token_dim} flow_blocks: 10 flow_hidden_dim: 384 stem_specs: cross_attn: crossattn_dim_head: 80 + token_dim: ${abc.arc_token_dim} +norm_stats: + sample_frac: 0.1 evaluator: action_mode: arc ground_truth_action_key: actions_cartesian_untokenized + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} limit_val_episodes: 4 + requires_ordered_validation: true logger: wandb: - tags: [hpt180, abc, towels, visual_only, no_language, hybrid, D40, M100, R24deg, per_waypoint, open_loop, video] + project: abc-yam + tags: + - hpt180 + - abc + - towels + - visual_only + - no_language + - hybrid + - D40 + - M100 + - R24deg + - per_waypoint + - open_loop + - video +name: abc_towels_hpt180_hybrid_visual_openloop +description: HPT180 ABC towels visual-only hybrid D40 M100 R24deg diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_dur.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_dur.yaml index 3f67b9d2fc..fa7500f012 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_dur.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_dur.yaml @@ -1,11 +1,57 @@ # @package _global_ defaults: - - /experiment/abc_arc/robot_bc/multitask/arc_vel - - _self_ -name: multitask_arc_dur -description: HPT300 ABC four-task first-delay absolute-duration ARC +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: arc_clock_mode: duration + action_mode: hybrid_arc_tokenizer_cartesian + action_horizon: 100 + arc_distance: 0.81 + arc_rotation_distance: 0.4188790204786391 + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic + bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging + case'] +launch_params: + gpus_per_node: 2 + nodes: 1 +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +norm_stats: + sample_frac: 0.1 +evaluator: + action_mode: arc + ground_truth_action_key: actions_cartesian_untokenized + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} + limit_val_episodes: 16 + requires_ordered_validation: true logger: wandb: - tags: [hpt300m, abc, multitask4, visual_only, duration, open_loop, video] + project: abc-yam + tags: + - hpt300m + - abc + - multitask4 + - visual_only + - duration + - open_loop + - video +name: multitask_arc_dur +description: HPT300 ABC four-task first-delay absolute-duration ARC diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_vel.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_vel.yaml index 85c3ddb233..7c2e679f02 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_vel.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/arc_vel.yaml @@ -1,21 +1,60 @@ # @package _global_ defaults: - - /experiment/abc_arc/robot_bc/multitask/baseline - - _self_ -name: abc_multitask4_hpt300_hybrid_visual_openloop -description: HPT300M ABC four-task visual-only hybrid ARC D81 M100 R24deg +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: action_mode: hybrid_arc_tokenizer_cartesian - # 873 training episodes, 2,574,495 complete 100-frame windows at 30Hz. - # Mean combined left+right EEF travel = 0.8102208553 m (no validation leakage). + action_horizon: 100 arc_distance: 0.81 arc_rotation_distance: 0.4188790204786391 + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic + bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging + case'] +launch_params: + gpus_per_node: 2 + nodes: 1 hpt: action_horizon: ${abc.arc_token_rows} token_dim: ${abc.arc_token_dim} +norm_stats: + sample_frac: 0.1 evaluator: action_mode: arc ground_truth_action_key: actions_cartesian_untokenized + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} + limit_val_episodes: 16 + requires_ordered_validation: true logger: wandb: - tags: [hpt300m, abc, multitask4, visual_only, no_language, hybrid, M100, R24deg, per_waypoint, open_loop, video] + project: abc-yam + tags: + - hpt300m + - abc + - multitask4 + - visual_only + - no_language + - hybrid + - M100 + - R24deg + - per_waypoint + - open_loop + - video +name: abc_multitask4_hpt300_hybrid_visual_openloop +description: HPT300M ABC four-task visual-only hybrid ARC D81 M100 R24deg diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/baseline.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/baseline.yaml index a393e45738..8625bdf46d 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/baseline.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/baseline.yaml @@ -1,12 +1,52 @@ # @package _global_ defaults: - - /experiment/_shared/visual_hpt300 - - _self_ -name: abc_multitask4_hpt300_baseline_visual_openloop -description: HPT300M ABC four-task visual-only baseline 100-frame chunks +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: - task_predicate: "row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging case']" - # Shared with the paired ARC recipe so evaluator distance settings remain - # directly comparable even though this model predicts time-indexed actions. + action_mode: cartesian + action_horizon: 100 arc_distance: 0.81 arc_rotation_distance: 0.4188790204786391 + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic + bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging + case'] +launch_params: + gpus_per_node: 2 + nodes: 1 +norm_stats: + sample_frac: 0.1 +evaluator: + action_mode: baseline + ground_truth_action_key: actions_cartesian + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} + limit_val_episodes: 16 + requires_ordered_validation: true +logger: + wandb: + project: abc-yam + tags: + - hpt300m + - abc + - visual_only + - no_language + - open_loop + - video +name: abc_multitask4_hpt300_baseline_visual_openloop +description: HPT300M ABC four-task visual-only baseline 100-frame chunks diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop.yaml index 9d72873629..5a19010017 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop.yaml @@ -1,20 +1,63 @@ # @package _global_ defaults: - - /experiment/abc_arc/robot_bc/multitask/baseline - - override /model: abc_arc/abc_dit_yam_visual - - _self_ +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: abc_arc/abc_dit_yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ +abc: + action_mode: cartesian + action_horizon: 100 + arc_distance: 0.81 + arc_rotation_distance: 0.4188790204786391 + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic + bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging + case'] +launch_params: + gpus_per_node: 2 + nodes: 1 +trainer: + gradient_clip_val: 10.0 +norm_stats: + sample_frac: 0.1 +evaluator: + action_mode: baseline + ground_truth_action_key: actions_cartesian + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} + limit_val_episodes: 16 + requires_ordered_validation: true +logger: + wandb: + project: abc-yam + tags: + - abc_dit + - abc + - multitask4 + - clip_task + - open_loop + - video name: abc_multitask4_abcdit_baseline_visual_openloop description: ABC-DiT XL four-task baseline 100-frame chunks -# Released ABC-DiT XL shape, trained from scratch (no ABC checkpoint) on -# pretrained DINOv3 ViT-B/16 vision and frozen CLIP ViT-B/32 task text. abc_dit: hidden_size: 1536 depth: 32 num_heads: 24 action_dim: 14 action_horizon: ${abc.action_horizon} - # Local weight files; set per cluster. No default: a missing path must fail, - # never silently train from random weights. backbone_checkpoint: ${oc.env:EGOVERSE_DINOV3_CHECKPOINT} text_checkpoint: ${oc.env:EGOVERSE_CLIP_CHECKPOINT} data: @@ -23,8 +66,3 @@ data: resolver: key_map: task_key: task -trainer: - gradient_clip_val: 10.0 -logger: - wandb: - tags: [abc_dit, abc, multitask4, clip_task, open_loop, video] diff --git a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop.yaml b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop.yaml index dde079c0b2..5fe217a7ba 100644 --- a/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop.yaml +++ b/egomimic/hydra_configs/experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_hybrid_visual_openloop.yaml @@ -1,20 +1,73 @@ # @package _global_ defaults: - - /experiment/abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop - - _self_ -name: abc_multitask4_abcdit_hybrid_visual_openloop -description: ABC-DiT XL four-task hybrid ARC D81 M100 R24deg -# Same ARC delta as abc_multitask4_hpt300_hybrid_visual_openloop. +- /model/base/real_visual_settings +- /model_contract/arc_clock_settings +- override /model: abc_arc/abc_dit_yam_visual +- override /data: abc_arc/abc_visual +- override /evaluator: eval_open_loop_sim +- override /trainer: real_visual +- override /callbacks: checkpoints +- override /logger: wandb +- _self_ abc: action_mode: hybrid_arc_tokenizer_cartesian + action_horizon: 100 arc_distance: 0.81 arc_rotation_distance: 0.4188790204786391 -abc_dit: - action_horizon: ${abc.arc_token_rows} - token_dim: ${abc.arc_token_dim} + arc_waypoints: 100 + arc_chunking_mode: multistream + task_predicate: row['task'] in ['fold and stack the towels', 'take the fake fruits out of the plastic + bag', 'sort the stationery into containers', 'insert the wireless bluetooth earbuds into the charging + case'] +launch_params: + gpus_per_node: 2 + nodes: 1 +trainer: + gradient_clip_val: 10.0 +norm_stats: + sample_frac: 0.1 evaluator: action_mode: arc ground_truth_action_key: actions_cartesian_untokenized + control_horizon: 100 + control_dt: 0.03333333333333333 + execute_fraction: 0.3 + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + clock_mode: ${abc.arc_clock_mode} + limit_val_episodes: 16 + requires_ordered_validation: true logger: wandb: - tags: [abc_dit, abc, multitask4, clip_task, hybrid, M100, R24deg, per_waypoint, open_loop, video] + project: abc-yam + tags: + - abc_dit + - abc + - multitask4 + - clip_task + - hybrid + - M100 + - R24deg + - per_waypoint + - open_loop + - video +name: abc_multitask4_abcdit_hybrid_visual_openloop +description: ABC-DiT XL four-task hybrid ARC D81 M100 R24deg +abc_dit: + hidden_size: 1536 + depth: 32 + num_heads: 24 + action_dim: 14 + action_horizon: ${abc.arc_token_rows} + backbone_checkpoint: ${oc.env:EGOVERSE_DINOV3_CHECKPOINT} + text_checkpoint: ${oc.env:EGOVERSE_CLIP_CHECKPOINT} + token_dim: ${abc.arc_token_dim} +data: + train_datasets: + yam_bimanual: + resolver: + key_map: + task_key: task diff --git a/egomimic/hydra_configs/experiment/arc_real/earbuds_duration.yaml b/egomimic/hydra_configs/experiment/arc_real/earbuds_duration.yaml new file mode 100644 index 0000000000..66a17dcc42 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/earbuds_duration.yaml @@ -0,0 +1,94 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/earbuds_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: earbuds + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: earbuds_duration +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - earbuds + - plus69 + - hpt300 + - arcdur + - absolute_waypoint_time + - isotonic + - translation + - m200_d022 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_clock_mode: duration + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.22 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/arc_real/earbuds_time.yaml b/egomimic/hydra_configs/experiment/arc_real/earbuds_time.yaml new file mode 100644 index 0000000000..c91b8ae113 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/earbuds_time.yaml @@ -0,0 +1,69 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- override /model: hpt/yam_300m +- override /data: abc_arc/earbuds +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: time + spread: earbuds + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: 0.4 + M: 100 + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT +name: earbuds_time +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: 0.4 + resampled_vector_length: 100 +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - earbuds + - plus69 + - hpt300 + - time + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: baseline + min_distance_unit: 0.4 + resampled_vector_length: 100 diff --git a/egomimic/hydra_configs/experiment/arc_real/earbuds_velocity.yaml b/egomimic/hydra_configs/experiment/arc_real/earbuds_velocity.yaml new file mode 100644 index 0000000000..5ff98cf02d --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/earbuds_velocity.yaml @@ -0,0 +1,92 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/earbuds_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: earbuds + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: earbuds_velocity +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - earbuds + - plus69 + - hpt300 + - arcvel + - interval_speed + - translation + - m200_d022 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.22 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/arc_real/stationary_duration.yaml b/egomimic/hydra_configs/experiment/arc_real/stationary_duration.yaml new file mode 100644 index 0000000000..e4d2596f46 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/stationary_duration.yaml @@ -0,0 +1,94 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/stationary_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: stationery + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: stationary_duration +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - stationery + - sp366e51 + - hpt300 + - arcdur + - absolute_waypoint_time + - isotonic + - translation + - m200_d022 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_clock_mode: duration + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.22 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/arc_real/stationary_time.yaml b/egomimic/hydra_configs/experiment/arc_real/stationary_time.yaml new file mode 100644 index 0000000000..ee721f6112 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/stationary_time.yaml @@ -0,0 +1,69 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- override /model: hpt/yam_300m +- override /data: abc_arc/stationary +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: time + spread: stationery + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: 0.4 + M: 100 + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT +name: stationary_time +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: 0.4 + resampled_vector_length: 100 +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - stationery + - sp366e51 + - hpt300 + - time + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: baseline + min_distance_unit: 0.4 + resampled_vector_length: 100 diff --git a/egomimic/hydra_configs/experiment/arc_real/stationary_velocity.yaml b/egomimic/hydra_configs/experiment/arc_real/stationary_velocity.yaml new file mode 100644 index 0000000000..aa851013ba --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/stationary_velocity.yaml @@ -0,0 +1,92 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/stationary_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: stationery + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: stationary_velocity +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - stationery + - sp366e51 + - hpt300 + - arcvel + - interval_speed + - translation + - m200_d022 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.22 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/arc_real/towels_duration.yaml b/egomimic/hydra_configs/experiment/arc_real/towels_duration.yaml new file mode 100644 index 0000000000..748c66b0b6 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/towels_duration.yaml @@ -0,0 +1,94 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/towels_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: towels + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: towels_duration +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - towels + - since0924plus93 + - hpt300 + - arcdur + - absolute_waypoint_time + - isotonic + - translation + - m200_d033 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_clock_mode: duration + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.33 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/arc_real/towels_time.yaml b/egomimic/hydra_configs/experiment/arc_real/towels_time.yaml new file mode 100644 index 0000000000..ca66de8a8a --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/towels_time.yaml @@ -0,0 +1,69 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- override /model: hpt/yam_300m +- override /data: abc_arc/towels +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: time + spread: towels + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: 0.4 + M: 100 + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT +name: towels_time +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: 0.4 + resampled_vector_length: 100 +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - towels + - since0924plus93 + - hpt300 + - time + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: baseline + min_distance_unit: 0.4 + resampled_vector_length: 100 diff --git a/egomimic/hydra_configs/experiment/arc_real/towels_velocity.yaml b/egomimic/hydra_configs/experiment/arc_real/towels_velocity.yaml new file mode 100644 index 0000000000..3c90977788 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_real/towels_velocity.yaml @@ -0,0 +1,92 @@ +# @package _global_ +defaults: +- /model/base/real_yam_settings +- /model_contract/arc_clock_settings +- override /model: hpt/yam_300m_arc +- override /data: abc_arc/towels_arc +- override /evaluator: eval_open_loop_sim +- override /trainer: real_yam +- override /callbacks: real_yam +- override /logger: wandb +- _self_ +yam: + policy_domain: yam + variant: arc + spread: towels + train_root: unused-s3-resolver + valid_root: unused-s3-resolver + train_mode: total + valid_mode: total + D: ${abc.arc_distance} + M: ${abc.arc_waypoints} + time_rows: 100 + chunk_length: 100 + h_match_frames: 40 + limit_val_batches: 40 + prog_horizon_m: 0.19 + anchor_sampler: null + arcmatch_gt_span_m: null + action_dim: 14 + action_horizon: 100 + image_weights: DEFAULT + yam_source_frames: ${abc.arc_source_buffer_frames} +name: towels_velocity +description: h840t19_d320x6_cos10k_240k_${name} +arc_tokenizer: + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} +norm_stats: + sample_frac: 0.2 + num_workers: 8 + save_cache_dir: null +hpt: + action_horizon: ${abc.arc_token_rows} + token_dim: ${abc.arc_token_dim} +ft: + max_steps: 240000 + warmup_steps: 10000 +init_weights_from: null +paths: + dataset_dir: /storage/project/r-dxu345-0/shared/egoverseS3ZarrDatasets + work_dir: null +logger: + wandb: + save_dir: null + id: ${name}_s${seed} + project: arc + tags: + - rl2 + - towels + - since0924plus93 + - hpt300 + - arcvel + - interval_speed + - translation + - m200_d033 + - open_loop + resume: never +evaluator: + arc_video_trajectory_cap_mode: distance + limit_val_episodes: null + execute_fraction: 0.25 + distance_dtw_enabled: false + viz_every_n_epochs: 0 + ground_truth_action_key: actions_time + action_mode: arc + min_distance_unit: ${abc.arc_distance} + resampled_vector_length: ${abc.arc_waypoints} + arc_execution_cap_mode: waypoints + arc_chunking_mode: ${abc.arc_chunking_mode} + rotation_distance_unit: ${abc.arc_rotation_distance} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} +abc: + arc_streams: translation + action_mode: hybrid_arc_tokenizer_cartesian + arc_chunking_mode: multistream + arc_distance: 0.33 + arc_rotation_distance: 1.7453292519943295 + arc_waypoints: 200 + arc_source_buffer_frames: 100 + arc_gripper_stream_mode: events + arc_gripper_distance: null diff --git a/egomimic/hydra_configs/experiment/sim/libero/README.md b/egomimic/hydra_configs/experiment/arc_sim/libero/README.md similarity index 96% rename from egomimic/hydra_configs/experiment/sim/libero/README.md rename to egomimic/hydra_configs/experiment/arc_sim/libero/README.md index 1058971b18..105fad966c 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/README.md +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/README.md @@ -7,7 +7,7 @@ DP uses batch16, accumulation1 and suite-specific retained optimizer-step target Both use the released OAT observations/actions and 10% validation with seed42. Superseded AV0, native legacy and batch1024 recipes are preserved under -`experiment/sim/libero_historical`. They remain checkpoint reproduction recipes; +`experiment/arc_sim/libero_historical`. They remain checkpoint reproduction recipes; their names must not be used as current launch defaults. The historical native launcher remains routed to its historical profile and cannot launch current OAT-matched models without separate maintained launcher adoption and preflight. diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml index 2f763019b8..cac81924c5 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero10_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/action_flow_oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml index 8ef37f04bc..8602d00450 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_goal_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/action_flow_oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml index a0fc3d9a7a..d8692d2b12 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_object_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/action_flow_oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml index 622ecb76f1..9e000d23c0 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/action_flow_oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml similarity index 98% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml index b67b56bb65..1d9d934017 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Two action interfaces, with the released Spatial observation/AF components. defaults: - - /experiment/sim/libero/action_flow_libero_spatial_oat_dp_matched_s42 + - /experiment/arc_sim/libero/action_flow_libero_spatial_oat_dp_matched_s42 - /model/libero/action_flow_oat_dp_matched@_global_.libero_blueprint - override /evaluator: eval_planar_v2 - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml index 983788804a..a4b1f6c41b 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 +- /experiment/arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 - _self_ name: action_flow_libero_spatial_sharpa_mot_s42 description: MoT latent-token parameter sharing across original7D and operator-hand69D diff --git a/egomimic/hydra_configs/experiment/sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml index ec72eab478..69ac303e7e 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero10_oat_batch16_keep_steps_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Preserve the selected task-local batch16 recipe; the older global1024 profile stays separate. defaults: - - /experiment/sim/libero_historical/dp_libero10_oat_dp_matched_s42 + - /experiment/arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42 - _self_ name: dp_libero10_oat_batch16_keep_steps_s42 benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml index c0c028db08..9be19b7253 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_goal_oat_batch16_keep_steps_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Preserve the selected task-local batch16 recipe; the older global1024 profile stays separate. defaults: - - /experiment/sim/libero_historical/dp_libero_goal_oat_dp_matched_s42 + - /experiment/arc_sim/libero_historical/dp_libero_goal_oat_dp_matched_s42 - _self_ name: dp_libero_goal_oat_batch16_keep_steps_s42 benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml index a38fa905c5..ceada14025 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_object_oat_batch16_keep_steps_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Preserve the selected task-local batch16 recipe; the older global1024 profile stays separate. defaults: - - /experiment/sim/libero_historical/dp_libero_object_oat_dp_matched_s42 + - /experiment/arc_sim/libero_historical/dp_libero_object_oat_dp_matched_s42 - _self_ name: dp_libero_object_oat_batch16_keep_steps_s42 benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml similarity index 88% rename from egomimic/hydra_configs/experiment/sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml index 71b837a826..b923ebb4b8 100644 --- a/egomimic/hydra_configs/experiment/sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero/dp_libero_spatial_oat_batch16_keep_steps_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Preserve the selected task-local batch16 recipe; the older global1024 profile stays separate. defaults: - - /experiment/sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42 + - /experiment/arc_sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42 - _self_ name: dp_libero_spatial_oat_batch16_keep_steps_s42 benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/README.md b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/README.md similarity index 80% rename from egomimic/hydra_configs/experiment/sim/libero_historical/README.md rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/README.md index 98e9d44242..aba03db798 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/README.md +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/README.md @@ -1,8 +1,8 @@ # Historical LIBERO recipes These superseded recipes preserve the old architecture/objective and batch1024 -training contracts. Select them explicitly with `+experiment=sim/libero_historical/...` -only for historical reproduction. Current choices live in `experiment/sim/libero`. +training contracts. Select them explicitly with `+experiment=arc_sim/libero_historical/...` +only for historical reproduction. Current choices live in `experiment/arc_sim/libero`. Existing checkpoint-bound source trees remain unchanged and retain original paths. Do not silently redirect a historical checkpoint or native launcher to a current diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml index 4a5ca29024..d55f3b58e7 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_avln_80k_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_base + - /experiment/arc_sim/oat/libero_base - override /data: libero/action_flow_libero10_val01_h16 - override /model: bf/libero10_action_flow_latent_fm_sg_unite_h384 - override /callbacks: action_flow_avln_libero diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml similarity index 74% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml index b459669f54..ea1039444c 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Matched native noaug ablation: disable only decoded-action velocity loss. defaults: - - /experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero10_h240_euler50_dithalf_80k_av0_s42 run_provenance: diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml similarity index 94% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml index 94e82ccf2f..fd9e64196c 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Candidate only: maintained launcher whitelist/source audit/smoke still required. defaults: - - /experiment/sim/libero_historical/action_flow_libero10_avln_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero10_avln_80k_s42 - override /model: bf/libero10_action_flow_latent_fm_sg_unite_h240_euler50_dithalf - override /callbacks: action_flow_avln_libero_dit_half - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml similarity index 74% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml index 651616ce1f..5652ad8888 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Matched native noaug ablation: disable only decoded-action velocity loss. defaults: - - /experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42 run_provenance: diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml similarity index 85% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml index 50740a3c5d..3cfcb912fc 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Shared native noaug model/optimizer/diagnostics; only suite/data identity differs. defaults: - - /experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_goal_h240_euler50_dithalf_80k_s42 description: "LIBERO-Goal native7D H240/H16 B32 seed42 Euler50 DiT-half noaug" diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml similarity index 73% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml index d4197a31d1..980b8b2852 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Matched native noaug ablation: disable only decoded-action velocity loss. defaults: - - /experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42 run_provenance: diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml similarity index 85% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml index 0a66d85de0..34d2aaae55 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Shared native noaug model/optimizer/diagnostics; only suite/data identity differs. defaults: - - /experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_object_h240_euler50_dithalf_80k_s42 description: "LIBERO-Object native7D H240/H16 B32 seed42 Euler50 DiT-half noaug" diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml similarity index 73% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml index a85ff25d8c..9beb102d7f 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Matched native noaug ablation: disable only decoded-action velocity loss. defaults: - - /experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42 run_provenance: diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml similarity index 79% rename from egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml index 6882e851f7..636b641a44 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42.yaml @@ -1,7 +1,7 @@ # @package _global_ # Shared native noaug model/optimizer/diagnostics; only suite/data identity differs. defaults: - - /experiment/sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 + - /experiment/arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42 - _self_ name: action_flow_libero_spatial_h240_euler50_dithalf_80k_s42 description: "LIBERO-Spatial native7D H240/H16 B32 seed42 Euler50 DiT-half noaug" diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml similarity index 92% rename from egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml index 78bc0ca6cc..646eec0558 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml similarity index 92% rename from egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml index 0af08e41f0..dd782022a3 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_goal_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml similarity index 92% rename from egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml index 775f11c970..5dae0db0c2 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_object_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml similarity index 92% rename from egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml index 5d855ebcf9..2406aae321 100644 --- a/egomimic/hydra_configs/experiment/sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/libero_historical/dp_libero_spatial_oat_dp_matched_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/oat/libero_base +- /experiment/arc_sim/oat/libero_base - override /model: libero/oat_dp_matched - override /logger: wandb - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml similarity index 77% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml index 5334772956..760f837772 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_1_r192_d1p6_m24.yaml @@ -1,6 +1,6 @@ # @package _global_ # Calibration shortlist shared across all five suites; see docs/results/libero_arc_common_configs_20260921.md. -defaults: [/experiment/sim/oat/libero_arc_dur_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_dur_policy, _self_] benchmark: arc_max_rotation_degrees: 192 arc_max_translation: 1.6 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml similarity index 77% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml index 7006455dd3..6397a2c434 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_2_r384_d0p8_m32.yaml @@ -1,6 +1,6 @@ # @package _global_ # Calibration shortlist shared across all five suites; see docs/results/libero_arc_common_configs_20260921.md. -defaults: [/experiment/sim/oat/libero_arc_dur_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_dur_policy, _self_] benchmark: arc_max_rotation_degrees: 384 arc_max_translation: 0.8 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_policy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_policy.yaml similarity index 52% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_policy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_policy.yaml index 036b1b4dbd..f8f830ec06 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_dur_policy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_dur_policy.yaml @@ -1,5 +1,5 @@ # @package _global_ -defaults: [/experiment/sim/oat/libero_arc_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_policy, _self_] benchmark: arc_mode: dur arc_action_dim: 12 diff --git a/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_chebyshev.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_chebyshev.yaml new file mode 100644 index 0000000000..098049ea05 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_chebyshev.yaml @@ -0,0 +1,4 @@ +# @package _global_ +defaults: [/experiment/arc_sim/oat/libero_arc_global_common, _self_] +benchmark: + arc_basis: chebyshev diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_common.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_common.yaml similarity index 94% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_common.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_common.yaml index ef4f15b396..59a0f0db5c 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_common.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_common.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_arc_oat_dp_policy + - /experiment/arc_sim/oat/libero_arc_oat_dp_policy - _self_ # Experimental matched-basis comparison, not released OAT baseline parity. diff --git a/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_fourier.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_fourier.yaml new file mode 100644 index 0000000000..e707be33bc --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_fourier.yaml @@ -0,0 +1,4 @@ +# @package _global_ +defaults: [/experiment/arc_sim/oat/libero_arc_global_common, _self_] +benchmark: + arc_basis: fourier diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_pilot.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_pilot.yaml similarity index 93% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_pilot.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_pilot.yaml index 2f96595fbc..650017a284 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_pilot.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_pilot.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_arc_global_common + - /experiment/arc_sim/oat/libero_arc_global_common - _self_ # Same model/data/optimizer contract for all bases; select benchmark.arc_basis. diff --git a/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_uniform.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_uniform.yaml new file mode 100644 index 0000000000..f960eaf3c8 --- /dev/null +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_global_uniform.yaml @@ -0,0 +1,4 @@ +# @package _global_ +defaults: [/experiment/arc_sim/oat/libero_arc_global_common, _self_] +benchmark: + arc_basis: uniform diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_base.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_base.yaml similarity index 72% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_base.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_base.yaml index 5ac8bae55c..08c086de79 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_base.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_base.yaml @@ -1,5 +1,5 @@ # @package _global_ -defaults: [/experiment/sim/oat/libero_base, _self_] +defaults: [/experiment/arc_sim/oat/libero_base, _self_] benchmark: arc_mode: dur arc_action_dim: 12 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_dp_policy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_dp_policy.yaml similarity index 71% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_dp_policy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_dp_policy.yaml index 24341f7037..5d837b1019 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oat_dp_policy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oat_dp_policy.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_arc_policy + - /experiment/arc_sim/oat/libero_arc_policy - override /model: oat/arc_oat_dp_policy - _self_ benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oatpolicy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oatpolicy.yaml similarity index 62% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_oatpolicy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oatpolicy.yaml index 4edc9f9db1..4ebd531ddc 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oatpolicy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oatpolicy.yaml @@ -1,5 +1,5 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_arc_oat_base + - /experiment/arc_sim/oat/libero_arc_oat_base - override /model: oat/arc_oat_policy - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oattok.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oattok.yaml similarity index 69% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_oattok.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oattok.yaml index 84bcf2d58e..3f68834f90 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_oattok.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_oattok.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_arc_oat_base + - /experiment/arc_sim/oat/libero_arc_oat_base - override /model: oat/arc_tokenizer - _self_ benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_policy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_policy.yaml similarity index 65% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_policy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_policy.yaml index 5900cd0576..1e95e9c371 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_policy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_policy.yaml @@ -1,5 +1,5 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_base + - /experiment/arc_sim/oat/libero_base - override /model: oat/arc_policy - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_shared_r384_d1p6_m36.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_shared_r384_d1p6_m36.yaml similarity index 83% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_shared_r384_d1p6_m36.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_shared_r384_d1p6_m36.yaml index 7dab7e3ca1..87bfa240f5 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_shared_r384_d1p6_m36.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_shared_r384_d1p6_m36.yaml @@ -1,6 +1,6 @@ # @package _global_ # Calibration shortlist shared across all five suites; see docs/results/libero_arc_common_configs_20260921.md. -defaults: [/experiment/sim/oat/libero_arc_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_policy, _self_] benchmark: # Supply benchmark.arc_mode=stk or dur explicitly. arc_mode: null diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml similarity index 77% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml index 22e927823e..940e343846 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_1_r192_d1p6_m36.yaml @@ -1,6 +1,6 @@ # @package _global_ # Calibration shortlist shared across all five suites; see docs/results/libero_arc_common_configs_20260921.md. -defaults: [/experiment/sim/oat/libero_arc_stk_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_stk_policy, _self_] benchmark: arc_max_rotation_degrees: 192 arc_max_translation: 1.6 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml similarity index 77% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml index a1bf38a23d..c23b702f6e 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_2_r192_d0p8_m32.yaml @@ -1,6 +1,6 @@ # @package _global_ # Calibration shortlist shared across all five suites; see docs/results/libero_arc_common_configs_20260921.md. -defaults: [/experiment/sim/oat/libero_arc_stk_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_stk_policy, _self_] benchmark: arc_max_rotation_degrees: 192 arc_max_translation: 0.8 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_policy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_policy.yaml similarity index 52% rename from egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_policy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_policy.yaml index 599fa174b2..a88d43ea1d 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_stk_policy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_arc_stk_policy.yaml @@ -1,5 +1,5 @@ # @package _global_ -defaults: [/experiment/sim/oat/libero_arc_policy, _self_] +defaults: [/experiment/arc_sim/oat/libero_arc_policy, _self_] benchmark: arc_mode: stk arc_action_dim: 12 diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_base.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_base.yaml similarity index 100% rename from egomimic/hydra_configs/experiment/sim/oat/libero_base.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_base.yaml diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_oatpolicy.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_oatpolicy.yaml similarity index 64% rename from egomimic/hydra_configs/experiment/sim/oat/libero_oatpolicy.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_oatpolicy.yaml index 0d77b2a049..92cc124be7 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_oatpolicy.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_oatpolicy.yaml @@ -1,5 +1,5 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_base + - /experiment/arc_sim/oat/libero_base - override /model: oat/policy - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_oattok.yaml b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_oattok.yaml similarity index 72% rename from egomimic/hydra_configs/experiment/sim/oat/libero_oattok.yaml rename to egomimic/hydra_configs/experiment/arc_sim/oat/libero_oattok.yaml index f3374a8f25..89a3306ab2 100644 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_oattok.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/oat/libero_oattok.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/oat/libero_base + - /experiment/arc_sim/oat/libero_base - override /model: oat/tokenizer - _self_ benchmark: diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml similarity index 95% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml index c8d5ac07a6..8ca56f5ef9 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_usocket_candidate_common + - /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /model: bf/us_action_flow_bc_bridge_likelihood - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml index 6696fb6763..97d4a065ae 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_graph_section_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_usocket_candidate_common + - /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /model: bf/us_action_flow_bc_graph_section - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml similarity index 98% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml index e8a8497fb5..c3c5dba70e 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_usocket_candidate_common + - /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /model: bf/us_action_flow_bc_latent_fm_sg_200m_adamw - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml similarity index 98% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml index 637a40b472..ae340164b5 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_usocket_candidate_common + - /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /model: bf/us_action_flow_bc_latent_fm_sg_200m_muon - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml index b49ae8f6b7..f7aeaf6173 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_usocket_candidate_common + - /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /model: bf/us_action_flow_bc_latent_fm_sg - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon100_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon100_s42.yaml similarity index 73% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon100_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon100_s42.yaml index 6a4046b17b..8812de9c64 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon100_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon100_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_bc_usocket_recon1_s42 + - /experiment/arc_sim/pusht/action_flow_bc_usocket_recon1_s42 - _self_ name: action_flow_bc_usocket_recon100_s42 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_s42.yaml similarity index 72% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_s42.yaml index 03252fdeef..0b494696a2 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_bc_usocket_recon1_s42 + - /experiment/arc_sim/pusht/action_flow_bc_usocket_recon1_s42 - _self_ name: action_flow_bc_usocket_recon10_s42 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml similarity index 83% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml index 74892f1b97..71c860d30a 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/action_flow_bc_usocket_recon10_s42 + - /experiment/arc_sim/pusht/action_flow_bc_usocket_recon10_s42 - _self_ name: action_flow_bc_usocket_recon10_warmup10k_s42 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon1_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon1_s42.yaml similarity index 99% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon1_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon1_s42.yaml index 757b58cfe4..12f4940e00 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_bc_usocket_recon1_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_bc_usocket_recon1_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_base + - /experiment/arc_sim/pusht/planar_v2_base - override /data: pusht/action_flow_usocket_val01_h16_common4 - override /model: bf/us_action_flow_bc_nt16_d8_h512 - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml index e52207a409..2c4301a68a 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/parents/action_flow_usocket_h384_sum14 + - /experiment/arc_sim/pusht/parents/action_flow_usocket_h384_sum14 - override /data: pusht/action_flow_unite_chain_val01_h16 - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml index 83d450883a..5bda5ea1d9 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/parents/action_flow_cotrain_uc_manual4919 + - /experiment/arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919 - _self_ name: action_flow_cotrain_uc_multiplier_interpolation diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_candidate_common.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_candidate_common.yaml similarity index 99% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_candidate_common.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_candidate_common.yaml index 325fe397e6..f76a4e02d1 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_candidate_common.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_candidate_common.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_base + - /experiment/arc_sim/pusht/planar_v2_base - override /data: pusht/action_flow_usocket_val01_h16_common4 - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml similarity index 93% rename from egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml index 2def6a88fc..0f95e1d230 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/parents/action_flow_usocket_h384_sum14 + - /experiment/arc_sim/pusht/parents/action_flow_usocket_h384_sum14 - _self_ name: action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/action_latent_vfm_usocket_val01_h16.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/action_latent_vfm_usocket_val01_h16.yaml similarity index 100% rename from egomimic/hydra_configs/experiment/sim/pusht/action_latent_vfm_usocket_val01_h16.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/action_latent_vfm_usocket_val01_h16.yaml diff --git a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml similarity index 98% rename from egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml index b4634419d1..e9f41834bc 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/pusht/parents/action_flow_usocket_h384 +- /experiment/arc_sim/pusht/parents/action_flow_usocket_h384 - override /data: pusht/parents/action_flow_uc_h16 - override /model: bf/cotrain_uc_action_flow_latent_fm_sg_unite_h384d12h12 - override /callbacks: action_flow_avln_cotrain diff --git a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384.yaml similarity index 98% rename from egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384.yaml index 34894a2885..e33f8ba45f 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/pusht/action_flow_usocket_candidate_common +- /experiment/arc_sim/pusht/action_flow_usocket_candidate_common - override /data: pusht/action_flow_unite_usocket_val01_h16 - override /model: bf/us_action_flow_latent_fm_sg_unite_h384 - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384_sum14.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384_sum14.yaml similarity index 92% rename from egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384_sum14.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384_sum14.yaml index edf474c11f..ab70e3efe0 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/parents/action_flow_usocket_h384_sum14.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/parents/action_flow_usocket_h384_sum14.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: -- /experiment/sim/pusht/parents/action_flow_usocket_h384 +- /experiment/arc_sim/pusht/parents/action_flow_usocket_h384 - _self_ model: flow_loss_aggregation: sum_samples diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml similarity index 95% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml index 6ca0e3f2ed..ac3d2b4a3d 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked + - /experiment/arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked - override /model: bf/bf_planar_v2_dp_af_obs_multiplier_261m - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml similarity index 94% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml index 5cddddac2f..c613030a54 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_cotrain_clean_dp_standard + - /experiment/arc_sim/pusht/planar_v2_cotrain_clean_dp_standard - override /data: pusht/parents/standard_dp_uc_h16 - _self_ name: planar_usocket_chain_manual4919_standard_dp_retimed_masked_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_base.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_base.yaml similarity index 100% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_base.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_base.yaml diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_arc_bc.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_arc_bc.yaml similarity index 91% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_arc_bc.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_arc_bc.yaml index 1e42eef940..47b0e95000 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_arc_bc.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_arc_bc.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_chain_base + - /experiment/arc_sim/pusht/planar_v2_chain_base - override /model: bf/bf_planar_v2_flow - _self_ name: planar_v2_chain_arc_D200_M100 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_base.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_base.yaml similarity index 93% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_base.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_base.yaml index 29ae4ccd7f..ee3288e39b 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_base.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_base.yaml @@ -1,7 +1,7 @@ # @package _global_ # ChainGripper data, action-space identity, and split provenance. defaults: - - /experiment/sim/pusht/planar_v2_base + - /experiment/arc_sim/pusht/planar_v2_base - override /data: pusht/planar_v2_chain_gripper - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_direct_bc.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_direct_bc.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_direct_bc.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_direct_bc.yaml index e70dcca06e..bc137231c2 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_direct_bc.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_direct_bc.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_chain_base + - /experiment/arc_sim/pusht/planar_v2_chain_base - override /model: bf/bf_planar_v2_flow - _self_ name: planar_v2_chain_direct_bc_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_paper.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_paper.yaml similarity index 95% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_paper.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_paper.yaml index bd9f40a8be..a9435d0efa 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_paper.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_paper.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_chain_base + - /experiment/arc_sim/pusht/planar_v2_chain_base - override /model: bf/bf_planar_v2_dp_paper - _self_ name: planar_v2_chain_dp_paper_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_standard.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_standard.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_standard.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_standard.yaml index 621259bc8a..463a2b38d8 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_chain_dp_standard.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_chain_dp_standard.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_chain_base + - /experiment/arc_sim/pusht/planar_v2_chain_base - override /model: bf/bf_planar_v2_dp_standard - _self_ name: planar_v2_chain_dp_standard_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml index b349a912bc..47f0cf7c31 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_cotrain_clean_dp_standard.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_base + - /experiment/arc_sim/pusht/planar_v2_base - override /data: pusht/planar_v2_cotrain_clean - override /model: bf/bf_planar_v2_dp_standard - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_bc.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_bc.yaml similarity index 90% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_bc.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_bc.yaml index aa7fe6c91a..0587815eb1 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_bc.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_bc.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_flow - _self_ name: planar_v2_usocket_arc_D200_M100 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_graph_tok.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_graph_tok.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_graph_tok.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_graph_tok.yaml index 28e27c8e83..3d75796d53 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_graph_tok.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_graph_tok.yaml @@ -6,7 +6,7 @@ # window parameters live in one place (the `planar` block below) and are read by # both the tokenize and detokenize nodes. defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_arc_graph_tok - _self_ name: planar_v2_usocket_arc_graph_tok_D200_M100 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml similarity index 94% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml index aa90cc7dff..60b026d6bc 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc.yaml @@ -3,7 +3,7 @@ # by 24 degrees of accumulated yaw. The tokenizer's chordal angular metric is # sqrt(2) * sin(radians(24) / 4) = 0.14776. defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_flow - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml similarity index 78% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml index 61681aba25..5c21f60051 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg.yaml @@ -1,7 +1,7 @@ # @package _global_ # Paired with the uniform Paper-DP row; only waypoint allocation changes. defaults: - - /experiment/sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg + - /experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg - _self_ name: planar_v2_usocket_arc_paper_curvature_D40_M16_R24deg diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml similarity index 96% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml index ae33032e6f..e165fd5351 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg.yaml @@ -1,7 +1,7 @@ # @package _global_ # Paper-DP baseline for the duration-timed D40/M16/R24-degree ARC token. defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_dp_paper - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_base.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_base.yaml similarity index 97% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_base.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_base.yaml index 096d954c28..6b5d48b289 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_base.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_base.yaml @@ -1,7 +1,7 @@ # @package _global_ # U-Socket data, action-space identity, and split provenance. defaults: - - /experiment/sim/pusht/planar_v2_base + - /experiment/arc_sim/pusht/planar_v2_base - override /data: pusht/planar_v2_usocket - _self_ diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_direct_bc.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_direct_bc.yaml similarity index 85% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_direct_bc.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_direct_bc.yaml index d269a56a5c..f47a84846f 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_direct_bc.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_direct_bc.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_flow - _self_ name: planar_v2_usocket_direct_bc_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_paper.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_paper.yaml similarity index 94% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_paper.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_paper.yaml index 98a58b3fde..3e540a1e28 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_paper.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_paper.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_dp_paper - _self_ name: planar_v2_usocket_dp_paper_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_standard.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_standard.yaml similarity index 86% rename from egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_standard.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_standard.yaml index 14c99cc725..4f6292c32d 100644 --- a/egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_dp_standard.yaml +++ b/egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_dp_standard.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /experiment/sim/pusht/planar_v2_usocket_base + - /experiment/arc_sim/pusht/planar_v2_usocket_base - override /model: bf/bf_planar_v2_dp_standard - _self_ name: planar_v2_usocket_dp_standard_h16 diff --git a/egomimic/hydra_configs/experiment/sim/pusht/unite_cotrain_usocket_chain_val01_h16.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/unite_cotrain_usocket_chain_val01_h16.yaml similarity index 100% rename from egomimic/hydra_configs/experiment/sim/pusht/unite_cotrain_usocket_chain_val01_h16.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/unite_cotrain_usocket_chain_val01_h16.yaml diff --git a/egomimic/hydra_configs/experiment/sim/pusht/unite_usocket_register_sweep_val01_h16.yaml b/egomimic/hydra_configs/experiment/arc_sim/pusht/unite_usocket_register_sweep_val01_h16.yaml similarity index 100% rename from egomimic/hydra_configs/experiment/sim/pusht/unite_usocket_register_sweep_val01_h16.yaml rename to egomimic/hydra_configs/experiment/arc_sim/pusht/unite_usocket_register_sweep_val01_h16.yaml diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/direct_fm3d_gaussian_paraboloid.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/direct_fm3d_gaussian_paraboloid.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/direct_fm3d_gaussian_paraboloid.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/direct_fm3d_gaussian_paraboloid.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/direct_fm3d_gaussian_torus.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/direct_fm3d_gaussian_torus.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/direct_fm3d_gaussian_torus.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/direct_fm3d_gaussian_torus.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent2_codec10k.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent2_codec10k.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent2_codec10k.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent2_codec10k.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4_codec10k.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4_codec10k.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4_codec10k.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4_codec10k.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4.json diff --git a/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json b/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json similarity index 100% rename from egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json rename to egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json diff --git a/egomimic/hydra_configs/experiment/real/earbuds_duration.yaml b/egomimic/hydra_configs/experiment/real/earbuds_duration.yaml deleted file mode 100644 index 5d74559d53..0000000000 --- a/egomimic/hydra_configs/experiment/real/earbuds_duration.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Absolute-duration single-clock ARC (M 200, D 0.22 m, 100-frame window, speed-bounded isotonic decode) on the -# earbuds_time pool and graph. -defaults: - - /experiment/real/earbuds_time - - /experiment/_shared/arc_single_clock - - _self_ -name: earbuds_duration -abc: - arc_distance: 0.22 -logger: - wandb: - tags: [rl2, earbuds, plus69, hpt300, arcdur, absolute_waypoint_time, isotonic, translation, m200_d022, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/earbuds_time.yaml b/egomimic/hydra_configs/experiment/real/earbuds_time.yaml deleted file mode 100644 index cc14fb3802..0000000000 --- a/egomimic/hydra_configs/experiment/real/earbuds_time.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Time baseline: HPT-300M flow on 100-frame time chunks, the frozen wireless_earbuds_updated + Elmo's 69 takes of 2026-10-20/21 (339 train / 15 val), -# complete-episode open-loop validation on the 25 % prefix. -defaults: - - /experiment/_shared/frozen_yam/hpt300_openloop - - override /data: abc_arc/earbuds - - _self_ -name: earbuds_time -frozen_yam: - spread: earbuds -logger: - wandb: - tags: [rl2, earbuds, plus69, hpt300, time, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/earbuds_velocity.yaml b/egomimic/hydra_configs/experiment/real/earbuds_velocity.yaml deleted file mode 100644 index 0d0ea1880d..0000000000 --- a/egomimic/hydra_configs/experiment/real/earbuds_velocity.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# @package _global_ -# Interval arcvel: the earbuds_duration recipe with velocity clocks (each waypoint segment's mean speed). -defaults: - - /experiment/real/earbuds_duration - - _self_ -name: earbuds_velocity -abc: - arc_clock_mode: velocity -logger: - wandb: - tags: [rl2, earbuds, plus69, hpt300, arcvel, interval_speed, translation, m200_d022, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/stationary_duration.yaml b/egomimic/hydra_configs/experiment/real/stationary_duration.yaml deleted file mode 100644 index 5eb14db111..0000000000 --- a/egomimic/hydra_configs/experiment/real/stationary_duration.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Absolute-duration single-clock ARC (M 200, D 0.22 m, 100-frame window, speed-bounded isotonic decode) on the -# stationary_time pool and graph. -defaults: - - /experiment/real/stationary_time - - /experiment/_shared/arc_single_clock - - _self_ -name: stationary_duration -abc: - arc_distance: 0.22 -logger: - wandb: - tags: [rl2, stationery, sp366e51, hpt300, arcdur, absolute_waypoint_time, isotonic, translation, m200_d022, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/stationary_time.yaml b/egomimic/hydra_configs/experiment/real/stationary_time.yaml deleted file mode 100644 index 2f31df9039..0000000000 --- a/egomimic/hydra_configs/experiment/real/stationary_time.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Time baseline: HPT-300M flow on 100-frame time chunks, the frozen slow-pace 366 + Elmo 51 organize_stationary pool (417 train / 24 val), -# complete-episode open-loop validation on the 25 % prefix. -defaults: - - /experiment/_shared/frozen_yam/hpt300_openloop - - override /data: abc_arc/stationary - - _self_ -name: stationary_time -frozen_yam: - spread: stationery -logger: - wandb: - tags: [rl2, stationery, sp366e51, hpt300, time, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/stationary_velocity.yaml b/egomimic/hydra_configs/experiment/real/stationary_velocity.yaml deleted file mode 100644 index 8630f78bc3..0000000000 --- a/egomimic/hydra_configs/experiment/real/stationary_velocity.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# @package _global_ -# Interval arcvel: the stationary_duration recipe with velocity clocks (each waypoint segment's mean speed). -defaults: - - /experiment/real/stationary_duration - - _self_ -name: stationary_velocity -abc: - arc_clock_mode: velocity -logger: - wandb: - tags: [rl2, stationery, sp366e51, hpt300, arcvel, interval_speed, translation, m200_d022, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/towels_duration.yaml b/egomimic/hydra_configs/experiment/real/towels_duration.yaml deleted file mode 100644 index a9cd131c5c..0000000000 --- a/egomimic/hydra_configs/experiment/real/towels_duration.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Absolute-duration single-clock ARC (M 200, D 0.33 m, 100-frame window, speed-bounded isotonic decode) on the -# towels_time pool and graph. -defaults: - - /experiment/real/towels_time - - /experiment/_shared/arc_single_clock - - _self_ -name: towels_duration -abc: - arc_distance: 0.33 -logger: - wandb: - tags: [rl2, towels, since0924plus93, hpt300, arcdur, absolute_waypoint_time, isotonic, translation, m200_d033, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/towels_time.yaml b/egomimic/hydra_configs/experiment/real/towels_time.yaml deleted file mode 100644 index 407bd4fe35..0000000000 --- a/egomimic/hydra_configs/experiment/real/towels_time.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Time baseline: HPT-300M flow on 100-frame time chunks, the frozen since0924 + Aidan's 93 fold_towels takes (462 train / 17 val), -# complete-episode open-loop validation on the 25 % prefix. -defaults: - - /experiment/_shared/frozen_yam/hpt300_openloop - - override /data: abc_arc/towels - - _self_ -name: towels_time -frozen_yam: - spread: towels -logger: - wandb: - tags: [rl2, towels, since0924plus93, hpt300, time, open_loop] diff --git a/egomimic/hydra_configs/experiment/real/towels_velocity.yaml b/egomimic/hydra_configs/experiment/real/towels_velocity.yaml deleted file mode 100644 index c249b4022d..0000000000 --- a/egomimic/hydra_configs/experiment/real/towels_velocity.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# @package _global_ -# Interval arcvel: the towels_duration recipe with velocity clocks (each waypoint segment's mean speed). -defaults: - - /experiment/real/towels_duration - - _self_ -name: towels_velocity -abc: - arc_clock_mode: velocity -logger: - wandb: - tags: [rl2, towels, since0924plus93, hpt300, arcvel, interval_speed, translation, m200_d033, open_loop] diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_chebyshev.yaml b/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_chebyshev.yaml deleted file mode 100644 index 3dc1f679ec..0000000000 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_chebyshev.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# @package _global_ -defaults: [/experiment/sim/oat/libero_arc_global_common, _self_] -benchmark: - arc_basis: chebyshev diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_fourier.yaml b/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_fourier.yaml deleted file mode 100644 index 302e8e4a15..0000000000 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_fourier.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# @package _global_ -defaults: [/experiment/sim/oat/libero_arc_global_common, _self_] -benchmark: - arc_basis: fourier diff --git a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_uniform.yaml b/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_uniform.yaml deleted file mode 100644 index 42fefa88b2..0000000000 --- a/egomimic/hydra_configs/experiment/sim/oat/libero_arc_global_uniform.yaml +++ /dev/null @@ -1,4 +0,0 @@ -# @package _global_ -defaults: [/experiment/sim/oat/libero_arc_global_common, _self_] -benchmark: - arc_basis: uniform diff --git a/egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml b/egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml index 236e81f786..5f149b94c7 100644 --- a/egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml +++ b/egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml @@ -7,6 +7,9 @@ # to quat or 6D makes that 16 or 20, and BOTH action_dim and the denoiser's # input_dim have to move with it -- DiffusionNoisingStage validates the target's # width, so a mismatch fails at the first batch rather than training silently. +defaults: + - /model/base/yam_diffusion_settings@_global_ + - _self_ _target_: egomimic.pl_utils.pl_model.ModelWrapper pipeline: diff --git a/egomimic/hydra_configs/model/hpt_cotrain_enc_dec_base.yaml b/egomimic/hydra_configs/model/base/hpt_cotrain_enc_dec.yaml similarity index 100% rename from egomimic/hydra_configs/model/hpt_cotrain_enc_dec_base.yaml rename to egomimic/hydra_configs/model/base/hpt_cotrain_enc_dec.yaml diff --git a/egomimic/hydra_configs/model/hpt_cotrain_flow_shared_head.yaml b/egomimic/hydra_configs/model/base/hpt_cotrain_flow.yaml similarity index 96% rename from egomimic/hydra_configs/model/hpt_cotrain_flow_shared_head.yaml rename to egomimic/hydra_configs/model/base/hpt_cotrain_flow.yaml index a66c40c87c..6b9fd4d279 100644 --- a/egomimic/hydra_configs/model/hpt_cotrain_flow_shared_head.yaml +++ b/egomimic/hydra_configs/model/base/hpt_cotrain_flow.yaml @@ -1,4 +1,4 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. +# Shared model definition; concrete recipes preserve their own source/timing settings. _target_: egomimic.pl_utils.pl_model.ModelWrapper optimizer: _target_: torch.optim.AdamW @@ -449,17 +449,12 @@ inference_domains: timing: kind: uniform_resampled_target temporally_resolved: true - dt: 0.00909090909090909 first_source_frame: 0 - last_source_frame: 27 source_fps: 30 output: frame: camframe key: actions representation: cartesian - shape: - - 28 - - 12 timing: kind: control_period temporally_resolved: true @@ -480,16 +475,11 @@ inference_domains: - 12 adapter: _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 28 - - 12 decoder: _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder native_shape: - 100 - 12 - native_dt: 0.00909090909090909 - output_horizon: 28 output_dt: 0.03333333333333333 angular_indices: - 3 @@ -504,9 +494,7 @@ inference_domains: description: Control-rate execution prefix. type: integer min: 1 - max: 28 step: 1 - default: 28 target: kind: policy_attribute attribute_path: replan_every @@ -523,8 +511,6 @@ inference_domains: stage_id: sampler attribute_path: num_inference_steps inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. data_requirements: preprocessing: '6': @@ -538,5 +524,4 @@ data_requirements: action_mode: cartesian coord_frame: camframe rotation_mode: euler - stride: 3 source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/hpt_bc_flow_human.yaml b/egomimic/hydra_configs/model/base/hpt_human_flow.yaml similarity index 93% rename from egomimic/hydra_configs/model/hpt_bc_flow_human.yaml rename to egomimic/hydra_configs/model/base/hpt_human_flow.yaml index 84314edc2d..a163c1113a 100644 --- a/egomimic/hydra_configs/model/hpt_bc_flow_human.yaml +++ b/egomimic/hydra_configs/model/base/hpt_human_flow.yaml @@ -1,4 +1,4 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. +# Shared model definition; concrete recipes preserve their own source/timing settings. _target_: egomimic.pl_utils.pl_model.ModelWrapper optimizer: _target_: torch.optim.AdamW @@ -160,17 +160,12 @@ inference_domains: timing: kind: uniform_resampled_target temporally_resolved: true - dt: 0.00909090909090909 first_source_frame: 0 - last_source_frame: 27 source_fps: 30 output: frame: camframe key: actions representation: cartesian - shape: - - 28 - - 12 timing: kind: control_period temporally_resolved: true @@ -191,16 +186,11 @@ inference_domains: - 12 adapter: _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 28 - - 12 decoder: _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder native_shape: - 100 - 12 - native_dt: 0.00909090909090909 - output_horizon: 28 output_dt: 0.03333333333333333 angular_indices: - 3 @@ -215,9 +205,7 @@ inference_domains: description: Control-rate execution prefix. type: integer min: 1 - max: 28 step: 1 - default: 28 target: kind: policy_attribute attribute_path: replan_every @@ -234,8 +222,6 @@ inference_domains: stage_id: sampler attribute_path: num_inference_steps inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. data_requirements: preprocessing: '3': @@ -243,5 +229,4 @@ data_requirements: action_mode: cartesian coord_frame: camframe rotation_mode: euler - stride: 3 source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/hpt_bc_keypoints_base.yaml b/egomimic/hydra_configs/model/base/hpt_keypoints.yaml similarity index 100% rename from egomimic/hydra_configs/model/hpt_bc_keypoints_base.yaml rename to egomimic/hydra_configs/model/base/hpt_keypoints.yaml diff --git a/egomimic/hydra_configs/model/base/hpt_pickplace_qwen.yaml b/egomimic/hydra_configs/model/base/hpt_pickplace_qwen.yaml new file mode 100644 index 0000000000..61625301dc --- /dev/null +++ b/egomimic/hydra_configs/model/base/hpt_pickplace_qwen.yaml @@ -0,0 +1,126 @@ +# Shared model definition; concrete recipes preserve their own source/timing settings. +_target_: egomimic.pl_utils.pl_model.ModelWrapper +optimizer: + _target_: torch.optim.AdamW + _partial_: true + lr: 0.0002 + weight_decay: 0.0001 +scheduler: + _target_: egomimic.utils.schedulers.warmup_then_cosine + _partial_: true + warmup_epochs: 100 + total_epochs: 2000 + eta_min: 1.0e-05 +scheduler_interval: epoch +pipeline: + _target_: egomimic.pipeline.algo.PipelineAlgo + stage_ids: + stems: 1 + trunk: 2 + sampler: 5 +deployment_domain: eva_bimanual +inference_domains: + eva_bimanual: + input: + history_length: 1 + keys: + - observations.state.ee_pose + - observations.images.front_img_1 + - annotations + - embodiment + history_keys: + - observations.state.ee_pose + - observations.images.front_img_1 + constants: + embodiment: 6 + native_output: + frame: camframe + key: pred_action + representation: normalized_action + shape: + - 100 + - 14 + timing: + kind: uniform_resampled_target + temporally_resolved: true + dt: 0.014814814814814815 + first_source_frame: 0 + last_source_frame: 44 + source_fps: 30 + output: + frame: camframe + key: actions + representation: cartesian + shape: + - 45 + - 14 + timing: + kind: control_period + temporally_resolved: true + dt: 0.03333333333333333 + compatibility: + normalizer_schema: + action_key: actions_cartesian + native_shape: + - 100 + - 14 + embodiment: 6 + tokenizer: null + profiles: + default: + stage_id: sampler + native_shape: + - 100 + - 14 + adapter: + _target_: egomimic.robot.graph_policy.CartesianGraphAdapter + decoder: + _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder + native_shape: + - 100 + - 14 + native_dt: 0.014814814814814815 + output_horizon: 45 + output_dt: 0.03333333333333333 + angular_indices: + - 3 + - 4 + - 5 + - 10 + - 11 + - 12 + rotation_mode: euler + action_frame: model_frame + overrides: + replan_every: + label: Actions before replanning + description: Control-rate execution prefix. + type: integer + min: 1 + max: 45 + step: 1 + default: 30 + target: + kind: policy_attribute + attribute_path: replan_every + inference_steps: + label: Sampling steps + description: Configured sampler iterations. + type: integer + min: 1 + max: 100 + step: 1 + default: 50 + target: + kind: stage_attribute + stage_id: sampler + attribute_path: num_inference_steps +inference: ${model.inference_domains.${model.deployment_domain}} +data_requirements: + preprocessing: + '6': + transforms: + action_mode: cartesian + coord_frame: camframe + rotation_mode: euler + source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/pi0.5_base.yaml b/egomimic/hydra_configs/model/base/pi05.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi0.5_base.yaml rename to egomimic/hydra_configs/model/base/pi05.yaml diff --git a/egomimic/hydra_configs/model/pi0.5_bc_aria.yaml b/egomimic/hydra_configs/model/base/pi_human_legacy.yaml similarity index 91% rename from egomimic/hydra_configs/model/pi0.5_bc_aria.yaml rename to egomimic/hydra_configs/model/base/pi_human_legacy.yaml index 8d560b11fe..df2e674ae0 100644 --- a/egomimic/hydra_configs/model/pi0.5_bc_aria.yaml +++ b/egomimic/hydra_configs/model/base/pi_human_legacy.yaml @@ -1,4 +1,4 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. +# Shared model definition; concrete recipes preserve their own source/timing settings. _target_: egomimic.pl_utils.pl_model.ModelWrapper enable_grad_norm: false optimizer: @@ -92,17 +92,12 @@ inference_domains: timing: kind: uniform_resampled_target temporally_resolved: true - dt: 0.00909090909090909 first_source_frame: 0 - last_source_frame: 27 source_fps: 30 output: frame: camframe key: actions representation: cartesian - shape: - - 28 - - 12 timing: kind: control_period temporally_resolved: true @@ -134,16 +129,11 @@ inference_domains: - 12 adapter: _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 28 - - 12 decoder: _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder native_shape: - 100 - 12 - native_dt: 0.00909090909090909 - output_horizon: 28 output_dt: 0.03333333333333333 angular_indices: - 3 @@ -158,9 +148,7 @@ inference_domains: description: Control-rate execution prefix. type: integer min: 1 - max: 28 step: 1 - default: 28 target: kind: policy_attribute attribute_path: replan_every @@ -182,8 +170,6 @@ diagnostic_provider: stage_id: sampler emit_combined: true mask_padding: false - -# Required preprocessing semantics, independent of episode selection or location. data_requirements: preprocessing: '3': @@ -191,5 +177,4 @@ data_requirements: action_mode: cartesian coord_frame: camframe rotation_mode: euler - stride: 3 source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/base/real_visual_settings.yaml b/egomimic/hydra_configs/model/base/real_visual_settings.yaml new file mode 100644 index 0000000000..7b67e20af5 --- /dev/null +++ b/egomimic/hydra_configs/model/base/real_visual_settings.yaml @@ -0,0 +1,17 @@ +# @package _global_ +hpt: + embed_dim: 840 + num_blocks: 19 + num_heads: 10 + action_dim: 14 + action_horizon: 100 + flow_blocks: 6 + flow_hidden_dim: 320 + stem_specs: + random_horizon_masking: false + cross_attn: + crossattn_latent: 16 + crossattn_heads: ${hpt.num_heads} + crossattn_dim_head: 84 + crossattn_modality_dropout: 0.1 + modality_embed_dim: ${hpt.embed_dim} diff --git a/egomimic/hydra_configs/model/base/real_yam_settings.yaml b/egomimic/hydra_configs/model/base/real_yam_settings.yaml new file mode 100644 index 0000000000..92a00aee15 --- /dev/null +++ b/egomimic/hydra_configs/model/base/real_yam_settings.yaml @@ -0,0 +1,15 @@ +# @package _global_ +hpt: + embed_dim: 840 + num_blocks: 19 + num_heads: 10 + stem_specs: + random_horizon_masking: false + cross_attn: + crossattn_latent: 16 + crossattn_heads: 10 + crossattn_dim_head: 84 + crossattn_modality_dropout: 0.1 + modality_embed_dim: 840 + action_dim: ${yam.action_dim} + action_horizon: 100 diff --git a/egomimic/hydra_configs/model/base/yam_diffusion_settings.yaml b/egomimic/hydra_configs/model/base/yam_diffusion_settings.yaml new file mode 100644 index 0000000000..5e1abe2067 --- /dev/null +++ b/egomimic/hydra_configs/model/base/yam_diffusion_settings.yaml @@ -0,0 +1,31 @@ +# @package _global_ +# Model hyperparameters only; task, dataset and trainer choices stay experiment-owned. +abc: + action_dim: 14 + action_horizon: 100 + condition_dim: 78 + encoder: + _target_: egomimic.pipeline.stages_sampler.DPStyleObsEncoder + obs_specs: + ee_pose: + input_dim: 14 + input_slice: + - 0 + - 14 + img_encoders: + front_img_1: + _target_: egomimic.models.stems.visual_core.VisualCore + in_channels: 3 + image_size: 480 + image_width: 640 + num_kp: 32 + feature_dimension: 64 + pretrained: false + crop_aug: true + crop_height: 432 + crop_width: 576 + crop_eval_mode: center + crop_sample_mode: v02 + crop_scope: frame + norm_layer: group + pool_type: spatial_softmax diff --git a/egomimic/hydra_configs/model/base/yam_hpt_settings.yaml b/egomimic/hydra_configs/model/base/yam_hpt_settings.yaml new file mode 100644 index 0000000000..60ddc52c84 --- /dev/null +++ b/egomimic/hydra_configs/model/base/yam_hpt_settings.yaml @@ -0,0 +1,16 @@ +# @package _global_ +# Model hyperparameters only; task, dataset and trainer choices stay experiment-owned. +hpt: + embed_dim: 256 + num_blocks: 16 + num_heads: 8 + action_horizon: 100 + action_dim: 14 + stem_specs: + random_horizon_masking: false + cross_attn: + crossattn_latent: 16 + crossattn_heads: 8 + crossattn_dim_head: 64 + crossattn_modality_dropout: 0.1 + modality_embed_dim: ${hpt.embed_dim} diff --git a/egomimic/hydra_configs/model/bf/bf_planar_v2_arc_graph_tok.yaml b/egomimic/hydra_configs/model/bf/bf_planar_v2_arc_graph_tok.yaml index f0acf78ad5..7efc650956 100644 --- a/egomimic/hydra_configs/model/bf/bf_planar_v2_arc_graph_tok.yaml +++ b/egomimic/hydra_configs/model/bf/bf_planar_v2_arc_graph_tok.yaml @@ -8,7 +8,7 @@ # in the inference graph, so the dependency resolver drops it from training on # its own rather than through a mode flag. # -# Contrast with experiment/sim/pusht/planar_v2_usocket_arc_bc, where the same +# Contrast with experiment/arc_sim/pusht/planar_v2_usocket_arc_bc, where the same # tokenization happens in the loader's transform_list: there the stage list # gives no sign the target is a token, and the linter cannot see the boundary. _target_: egomimic.pl_utils.pl_model.ModelWrapper diff --git a/egomimic/hydra_configs/model/hpt_bc_flow_eva.yaml b/egomimic/hydra_configs/model/hpt/bc_flow_eva.yaml similarity index 100% rename from egomimic/hydra_configs/model/hpt_bc_flow_eva.yaml rename to egomimic/hydra_configs/model/hpt/bc_flow_eva.yaml diff --git a/egomimic/hydra_configs/model/hpt/bc_flow_human.yaml b/egomimic/hydra_configs/model/hpt/bc_flow_human.yaml new file mode 100644 index 0000000000..6c07896e80 --- /dev/null +++ b/egomimic/hydra_configs/model/hpt/bc_flow_human.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_human_flow@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.00909090909090909 + last_source_frame: 27 + output: + shape: + - 28 + - 12 + profiles: + default: + adapter: + shape: + - 28 + - 12 + decoder: + native_dt: 0.00909090909090909 + output_horizon: 28 + overrides: + replan_every: + max: 28 + default: 28 +data_requirements: + preprocessing: + '3': + transforms: + stride: 3 diff --git a/egomimic/hydra_configs/model/hpt/bc_flow_mecka.yaml b/egomimic/hydra_configs/model/hpt/bc_flow_mecka.yaml new file mode 100644 index 0000000000..e3b4e0eaf2 --- /dev/null +++ b/egomimic/hydra_configs/model/hpt/bc_flow_mecka.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_human_flow@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.009764309764309764 + last_source_frame: 29 + output: + shape: + - 30 + - 12 + profiles: + default: + adapter: + shape: + - 30 + - 12 + decoder: + native_dt: 0.009764309764309764 + output_horizon: 30 + overrides: + replan_every: + max: 30 + default: 30 +data_requirements: + preprocessing: + '3': + transforms: + stride: 1 diff --git a/egomimic/hydra_configs/model/hpt_bc_flow_scale.yaml b/egomimic/hydra_configs/model/hpt/bc_flow_scale.yaml similarity index 100% rename from egomimic/hydra_configs/model/hpt_bc_flow_scale.yaml rename to egomimic/hydra_configs/model/hpt/bc_flow_scale.yaml diff --git a/egomimic/hydra_configs/model/hpt_bc_keypoints_wrist.yaml b/egomimic/hydra_configs/model/hpt/bc_keypoints_wrist.yaml similarity index 84% rename from egomimic/hydra_configs/model/hpt_bc_keypoints_wrist.yaml rename to egomimic/hydra_configs/model/hpt/bc_keypoints_wrist.yaml index 1afe46979d..3b96bd78e1 100644 --- a/egomimic/hydra_configs/model/hpt_bc_keypoints_wrist.yaml +++ b/egomimic/hydra_configs/model/hpt/bc_keypoints_wrist.yaml @@ -1,6 +1,6 @@ # The architecture is shared; target coordinates are an explicit model contract. defaults: - - hpt_bc_keypoints_base + - /model/base/hpt_keypoints@_here_ - _self_ data_requirements: diff --git a/egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pertoken.yaml b/egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pertoken.yaml similarity index 56% rename from egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pertoken.yaml rename to egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pertoken.yaml index 4ce29d79ba..957db38b4b 100644 --- a/egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pertoken.yaml +++ b/egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pertoken.yaml @@ -1,19 +1,8 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. -_target_: egomimic.pl_utils.pl_model.ModelWrapper -optimizer: - _target_: torch.optim.AdamW - _partial_: true - lr: 0.0002 - weight_decay: 0.0001 -scheduler: - _target_: egomimic.utils.schedulers.warmup_then_cosine - _partial_: true - warmup_epochs: 100 - total_epochs: 2000 - eta_min: 1.0e-05 -scheduler_interval: epoch +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_pickplace_qwen@_here_ +- _self_ pipeline: - _target_: egomimic.pipeline.algo.PipelineAlgo stages: - _target_: egomimic.pipeline.stages_hpt.AnnotationPromptStage annotation_key: annotations @@ -152,115 +141,3 @@ pipeline: mlp_layers: 4 mlp_ratio: 4 - _target_: egomimic.pipeline.stages_flow.FlowVelocityLossStage - stage_ids: - stems: 1 - trunk: 2 - sampler: 5 -deployment_domain: eva_bimanual -inference_domains: - eva_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.front_img_1 - - annotations - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.front_img_1 - constants: - embodiment: 6 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 14 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.014814814814814815 - first_source_frame: 0 - last_source_frame: 44 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 45 - - 14 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 14 - embodiment: 6 - tokenizer: null - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 14 - adapter: - _target_: egomimic.robot.graph_policy.CartesianGraphAdapter - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 14 - native_dt: 0.014814814814814815 - output_horizon: 45 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 10 - - 11 - - 12 - rotation_mode: euler - action_frame: model_frame - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 45 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 50 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: num_inference_steps -inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. -data_requirements: - preprocessing: - '6': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pooled.yaml b/egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pooled.yaml similarity index 56% rename from egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pooled.yaml rename to egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pooled.yaml index 30be0568de..da356687f1 100644 --- a/egomimic/hydra_configs/model/hpt_bc_pickplace_qwen_pooled.yaml +++ b/egomimic/hydra_configs/model/hpt/bc_pickplace_qwen_pooled.yaml @@ -1,19 +1,8 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. -_target_: egomimic.pl_utils.pl_model.ModelWrapper -optimizer: - _target_: torch.optim.AdamW - _partial_: true - lr: 0.0002 - weight_decay: 0.0001 -scheduler: - _target_: egomimic.utils.schedulers.warmup_then_cosine - _partial_: true - warmup_epochs: 100 - total_epochs: 2000 - eta_min: 1.0e-05 -scheduler_interval: epoch +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_pickplace_qwen@_here_ +- _self_ pipeline: - _target_: egomimic.pipeline.algo.PipelineAlgo stages: - _target_: egomimic.pipeline.stages_hpt.AnnotationPromptStage annotation_key: annotations @@ -152,115 +141,3 @@ pipeline: mlp_layers: 4 mlp_ratio: 4 - _target_: egomimic.pipeline.stages_flow.FlowVelocityLossStage - stage_ids: - stems: 1 - trunk: 2 - sampler: 5 -deployment_domain: eva_bimanual -inference_domains: - eva_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.front_img_1 - - annotations - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.front_img_1 - constants: - embodiment: 6 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 14 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.014814814814814815 - first_source_frame: 0 - last_source_frame: 44 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 45 - - 14 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 14 - embodiment: 6 - tokenizer: null - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 14 - adapter: - _target_: egomimic.robot.graph_policy.CartesianGraphAdapter - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 14 - native_dt: 0.014814814814814815 - output_horizon: 45 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 10 - - 11 - - 12 - rotation_mode: euler - action_frame: model_frame - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 45 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 50 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: num_inference_steps -inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. -data_requirements: - preprocessing: - '6': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/abc_arc/hpt_bimanual_visual.yaml b/egomimic/hydra_configs/model/hpt/bimanual_visual.yaml similarity index 100% rename from egomimic/hydra_configs/model/abc_arc/hpt_bimanual_visual.yaml rename to egomimic/hydra_configs/model/hpt/bimanual_visual.yaml diff --git a/egomimic/hydra_configs/model/hpt_cotrain_flow_seperate_head.yaml b/egomimic/hydra_configs/model/hpt/cotrain_flow_seperate_head.yaml similarity index 100% rename from egomimic/hydra_configs/model/hpt_cotrain_flow_seperate_head.yaml rename to egomimic/hydra_configs/model/hpt/cotrain_flow_seperate_head.yaml diff --git a/egomimic/hydra_configs/model/hpt/cotrain_flow_shared_head.yaml b/egomimic/hydra_configs/model/hpt/cotrain_flow_shared_head.yaml new file mode 100644 index 0000000000..3337e95b8d --- /dev/null +++ b/egomimic/hydra_configs/model/hpt/cotrain_flow_shared_head.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_cotrain_flow@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.00909090909090909 + last_source_frame: 27 + output: + shape: + - 28 + - 12 + profiles: + default: + adapter: + shape: + - 28 + - 12 + decoder: + native_dt: 0.00909090909090909 + output_horizon: 28 + overrides: + replan_every: + max: 28 + default: 28 +data_requirements: + preprocessing: + '3': + transforms: + stride: 3 diff --git a/egomimic/hydra_configs/model/hpt/cotrain_mecka_flow_shared_head.yaml b/egomimic/hydra_configs/model/hpt/cotrain_mecka_flow_shared_head.yaml new file mode 100644 index 0000000000..e0d1fe16ac --- /dev/null +++ b/egomimic/hydra_configs/model/hpt/cotrain_mecka_flow_shared_head.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/hpt_cotrain_flow@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.009764309764309764 + last_source_frame: 29 + output: + shape: + - 30 + - 12 + profiles: + default: + adapter: + shape: + - 30 + - 12 + decoder: + native_dt: 0.009764309764309764 + output_horizon: 30 + overrides: + replan_every: + max: 30 + default: 30 +data_requirements: + preprocessing: + '3': + transforms: + stride: 1 diff --git a/egomimic/hydra_configs/model/egobridge.yaml b/egomimic/hydra_configs/model/hpt/egobridge.yaml similarity index 100% rename from egomimic/hydra_configs/model/egobridge.yaml rename to egomimic/hydra_configs/model/hpt/egobridge.yaml diff --git a/egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt300_time.yaml b/egomimic/hydra_configs/model/hpt/yam_300m.yaml similarity index 60% rename from egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt300_time.yaml rename to egomimic/hydra_configs/model/hpt/yam_300m.yaml index 1ba32594af..fac79622d3 100644 --- a/egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt300_time.yaml +++ b/egomimic/hydra_configs/model/hpt/yam_300m.yaml @@ -1,6 +1,6 @@ -# HPT300 frozen YAM graph; 14D time-indexed targets. +# HPT300 YAM graph; 14D time-indexed targets. defaults: -- /model/abc_arc/frozen_yam_hpt_flow@_here_ +- /model/hpt/yam_flow@_here_ - _self_ action_token_dim: 14 embed_dim: 840 diff --git a/egomimic/hydra_configs/model/hpt/yam_300m_arc.yaml b/egomimic/hydra_configs/model/hpt/yam_300m_arc.yaml new file mode 100644 index 0000000000..48fbdf0bbe --- /dev/null +++ b/egomimic/hydra_configs/model/hpt/yam_300m_arc.yaml @@ -0,0 +1,48 @@ +defaults: +- /model/hpt/yam_300m@_here_ +- /model_contract/abc_visual@_here_ +- _self_ +action_token_dim: ${abc.arc_token_dim} +action_horizon: ${abc.arc_token_rows} +native_action_horizon: ${abc.arc_token_rows} +native_action_dim: ${abc.arc_token_dim} +inference: ${model.inference_contracts.hybrid_arc_tokenizer_cartesian} +data_requirements: + preprocessing: + '7': + key_map: + _target_: egomimic.rldb.embodiment.bimanual_arc.get_keymap + horizon: 200 + embodiment: yam + drop_wrist_images: false + yam_source_frames: ${abc.arc_source_buffer_frames} + transforms: + _target_: egomimic.rldb.embodiment.bimanual_arc.get_transform_list + variant: arc + chunk_length: 200 + time_rows: ${abc.arc_source_buffer_frames} + min_distance_unit: ${abc.arc_distance} + rotation_distance_unit: ${abc.arc_rotation_distance} + resampled_vector_length: ${abc.arc_waypoints} + stride: 1 + rotation_mode: euler + embodiment: yam + yam_source_frames: ${abc.arc_source_buffer_frames} + clock_mode: ${abc.arc_clock_mode} + streams: ${abc.arc_streams} + gripper_stream_mode: ${abc.arc_gripper_stream_mode} + source_fps: 30.0 +graph: + noising: + action_horizon: ${abc.arc_token_rows} + sampler: + action_horizon: ${abc.arc_token_rows} + model: + act_seq: ${abc.arc_token_rows} +inference_contracts: + hybrid_arc_tokenizer_cartesian: + profiles: + default: + adapter: + decoder: + execute_percent: 50 diff --git a/egomimic/hydra_configs/model/abc/yam_bimanual_hpt.yaml b/egomimic/hydra_configs/model/hpt/yam_bimanual.yaml similarity index 99% rename from egomimic/hydra_configs/model/abc/yam_bimanual_hpt.yaml rename to egomimic/hydra_configs/model/hpt/yam_bimanual.yaml index b21bbb7697..62444ac1fa 100644 --- a/egomimic/hydra_configs/model/abc/yam_bimanual_hpt.yaml +++ b/egomimic/hydra_configs/model/hpt/yam_bimanual.yaml @@ -16,6 +16,9 @@ # cross_attn modality_embed_dim, the trunk's embed_dim and the head's cond_dim # must move together -- the stem stage rejects a mismatch rather than letting a # silent broadcast through. +defaults: + - /model/base/yam_hpt_settings@_global_ + - _self_ _target_: egomimic.pl_utils.pl_model.ModelWrapper pipeline: diff --git a/egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt_flow.yaml b/egomimic/hydra_configs/model/hpt/yam_flow.yaml similarity index 99% rename from egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt_flow.yaml rename to egomimic/hydra_configs/model/hpt/yam_flow.yaml index 5ea881c9f3..a6463f6691 100644 --- a/egomimic/hydra_configs/model/abc_arc/frozen_yam_hpt_flow.yaml +++ b/egomimic/hydra_configs/model/hpt/yam_flow.yaml @@ -1,6 +1,6 @@ # Frozen FT dimensions; one shared graph for BC, cotrain and all token widths. defaults: -- /model_contract/frozen_yam_wrists@_here_ +- /model_contract/yam_wrists@_here_ - _self_ _target_: egomimic.pl_utils.pl_model.ModelWrapper pipeline: diff --git a/egomimic/hydra_configs/model/abc_arc/hpt_yam_visual.yaml b/egomimic/hydra_configs/model/hpt/yam_visual.yaml similarity index 100% rename from egomimic/hydra_configs/model/abc_arc/hpt_yam_visual.yaml rename to egomimic/hydra_configs/model/hpt/yam_visual.yaml diff --git a/egomimic/hydra_configs/model/hpt_bc_flow_aria.yaml b/egomimic/hydra_configs/model/hpt_bc_flow_aria.yaml deleted file mode 100644 index b6a83fd655..0000000000 --- a/egomimic/hydra_configs/model/hpt_bc_flow_aria.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# Compatibility name for saved commands/configs; architecture is shared. -# Canonical definition: hpt_bc_flow_human.yaml -defaults: - - hpt_bc_flow_human - - _self_ diff --git a/egomimic/hydra_configs/model/hpt_bc_flow_mecka.yaml b/egomimic/hydra_configs/model/hpt_bc_flow_mecka.yaml deleted file mode 100644 index 4204f873c1..0000000000 --- a/egomimic/hydra_configs/model/hpt_bc_flow_mecka.yaml +++ /dev/null @@ -1,247 +0,0 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. -_target_: egomimic.pl_utils.pl_model.ModelWrapper -optimizer: - _target_: torch.optim.AdamW - _partial_: true - lr: 0.0003 - weight_decay: 0.0001 -scheduler: - _target_: torch.optim.lr_scheduler.CosineAnnealingLR - _partial_: true - T_max: 1400 - eta_min: 1.0e-05 -pipeline: - _target_: egomimic.pipeline.algo.PipelineAlgo - stages: - - _target_: egomimic.pipeline.stages_hpt.HPTStemStage - domain_first: true - preserve_config_order: true - selector_aliases: - '3': human_bimanual - stems: - observations.images.front_img_1: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 256 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - encoder: - _target_: egomimic.models.stems.hpt_stems.ResNet - output_dim: 256 - train_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.ColorJitter - brightness: 0.1 - contrast: 0.1 - saturation: 0.1 - hue: 0.05 - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - eval_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - domain_stems: - human_bimanual: - observations.state.ee_pose: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 12 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - - _target_: egomimic.pipeline.stages_hpt.HPTTrunkStage - embed_dim: 256 - token_postprocessing: action_token - action_token_count: 64 - squeeze_action_token: false - use_domain_embedding: false - trunk: - _target_: egomimic.models.cores.hpt_transformer.SimpleTransformer - embed_dim: 256 - num_blocks: 16 - drop_path_rate: 0.1 - weight_init_style: pytorch - attn_target: - _target_: egomimic.models.cores.hpt_transformer.MultiheadAttention - _partial_: true - embed_dim: 256 - num_heads: 8 - batch_first: true - bias: true - add_bias_kv: true - - _target_: egomimic.pipeline.stages_io.ActionTargetBuilder - action_key: actions_cartesian - - _target_: egomimic.pipeline.stages_flow.FlowNoisingStage - action_horizon: 100 - action_dim: 12 - time_dist: beta - - _target_: egomimic.pipeline.stages_flow.FlowDenoiserStage - action_horizon: 100 - action_dim: 12 - condition_input_dim: 256 - condition_ndim: 3 - condition_as_tokens: false - num_inference_steps: 50 - model: - _target_: egomimic.models.denoising_nets.CrossTransformer - nblocks: 6 - cond_dim: 256 - hidden_dim: 128 - act_dim: 12 - act_seq: 100 - n_heads: 4 - dropout: 0.1 - mlp_layers: 4 - mlp_ratio: 4 - - _target_: egomimic.pipeline.stages_flow.FlowVelocityLossStage - stage_ids: - stems: 0 - trunk: 1 - sampler: 4 -deployment_domain: human_bimanual -inference_domains: - human_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.front_img_1 - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.front_img_1 - constants: - embodiment: 3 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 12 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.009764309764309764 - first_source_frame: 0 - last_source_frame: 29 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 30 - - 12 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 12 - embodiment: 3 - tokenizer: null - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 12 - adapter: - _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 30 - - 12 - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 12 - native_dt: 0.009764309764309764 - output_horizon: 30 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 9 - - 10 - - 11 - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 30 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 50 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: num_inference_steps -inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. -data_requirements: - preprocessing: - '3': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - stride: 1 - source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/hpt_cotrain_mecka_flow_shared_head.yaml b/egomimic/hydra_configs/model/hpt_cotrain_mecka_flow_shared_head.yaml deleted file mode 100644 index 1bf8b8174a..0000000000 --- a/egomimic/hydra_configs/model/hpt_cotrain_mecka_flow_shared_head.yaml +++ /dev/null @@ -1,542 +0,0 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. -_target_: egomimic.pl_utils.pl_model.ModelWrapper -optimizer: - _target_: torch.optim.AdamW - _partial_: true - lr: 0.0001 - weight_decay: 0.0001 -scheduler: - _target_: torch.optim.lr_scheduler.CosineAnnealingLR - _partial_: true - T_max: 1400 - eta_min: 1.0e-05 -pipeline: - _target_: egomimic.pipeline.algo.PipelineAlgo - stages: - - _target_: egomimic.pipeline.stages_hpt.HPTStemStage - domain_first: true - preserve_config_order: true - selector_aliases: - '6': eva_bimanual - '3': human_bimanual - stems: - observations.images.front_img_1: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 256 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - encoder: - _target_: egomimic.models.stems.hpt_stems.ResNet - output_dim: 256 - num_of_copy: 1 - train_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.ColorJitter - brightness: 0.1 - contrast: 0.1 - saturation: 0.1 - hue: 0.05 - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - eval_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - domain_stems: - human_bimanual: - observations.state.ee_pose: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 12 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - eva_bimanual: - observations.state.ee_pose: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 14 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - observations.images.right_wrist_img: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 256 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - encoder: - _target_: egomimic.models.stems.hpt_stems.ResNet - output_dim: 256 - num_of_copy: 1 - train_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.ColorJitter - brightness: 0.1 - contrast: 0.1 - saturation: 0.1 - hue: 0.05 - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - eval_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - observations.images.left_wrist_img: - _target_: egomimic.models.stems.composed.EncodedPolicyStem - stem: - _target_: egomimic.models.stems.hpt_stems.MLPPolicyStem - input_dim: 256 - output_dim: 256 - widths: - - 256 - specs: - random_horizon_masking: false - cross_attn: - crossattn_latent: 16 - crossattn_heads: 8 - crossattn_dim_head: 64 - crossattn_modality_dropout: 0.1 - modality_embed_dim: 256 - feature_position_encoding: true - encoder: - _target_: egomimic.models.stems.hpt_stems.ResNet - output_dim: 256 - num_of_copy: 1 - train_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.ColorJitter - brightness: 0.1 - contrast: 0.1 - saturation: 0.1 - hue: 0.05 - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - eval_transform: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Normalize - mean: - - 0.485 - - 0.456 - - 0.406 - std: - - 0.229 - - 0.224 - - 0.225 - - _target_: egomimic.pipeline.stages_hpt.HPTTrunkStage - embed_dim: 256 - token_postprocessing: action_token - action_token_count: 64 - squeeze_action_token: false - use_domain_embedding: false - trunk: - _target_: egomimic.models.cores.hpt_transformer.SimpleTransformer - embed_dim: 256 - num_blocks: 16 - drop_path_rate: 0.1 - weight_init_style: pytorch - attn_target: - _target_: egomimic.models.cores.hpt_transformer.MultiheadAttention - _partial_: true - embed_dim: 256 - num_heads: 8 - batch_first: true - bias: true - add_bias_kv: true - - _target_: egomimic.pipeline.stages_io.ActionTargetBuilder - action_key: actions_cartesian - - _target_: egomimic.pipeline.stages_layout.ActionLayoutStage - key: target - layouts: - '3': - - 0 - - 1 - - 2 - - 3 - - 4 - - 5 - - null - - 6 - - 7 - - 8 - - 9 - - 10 - - 11 - - null - '6': - - 0 - - 1 - - 2 - - 3 - - 4 - - 5 - - 6 - - 7 - - 8 - - 9 - - 10 - - 11 - - 12 - - 13 - input_widths: - '3': 12 - '6': 14 - - _target_: egomimic.pipeline.stages_flow.FlowNoisingStage - action_horizon: 100 - action_dim: 14 - time_dist: beta - - _target_: egomimic.pipeline.stages_flow.FlowDenoiserStage - action_horizon: 100 - action_dim: 14 - condition_input_dim: 256 - condition_ndim: 3 - condition_as_tokens: false - num_inference_steps: 50 - model: - _target_: egomimic.models.denoising_nets.CrossTransformer - nblocks: 6 - cond_dim: 256 - hidden_dim: 128 - act_dim: 14 - act_seq: 100 - n_heads: 4 - dropout: 0.1 - mlp_layers: 4 - mlp_ratio: 4 - - _target_: egomimic.pipeline.stages_flow.FlowVelocityLossStage - - _target_: egomimic.pipeline.stages_layout.ActionLayoutStage - key: pred_action - mode: inference - layouts: - '3': - - 0 - - 1 - - 2 - - 3 - - 4 - - 5 - - 7 - - 8 - - 9 - - 10 - - 11 - - 12 - '6': - - 0 - - 1 - - 2 - - 3 - - 4 - - 5 - - 6 - - 7 - - 8 - - 9 - - 10 - - 11 - - 12 - - 13 - input_widths: - '3': 14 - '6': 14 - stage_ids: - stems: 0 - trunk: 1 - sampler: 5 -deployment_domain: eva_bimanual -inference_domains: - eva_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.right_wrist_img - - observations.images.left_wrist_img - - observations.images.front_img_1 - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.right_wrist_img - - observations.images.left_wrist_img - - observations.images.front_img_1 - constants: - embodiment: 6 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 14 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.014814814814814815 - first_source_frame: 0 - last_source_frame: 44 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 45 - - 14 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 14 - embodiment: 6 - tokenizer: null - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 14 - adapter: - _target_: egomimic.robot.graph_policy.CartesianGraphAdapter - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 14 - native_dt: 0.014814814814814815 - output_horizon: 45 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 10 - - 11 - - 12 - rotation_mode: euler - action_frame: model_frame - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 45 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 50 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: num_inference_steps - human_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.front_img_1 - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.front_img_1 - constants: - embodiment: 3 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 12 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.009764309764309764 - first_source_frame: 0 - last_source_frame: 29 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 30 - - 12 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 12 - embodiment: 3 - tokenizer: null - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 12 - adapter: - _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 30 - - 12 - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 12 - native_dt: 0.009764309764309764 - output_horizon: 30 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 9 - - 10 - - 11 - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 30 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 50 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: num_inference_steps -inference: ${model.inference_domains.${model.deployment_domain}} - -# Required preprocessing semantics, independent of episode selection or location. -data_requirements: - preprocessing: - '6': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - source_fps: 30.0 - '3': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - stride: 1 - source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/hpt_cotrain_scale_flow_shared_head.yaml b/egomimic/hydra_configs/model/hpt_cotrain_scale_flow_shared_head.yaml deleted file mode 100644 index dd248ac70e..0000000000 --- a/egomimic/hydra_configs/model/hpt_cotrain_scale_flow_shared_head.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# Compatibility name for saved commands/configs; architecture is shared. -# Canonical definition: hpt_cotrain_mecka_flow_shared_head.yaml -defaults: - - hpt_cotrain_mecka_flow_shared_head - - _self_ diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_bc_abc_eva_6d.yaml b/egomimic/hydra_configs/model/pi/bc_abc_eva_6d.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_bc_abc_eva_6d.yaml rename to egomimic/hydra_configs/model/pi/bc_abc_eva_6d.yaml diff --git a/egomimic/hydra_configs/model/pi/bc_aria_legacy.yaml b/egomimic/hydra_configs/model/pi/bc_aria_legacy.yaml new file mode 100644 index 0000000000..45c5b3799c --- /dev/null +++ b/egomimic/hydra_configs/model/pi/bc_aria_legacy.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/pi_human_legacy@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.00909090909090909 + last_source_frame: 27 + output: + shape: + - 28 + - 12 + profiles: + default: + adapter: + shape: + - 28 + - 12 + decoder: + native_dt: 0.00909090909090909 + output_horizon: 28 + overrides: + replan_every: + max: 28 + default: 28 +data_requirements: + preprocessing: + '3': + transforms: + stride: 3 diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_bc_eva.yaml b/egomimic/hydra_configs/model/pi/bc_eva.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_bc_eva.yaml rename to egomimic/hydra_configs/model/pi/bc_eva.yaml diff --git a/egomimic/hydra_configs/model/pi0.5_bc_eva.yaml b/egomimic/hydra_configs/model/pi/bc_eva_legacy.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi0.5_bc_eva.yaml rename to egomimic/hydra_configs/model/pi/bc_eva_legacy.yaml diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_bc_mecka_6d.yaml b/egomimic/hydra_configs/model/pi/bc_mecka_6d.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_bc_mecka_6d.yaml rename to egomimic/hydra_configs/model/pi/bc_mecka_6d.yaml diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_bc_mecka_abc_6d.yaml b/egomimic/hydra_configs/model/pi/bc_mecka_abc_6d.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_bc_mecka_abc_6d.yaml rename to egomimic/hydra_configs/model/pi/bc_mecka_abc_6d.yaml diff --git a/egomimic/hydra_configs/model/pi/bc_mecka_legacy.yaml b/egomimic/hydra_configs/model/pi/bc_mecka_legacy.yaml new file mode 100644 index 0000000000..fefaf9761b --- /dev/null +++ b/egomimic/hydra_configs/model/pi/bc_mecka_legacy.yaml @@ -0,0 +1,32 @@ +# Concrete model recipe; shared graph/contract fields are inherited. +defaults: +- /model/base/pi_human_legacy@_here_ +- _self_ +inference_domains: + human_bimanual: + native_output: + timing: + dt: 0.009764309764309764 + last_source_frame: 29 + output: + shape: + - 30 + - 12 + profiles: + default: + adapter: + shape: + - 30 + - 12 + decoder: + native_dt: 0.009764309764309764 + output_horizon: 30 + overrides: + replan_every: + max: 30 + default: 30 +data_requirements: + preprocessing: + '3': + transforms: + stride: 1 diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_cotrain_eva_aria_6d.yaml b/egomimic/hydra_configs/model/pi/cotrain_eva_aria_6d.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_cotrain_eva_aria_6d.yaml rename to egomimic/hydra_configs/model/pi/cotrain_eva_aria_6d.yaml diff --git a/egomimic/hydra_configs/model/pi0.5_cotrain_eva_aria.yaml b/egomimic/hydra_configs/model/pi/cotrain_eva_aria_legacy.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi0.5_cotrain_eva_aria.yaml rename to egomimic/hydra_configs/model/pi/cotrain_eva_aria_legacy.yaml diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_ft_aria_fold_grip.yaml b/egomimic/hydra_configs/model/pi/ft_aria_fold_grip.yaml similarity index 100% rename from egomimic/hydra_configs/model/pi05/pi0.5_ft_aria_fold_grip.yaml rename to egomimic/hydra_configs/model/pi/ft_aria_fold_grip.yaml diff --git a/egomimic/hydra_configs/model/pi0.5_bc_mecka.yaml b/egomimic/hydra_configs/model/pi0.5_bc_mecka.yaml deleted file mode 100644 index 19d127e3de..0000000000 --- a/egomimic/hydra_configs/model/pi0.5_bc_mecka.yaml +++ /dev/null @@ -1,195 +0,0 @@ -# Graph port of EgoVerse ec5c903c; see docs/integration/RECIPE_PARITY.md. -_target_: egomimic.pl_utils.pl_model.ModelWrapper -enable_grad_norm: false -optimizer: - _target_: torch.optim.AdamW - _partial_: true - lr: 3.0e-05 - betas: - - 0.9 - - 0.999 - eps: 1.0e-08 - weight_decay: 0.0 -scheduler: - _target_: transformers.get_cosine_schedule_with_warmup - _partial_: true - num_warmup_steps: 2000 - num_training_steps: 60000 - num_cycles: 0.5 -scheduler_interval: step -pipeline: - _target_: egomimic.pipeline.algo.PipelineAlgo - stage_ids: - sampler: 0 - stages: - - _target_: egomimic.pipeline.stages_pi05.PI05Stage - _recursive_: false - policy: - _target_: egomimic.models.pi05.policy.PI - config: - pytorch_training_precision: bfloat16 - pytorch_weight_path: ${oc.env:EGOVERSE_PI05_WEIGHTS,/storage/home/hcoda1/4/paphiwetsa3/r-dxu345-0/projects/EgoVerse/egomimic/algo/pi_checkpoints/pi05_base_pytorch} - model: - pi05: true - action_dim: 32 - action_horizon: 100 - max_token_len: 180 - tokenizer_model_name: google/paligemma-3b-mix-224 - tokenizer_max_length: 128 - sampling_mode: random - annotation_key: annotations - default_prompt: '' - proprio_in_prompt: true - embodiment_label: true - state_num_bins: 256 - control_mode: null - train_image_augs: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Resize - size: 224 - interpolation: 3 - eval_image_augs: - _target_: torchvision.transforms.Compose - transforms: - - _target_: torchvision.transforms.Resize - size: 224 - interpolation: 3 - camera_transforms: {} - ac_keys: - human_bimanual: actions_cartesian - domains: - - human_bimanual - action_converters: - rules: - HUMAN_BIMANUAL: - _target_: egomimic.utils.action_encoding.HumanBimanualCartesianEuler - fallback: - _target_: egomimic.utils.action_encoding.BaseActionConverter - action_encoding: legacy_normalized_ypr_rot6d -deployment_domain: human_bimanual -inference_domains: - human_bimanual: - input: - history_length: 1 - keys: - - observations.state.ee_pose - - observations.images.front_img_1 - - annotations - - embodiment - history_keys: - - observations.state.ee_pose - - observations.images.front_img_1 - constants: - embodiment: 3 - native_output: - frame: camframe - key: pred_action - representation: normalized_action - shape: - - 100 - - 12 - timing: - kind: uniform_resampled_target - temporally_resolved: true - dt: 0.009764309764309764 - first_source_frame: 0 - last_source_frame: 29 - source_fps: 30 - output: - frame: camframe - key: actions - representation: cartesian - shape: - - 30 - - 12 - timing: - kind: control_period - temporally_resolved: true - dt: 0.03333333333333333 - compatibility: - normalizer_schema: - action_key: actions_cartesian - native_shape: - - 100 - - 12 - embodiment: 3 - tokenizer: null - action_encoding: legacy_normalized_ypr_rot6d - backend_native_output: - shape: - - 100 - - 32 - conversion: - rules: - HUMAN_BIMANUAL: - _target_: egomimic.utils.action_encoding.HumanBimanualCartesianEuler - fallback: - _target_: egomimic.utils.action_encoding.BaseActionConverter - profiles: - default: - stage_id: sampler - native_shape: - - 100 - - 12 - adapter: - _target_: egomimic.pipeline.action_adapter.CanonicalSequenceAdapter - shape: - - 30 - - 12 - decoder: - _target_: egomimic.pipeline.sequence_decoder.UniformTimeGridDecoder - native_shape: - - 100 - - 12 - native_dt: 0.009764309764309764 - output_horizon: 30 - output_dt: 0.03333333333333333 - angular_indices: - - 3 - - 4 - - 5 - - 9 - - 10 - - 11 - overrides: - replan_every: - label: Actions before replanning - description: Control-rate execution prefix. - type: integer - min: 1 - max: 30 - step: 1 - default: 30 - target: - kind: policy_attribute - attribute_path: replan_every - inference_steps: - label: Sampling steps - description: Configured sampler iterations. - type: integer - min: 1 - max: 100 - step: 1 - default: 10 - target: - kind: stage_attribute - stage_id: sampler - attribute_path: backend.num_steps -inference: ${model.inference_domains.${model.deployment_domain}} -diagnostic_provider: - _target_: egomimic.models.pi05.diagnostics.PI05AttentionProvider - stage_id: sampler - emit_combined: true - mask_padding: false - -# Required preprocessing semantics, independent of episode selection or location. -data_requirements: - preprocessing: - '3': - transforms: - action_mode: cartesian - coord_frame: camframe - rotation_mode: euler - stride: 1 - source_fps: 30.0 diff --git a/egomimic/hydra_configs/model/pi0.5_bc_scale.yaml b/egomimic/hydra_configs/model/pi0.5_bc_scale.yaml deleted file mode 100644 index 7985ab1df2..0000000000 --- a/egomimic/hydra_configs/model/pi0.5_bc_scale.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# Compatibility name for saved commands/configs; architecture is shared. -# Canonical definition: pi0.5_bc_mecka.yaml -defaults: - - pi0.5_bc_mecka - - _self_ diff --git a/egomimic/hydra_configs/model/pi0.5_cotrain_mecka_scale.yaml b/egomimic/hydra_configs/model/pi0.5_cotrain_mecka_scale.yaml deleted file mode 100644 index 7985ab1df2..0000000000 --- a/egomimic/hydra_configs/model/pi0.5_cotrain_mecka_scale.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# Compatibility name for saved commands/configs; architecture is shared. -# Canonical definition: pi0.5_bc_mecka.yaml -defaults: - - pi0.5_bc_mecka - - _self_ diff --git a/egomimic/hydra_configs/model/pi05/pi0.5_ft_abc_eva_6d.yaml b/egomimic/hydra_configs/model/pi05/pi0.5_ft_abc_eva_6d.yaml deleted file mode 100644 index 0337fc4e2a..0000000000 --- a/egomimic/hydra_configs/model/pi05/pi0.5_ft_abc_eva_6d.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# Compatibility name for saved commands/configs; architecture is shared. -# Canonical definition: pi05/pi0.5_bc_abc_eva_6d.yaml -defaults: - - /model/pi05/pi0.5_bc_abc_eva_6d@_here_ - - _self_ diff --git a/egomimic/hydra_configs/experiment/_shared/clock_settings.yaml b/egomimic/hydra_configs/model_contract/arc_clock_settings.yaml similarity index 85% rename from egomimic/hydra_configs/experiment/_shared/clock_settings.yaml rename to egomimic/hydra_configs/model_contract/arc_clock_settings.yaml index 1c80fbda1b..4c9af2d92f 100644 --- a/egomimic/hydra_configs/experiment/_shared/clock_settings.yaml +++ b/egomimic/hydra_configs/model_contract/arc_clock_settings.yaml @@ -1,5 +1,4 @@ # @package _global_ -# Shared Cartesian clock contract; task recipes own D, R, M and the data pool. abc: arc_clock_mode: velocity arc_streams: full_separation @@ -8,7 +7,6 @@ abc: translation_rotation: 18 translation: 16 projection: 16 - # Duration rows are absolute seconds from the anchor; velocity rows are incoming rates. arc_timing_layouts: velocity: first_delay_incoming_v1 duration: first_delay_absolute_v1 diff --git a/egomimic/hydra_configs/model_contract/frozen_yam_wrists.yaml b/egomimic/hydra_configs/model_contract/yam_wrists.yaml similarity index 86% rename from egomimic/hydra_configs/model_contract/frozen_yam_wrists.yaml rename to egomimic/hydra_configs/model_contract/yam_wrists.yaml index b8f5192922..eb3bdcfecc 100644 --- a/egomimic/hydra_configs/model_contract/frozen_yam_wrists.yaml +++ b/egomimic/hydra_configs/model_contract/yam_wrists.yaml @@ -2,7 +2,7 @@ inference_contracts: time: input: constants: - embodiment: ${model.source_identities.${frozen_yam.policy_domain}} + embodiment: ${model.source_identities.${yam.policy_domain}} history_length: 1 keys: - observations.images.front_img_1 @@ -44,7 +44,7 @@ inference_contracts: - ${model.action_horizon} - ${model.action_token_dim} tokenizer: null - source_frames: ${model.contract_source_frames.${frozen_yam.policy_domain}} + source_frames: ${model.contract_source_frames.${yam.policy_domain}} profiles: default: stage_id: sampler @@ -132,7 +132,7 @@ arc_decoder_controls: target: kind: decoder_attribute attribute_path: hold_speed_percent -inference: ${model.inference_contracts.${frozen_yam.variant}} +inference: ${model.inference_contracts.${yam.variant}} source_identities: human: 3 yam: 7 @@ -142,17 +142,17 @@ data_contracts: '3': key_map: _target_: egomimic.rldb.embodiment.bimanual_arc.get_keymap - horizon: ${frozen_yam.chunk_length} + horizon: ${yam.chunk_length} embodiment: human drop_wrist_images: false transforms: _target_: egomimic.rldb.embodiment.bimanual_arc.get_transform_list - variant: ${frozen_yam.variant} + variant: ${yam.variant} embodiment: human - chunk_length: ${frozen_yam.chunk_length} - time_rows: ${frozen_yam.time_rows} - min_distance_unit: ${frozen_yam.D} - resampled_vector_length: ${frozen_yam.M} + chunk_length: ${yam.chunk_length} + time_rows: ${yam.time_rows} + min_distance_unit: ${yam.D} + resampled_vector_length: ${yam.M} stride: 1 rotation_mode: euler source_fps: 30.0 @@ -166,19 +166,19 @@ data_contracts: yam_source_frames: ${model.contract_source_frames.yam} transforms: _target_: egomimic.rldb.embodiment.bimanual_arc.get_transform_list - variant: ${frozen_yam.variant} - time_rows: ${frozen_yam.time_rows} - min_distance_unit: ${frozen_yam.D} - resampled_vector_length: ${frozen_yam.M} + variant: ${yam.variant} + time_rows: ${yam.time_rows} + min_distance_unit: ${yam.D} + resampled_vector_length: ${yam.M} stride: 1 rotation_mode: euler embodiment: yam yam_source_frames: ${model.contract_source_frames.yam} source_fps: 30.0 -data_requirements: ${model.data_contracts.${frozen_yam.policy_domain}} +data_requirements: ${model.data_contracts.${yam.policy_domain}} contract_source_frames: - yam: ${oc.select:frozen_yam.yam_source_frames,100} - human: ${frozen_yam.chunk_length} + yam: ${oc.select:yam.yam_source_frames,100} + human: ${yam.chunk_length} direct_time_data_requirements: preprocessing: '7': diff --git a/egomimic/hydra_configs/train_zarr_cartesian_pi.yaml b/egomimic/hydra_configs/train_zarr_cartesian_pi.yaml index 766fbf9c1b..970761132e 100644 --- a/egomimic/hydra_configs/train_zarr_cartesian_pi.yaml +++ b/egomimic/hydra_configs/train_zarr_cartesian_pi.yaml @@ -1,9 +1,9 @@ defaults: - train_zarr_cartesian -- override model: pi05/pi0.5_bc_abc_eva_6d +- override model: pi/bc_abc_eva_6d - override data: pi05/abc_stationery_pi_6d - override trainer: ddp_pi -- override evaluator: pi05/eval_pi_wristframe_6d +- override evaluator: eval_cartesian_wrist_6d_prompt - _self_ pi05: pretrained_weights: ??? diff --git a/egomimic/hydra_configs/train_zarr_keypoint_wrist.yaml b/egomimic/hydra_configs/train_zarr_keypoint_wrist.yaml index 07361f00b3..23adc5cfb3 100644 --- a/egomimic/hydra_configs/train_zarr_keypoint_wrist.yaml +++ b/egomimic/hydra_configs/train_zarr_keypoint_wrist.yaml @@ -1,6 +1,6 @@ defaults: - train_zarr_cartesian - - override model: hpt_bc_keypoints_wrist + - override model: hpt/bc_keypoints_wrist - override data: aria_keypoints_wrist - override evaluator: eval_keypoints_wrist - _self_ diff --git a/egomimic/hydra_configs/train_zarr_keypoints.yaml b/egomimic/hydra_configs/train_zarr_keypoints.yaml index 857c8c3b7f..faede4fbfe 100644 --- a/egomimic/hydra_configs/train_zarr_keypoints.yaml +++ b/egomimic/hydra_configs/train_zarr_keypoints.yaml @@ -1,6 +1,6 @@ defaults: - train_zarr_cartesian - - override model: hpt_bc_keypoints_base + - override model: base/hpt_keypoints - override data: aria_keypoints - override evaluator: eval_keypoints - _self_ diff --git a/egomimic/hydra_configs/trainer/real_visual.yaml b/egomimic/hydra_configs/trainer/real_visual.yaml new file mode 100644 index 0000000000..748dfc8aee --- /dev/null +++ b/egomimic/hydra_configs/trainer/real_visual.yaml @@ -0,0 +1,13 @@ +defaults: +- /trainer/ddp@_here_ +- _self_ +max_steps: 240000 +max_epochs: -1 +min_epochs: null +val_check_interval: 10000 +check_val_every_n_epoch: null +strategy: ddp_find_unused_parameters_true +precision: 32-true +num_sanity_val_steps: 0 +accumulate_grad_batches: 1 +log_every_n_steps: 10 diff --git a/egomimic/hydra_configs/trainer/real_yam.yaml b/egomimic/hydra_configs/trainer/real_yam.yaml new file mode 100644 index 0000000000..923cdf080a --- /dev/null +++ b/egomimic/hydra_configs/trainer/real_yam.yaml @@ -0,0 +1,14 @@ +defaults: +- /trainer/ddp@_here_ +- _self_ +strategy: auto +sync_batchnorm: false +check_val_every_n_epoch: 200 +num_sanity_val_steps: 2 +limit_val_batches: 1.0 +default_root_dir: null +max_epochs: -1 +min_epochs: 0 +devices: 1 +num_nodes: 1 +max_steps: ${ft.max_steps} diff --git a/scripts/audit_components.py b/scripts/audit_components.py index 9038d3014f..e4499e7276 100644 --- a/scripts/audit_components.py +++ b/scripts/audit_components.py @@ -29,7 +29,7 @@ # This is a shipped, explicitly non-runnable fragment, not ignored migration debt. FRAGMENTS = { - "model/pi0.5_base.yaml": "Select a concrete domains/ac_keys recipe; owner: graph integration" + "model/base/pi05.yaml": "Select a concrete domains/ac_keys recipe; owner: graph integration" } QWEN_METADATA = ROOT / "tests/fixtures/qwen3_embedding_06b_config.json" QWEN_METADATA_SHA256 = ( diff --git a/scripts/audit_hydra_configs.py b/scripts/audit_hydra_configs.py index 69ac68a58f..a81dc5e232 100644 --- a/scripts/audit_hydra_configs.py +++ b/scripts/audit_hydra_configs.py @@ -19,46 +19,53 @@ CONFIGS = ROOT / "egomimic/hydra_configs" EXPERIMENT_CONTEXTS = { - "experiment/_shared/clock_settings": "real/stationary_velocity", - "experiment/_shared/frozen_yam_arc": "real/earbuds_velocity", - "experiment/_shared/arc_single_clock": "real/earbuds_duration", - "experiment/_shared/frozen_yam/hpt300_openloop": "real/earbuds_time", - "experiment/_shared/frozen_yam/base": "real/earbuds_time", - "evaluator/frozen_yam_time": "real/earbuds_time", - "model/abc_arc/frozen_yam_hpt_flow": "real/earbuds_time", - "model/abc_arc/frozen_yam_hpt300_time": "real/earbuds_time", - "model_contract/frozen_yam_wrists": "real/earbuds_time", - "data/pusht/planar_chain_manual4919_standard_retimed": "sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", - "data/pusht/parents/standard_dp_uc_h16": "sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", - "model/bf/libero10_action_flow_latent_fm_sg_unite_h240_euler50_dithalf": "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", - "model/libero/oat_dp_matched": "sim/libero_historical/dp_libero10_oat_dp_matched_s42", - "model/abc/yam_bimanual_dp": "abc/yam_fstshirt_dp", - "model/abc/yam_bimanual_hpt": "abc/yam_fstshirt_hpt", + "model_contract/arc_clock_settings": "arc_real/stationary_velocity", + "model/hpt/yam_300m_arc": "arc_real/earbuds_velocity", + "trainer/real_yam": "arc_real/earbuds_time", + "trainer/real_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", + "model/hpt/yam_flow": "arc_real/earbuds_time", + "model/hpt/yam_300m": "arc_real/earbuds_time", + "model_contract/yam_wrists": "arc_real/earbuds_time", "model_contract/abc_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", - "model/abc_arc/hpt_yam_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", - "model/abc_arc/hpt_bimanual_visual": "abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop", + "model/hpt/yam_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", "model/abc_arc/abc_dit_yam_visual": "abc_arc/robot_bc/multitask/variants/abc_multitask4_abcdit_baseline_visual_openloop", "data/abc_arc/abc_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", - "data/abc_arc/mecka_fold_clothes_40h_human_baseline": "abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_baseline_openloop", - "data/abc_arc/mecka_fold_clothes_40h_human_hybrid_D40_M100_R24deg": "abc_arc/human_bc/mecka_fold_clothes_40h_human_visual_hybrid_openloop", - "experiment/_shared/visual_hpt300": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", - "model/bf/bf_planar_v2_arc_graph_tok": "sim/pusht/planar_v2_usocket_arc_graph_tok", - "model/bf/us_action_flow_latent_fm_sg_unite_h384": "sim/pusht/parents/action_flow_usocket_h384", - "data/libero/action_flow_libero10_val01_h16": "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", - "data/libero/oat": "sim/oat/libero_oattok", - "model/oat/tokenizer": "sim/oat/libero_oattok", - "model/oat/policy": "sim/oat/libero_oatpolicy", - "model/oat/arc_tokenizer": "sim/oat/libero_arc_oattok", - "model/oat/arc_policy": "sim/oat/libero_arc_policy", - "model/oat/arc_oat_policy": "sim/oat/libero_arc_oatpolicy", - "model/oat/arc_oat_dp_policy": "sim/oat/libero_arc_oat_dp_policy", + "data/pusht/planar_chain_manual4919_standard_retimed": "arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", + "data/pusht/parents/standard_dp_uc_h16": "arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked", + "model/bf/libero10_action_flow_latent_fm_sg_unite_h240_euler50_dithalf": "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", + "model/libero/oat_dp_matched": "arc_sim/libero_historical/dp_libero10_oat_dp_matched_s42", + "model/bf/bf_planar_v2_arc_graph_tok": "arc_sim/pusht/planar_v2_usocket_arc_graph_tok", + "model/bf/us_action_flow_latent_fm_sg_unite_h384": "arc_sim/pusht/parents/action_flow_usocket_h384", + "data/libero/action_flow_libero10_val01_h16": "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", + "data/libero/oat": "arc_sim/oat/libero_oattok", + "model/oat/tokenizer": "arc_sim/oat/libero_oattok", + "model/oat/policy": "arc_sim/oat/libero_oatpolicy", + "model/oat/arc_tokenizer": "arc_sim/oat/libero_arc_oattok", + "model/oat/arc_policy": "arc_sim/oat/libero_arc_policy", + "model/oat/arc_oat_policy": "arc_sim/oat/libero_arc_oatpolicy", + "model/oat/arc_oat_dp_policy": "arc_sim/oat/libero_arc_oat_dp_policy", + "data/abc_arc/stationary": "arc_real/stationary_time", + "data/abc_arc/stationary_arc": "arc_real/stationary_duration", + "data/abc_arc/earbuds": "arc_real/earbuds_time", + "data/abc_arc/earbuds_arc": "arc_real/earbuds_duration", + "data/abc_arc/towels": "arc_real/towels_time", + "data/abc_arc/towels_arc": "arc_real/towels_duration", + "data/abc_arc/stationery_yam_loader": "arc_real/stationary_time", } + +EXPERIMENT_CONTEXTS.update( + { + "model/hpt/bimanual_visual": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", + "data/abc_arc/mecka_fold_clothes_40h_human_baseline": "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", + "data/abc_arc/mecka_fold_clothes_40h_human_hybrid_D40_M100_R24deg": "abc_arc/robot_bc/abc_towels_hpt180_hybrid_visual_openloop", + } +) MODEL_CONTEXTS = { - "experiment/sim/pusht/action_flow_usocket_candidate_common": "bf/us_action_flow_latent_fm_sg_unite_h384", - "experiment/sim/pusht/action_latent_vfm_usocket_val01_h16": "bf/us_action_latent_vfm_nt16_d8_h512_s42", - "experiment/sim/pusht/unite_cotrain_usocket_chain_val01_h16": "bf/ct_unite_register_separate_nt8_h384_s42", - "experiment/sim/pusht/unite_usocket_register_sweep_val01_h16": "bf/us_unite_register_shared_nt4_s42", - "experiment/sim/oat/libero_base": "oat/tokenizer", + "experiment/arc_sim/pusht/action_flow_usocket_candidate_common": "bf/us_action_flow_latent_fm_sg_unite_h384", + "experiment/arc_sim/pusht/action_latent_vfm_usocket_val01_h16": "bf/us_action_latent_vfm_nt16_d8_h512_s42", + "experiment/arc_sim/pusht/unite_cotrain_usocket_chain_val01_h16": "bf/ct_unite_register_separate_nt8_h384_s42", + "experiment/arc_sim/pusht/unite_usocket_register_sweep_val01_h16": "bf/us_unite_register_shared_nt4_s42", + "experiment/arc_sim/oat/libero_base": "oat/tokenizer", } # Explicit non-production composition inputs, scoped and restored even on error. @@ -129,10 +136,47 @@ def audit_context(path): # This is a dataset fragment, not a selectable DataModule. Audit it # in the same nested package used by its real data parents. return "train_zarr_cartesian", [ - "+experiment=sim/pusht/planar_v2_usocket_dp_standard", + "+experiment=arc_sim/pusht/planar_v2_usocket_dp_standard", "+data@data.train_datasets.pushshapes_sim_u_socket=pusht/parents/episode_split", ] group, _, entry = name.partition("/") + component_contexts = { + # Chunk metrics must not inherit the public baseline's open-loop kwargs. + "evaluator/eval_cartesian_chunk_metrics": [ + "evaluator=eval_cartesian_chunk_metrics", + "++yam.limit_val_batches=40", + ], + "model/base/real_visual_settings": [ + "+experiment=abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop" + ], + "model/base/real_yam_settings": ["+experiment=arc_real/earbuds_time"], + "data/_base/yam_arc": ["+experiment=arc_real/earbuds_velocity"], + "model/base/yam_diffusion_settings": [ + "model=abc/yam_bimanual_dp", + "data=abc_arc/abc_fstshirt_bc", + ], + "model/base/yam_hpt_settings": [ + "model=hpt/yam_bimanual", + "data=abc_arc/abc_fstshirt_bc", + ], + "model/hpt/yam_bimanual": [ + "model=hpt/yam_bimanual", + "data=abc_arc/abc_fstshirt_bc", + ], + "model/base/hpt_human_flow": ["model=hpt/bc_flow_human"], + "model/base/hpt_cotrain_flow": ["model=hpt/cotrain_flow_shared_head"], + "model/base/hpt_pickplace_qwen": ["model=hpt/bc_pickplace_qwen_pooled"], + "model/base/pi_human_legacy": ["model=pi/bc_mecka_legacy"], + "evaluator/_base/yam_human_cartesian": ["evaluator=eval_bimanual_cartesian"], + "evaluator/_base/cartesian_prompt": ["evaluator=eval_cartesian_wrist_prompt"], + } + if name in component_contexts: + root = ( + "train_zarr_cartesian_pi" + if name == "evaluator/_base/cartesian_prompt" + else "train_zarr_cartesian" + ) + return root, component_contexts[name] if group == "visualization": # Standalone data-visualization roots composed from their own # directory (egomimic/scripts/data_visualization), not under a run. @@ -141,22 +185,51 @@ def audit_context(path): if group == "experiment": experiment = entry elif group in {"model", "data"} and entry.startswith(("bf/", "pusht/")): - experiment = "sim/pusht/planar_v2_usocket_dp_standard" - elif group in {"model", "data"} and entry.startswith(("abc/", "abc_arc/")): - # The retired language-conditioned ABC ablations are gone; the kept - # 20-episode fstshirt baseline is the remaining shared ABC context. - experiment = "abc/yam_fstshirt_dp" + experiment = "arc_sim/pusht/planar_v2_usocket_dp_standard" # Stationery fragments require the same FT sizes, source frames and fixed # ranges supplied by their real inheriting recipes, never invented values. - # Frozen rl2 YAM pools (and their loader) take their FT sizes, source frames - # and codec from the task's time recipe, never invented values. - for pool in ("stationary", "earbuds", "towels"): - if name in (f"data/abc_arc/{pool}", "data/abc_arc/stationery_yam_loader"): - experiment = f"real/{pool}_time" + if name.startswith("data/abc_arc/stationery_"): + experiment = "arc_real/earbuds_time" + if name.startswith(("data/abc_arc/towels_", "data/abc_arc/earbuds_")): + # YAM pools select their codec through the HPT300 recipe. + experiment = "arc_real/towels_time" experiment = EXPERIMENT_CONTEXTS.get(name, experiment) overrides = [] if experiment is None else [f"+experiment={experiment}"] + if name == "model/hpt/bimanual_visual": + overrides.append("++abc.population=human") + if name.startswith("data/abc_arc/mecka_fold_clothes_40h_human_"): + overrides.extend( + [ + "model=hpt/bimanual_visual", + "++abc.population=human", + "abc.arc_distance=0.4", + "abc.arc_chunking_mode=multistream", + "abc.task_predicate=\"row['task'] == 'fold_clothes'\"", + ] + ) + if "hybrid" in name: + overrides.append("++abc.arc_rotation_distance=0.4188790204786391") + if ( + experiment is None + and group in {"model", "data"} + and entry.startswith(("abc/", "abc_arc/")) + ): + # Retained reusable components no longer depend on deleted public + # experiment menus. Model-owned dimensions come from its base config. + overrides.extend(["model=abc/yam_bimanual_dp", "data=abc_arc/abc_fstshirt_bc"]) config_name = name if not entry else "train_zarr_cartesian" - if group in {"model", "evaluator"} and entry.startswith("pi05/"): + if ( + group == "model" and entry.startswith("pi/") and not entry.endswith("_legacy") + ) or ( + group == "evaluator" + and entry + in { + "eval_cartesian_wrist_prompt", + "eval_cartesian_camera_6d_prompt", + "eval_cartesian_wrist_6d_prompt", + "eval_cartesian_wrist_6d_gripper_prompt", + } + ): config_name = "train_zarr_cartesian_pi" if name == "evaluator/dataset_video": config_name = "viz_language" @@ -212,7 +285,7 @@ def _compose_for_audit(path): cfg.hydra.job.id = "config-audit" if "pi05" in cfg and OmegaConf.is_missing(cfg.pi05, "pretrained_weights"): cfg.pi05.pretrained_weights = "/tmp/egoverse-config-audit/pi-weights" - if "frozen_yam" in cfg: + if "yam" in cfg: for key, value in { "variant": "time", "spread": "smoke", @@ -220,8 +293,8 @@ def _compose_for_audit(path): "valid_root": "/tmp/audit/valid", "data_root": "/tmp/audit/yam", }.items(): - if OmegaConf.is_missing(cfg.frozen_yam, key): - cfg.frozen_yam[key] = value + if OmegaConf.is_missing(cfg.yam, key): + cfg.yam[key] = value for section, values in { "benchmark": {"dataset": "/tmp/audit/libero", "arc_basis": "chebyshev"}, "speed_diagnostic": {"speed_reference": 0.1}, @@ -233,7 +306,7 @@ def _compose_for_audit(path): cfg[section][key] = value if ( path.relative_to(CONFIGS).as_posix() - == "experiment/sim/oat/libero_arc_shared_r384_d1p6_m36.yaml" + == "experiment/arc_sim/oat/libero_arc_shared_r384_d1p6_m36.yaml" ): # This reusable shortlist explicitly requires stk/dur selection. # Supplying an audit context does not make null a runtime default. diff --git a/scripts/data/precompute_norm_stats.py b/scripts/data/precompute_norm_stats.py index 26b3289afa..0a41c1a14e 100644 --- a/scripts/data/precompute_norm_stats.py +++ b/scripts/data/precompute_norm_stats.py @@ -14,7 +14,7 @@ Usage (CPU node, repo root, emimic venv): python scripts/pi05/precompute_norm_stats.py \ - --data pi05/mecka_all_pi_6d --model pi05/pi0.5_bc_mecka_6d \ + --data pi05/mecka_all_pi_6d --model pi/bc_mecka_6d \ --sample-frac 0.1 --num-workers 30 \ --out /storage/project/r-dxu345-0/agao81/norm_stats/mecka_all_6d """ diff --git a/scripts/ice/README.md b/scripts/ice/README.md index 8c72c15db5..c2af13ffec 100644 --- a/scripts/ice/README.md +++ b/scripts/ice/README.md @@ -36,10 +36,10 @@ Export these before `sbatch --export=ALL`: `3683e3461596eef8df2432fa865779b3c77b2a2057dabd0fea125595729cf313`. - `ICE_DATASET_DIR` and a new, absolute `ICE_OUTPUT_DIR` - `ICE_EXPERIMENT`, exactly one of - `sim/pusht/planar_v2_usocket_direct_bc`, - `sim/pusht/planar_v2_usocket_arc_bc`, or - `sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc`, or - `sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg` + `arc_sim/pusht/planar_v2_usocket_direct_bc`, + `arc_sim/pusht/planar_v2_usocket_arc_bc`, or + `arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc`, or + `arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg` - `ICE_WANDB_ENTITY`, `ICE_WANDB_PROJECT`, and a unique `ICE_WANDB_RUN_ID` - one GPU selected by the launcher's `H100|H200` Slurm constraint diff --git a/scripts/ice/launch_planar_bc.sbatch b/scripts/ice/launch_planar_bc.sbatch index 2e61efb035..01b6561f9f 100755 --- a/scripts/ice/launch_planar_bc.sbatch +++ b/scripts/ice/launch_planar_bc.sbatch @@ -105,9 +105,9 @@ case "$ICE_LAUNCH_MODE" in *) die "ICE_LAUNCH_MODE must be preflight, dry-run, or run" ;; esac case "$ICE_EXPERIMENT" in - pusht/planar_v2_usocket_direct_bc|pusht/planar_v2_usocket_arc_bc|\ - pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc|\ - pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg) ;; + arc_sim/pusht/planar_v2_usocket_direct_bc|arc_sim/pusht/planar_v2_usocket_arc_bc|\ + arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc|\ + arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg) ;; *) die "ICE_EXPERIMENT must select a supported U-Socket BC config" ;; esac case "$ICE_VALIDATION_DEFERRED" in @@ -319,10 +319,10 @@ else: assert float(cfg.trainer.limit_val_batches) > 0 assert cfg.name.startswith("planar_v2_usocket_") assert experiment in { - "sim/pusht/planar_v2_usocket_direct_bc", - "sim/pusht/planar_v2_usocket_arc_bc", - "sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc", - "sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", + "arc_sim/pusht/planar_v2_usocket_direct_bc", + "arc_sim/pusht/planar_v2_usocket_arc_bc", + "arc_sim/pusht/planar_v2_usocket_arc_hybrid_D40_M16_R24deg_bc", + "arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", } PY diff --git a/scripts/ice/launch_planar_v2_dp_cotrain_clean.sbatch b/scripts/ice/launch_planar_v2_dp_cotrain_clean.sbatch index 20466d8ac6..cbd7db561b 100644 --- a/scripts/ice/launch_planar_v2_dp_cotrain_clean.sbatch +++ b/scripts/ice/launch_planar_v2_dp_cotrain_clean.sbatch @@ -15,7 +15,7 @@ PY=${ICE_DP_PYTHON:-/storage/ice1/5/4/paphiwetsa3/egoverse/unite-usocket-2026090 LAUNCHER=${BASH_SOURCE[0]} U_DATA=${ICE_DP_U_SOCKET_DATA:-$ROOT/data/u_socket_3000_v2_clean} CHAIN_DATA=${ICE_DP_CHAIN_GRIPPER_DATA:-$ROOT/data/chain_gripper_3000_v2} -EXPERIMENT=sim/pusht/planar_v2_cotrain_clean_dp_standard +EXPERIMENT=arc_sim/pusht/planar_v2_cotrain_clean_dp_standard PARAMETER_COUNT=51506981 FULL_STEPS=240000 VALIDATE_EVERY=10000 diff --git a/scripts/ice/launch_unite.sbatch b/scripts/ice/launch_unite.sbatch index 88bb4f65fa..fcb7a94e5c 100755 --- a/scripts/ice/launch_unite.sbatch +++ b/scripts/ice/launch_unite.sbatch @@ -118,10 +118,10 @@ case "$ICE_GPU_CONTRACT" in *) die "ICE_GPU_CONTRACT must be H100_OR_H200, L40S, or A100" ;; esac case "$ICE_EXPERIMENT:$ICE_MODEL" in - pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt8_s42) ;; - pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt8_flow42_s42) ;; - pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt16_s42) ;; - pusht/action_latent_vfm_usocket_val01_h16:bf/us_action_latent_vfm_nt16_d8_h512_s42) + arc_sim/pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt8_s42) ;; + arc_sim/pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt8_flow42_s42) ;; + arc_sim/pusht/unite_usocket_register_sweep_val01_h16:bf/us_unite_register_shared_nt16_s42) ;; + arc_sim/pusht/action_latent_vfm_usocket_val01_h16:bf/us_action_latent_vfm_nt16_d8_h512_s42) ICE_FAMILY=action_latent_vfm ICE_UNITE_DIAGNOSTICS=true ;; @@ -372,10 +372,10 @@ assert cfg.run_provenance.split_manifest_sha256 == split_manifest_sha256 assert int(cfg.data.train_dataloader_params.pushshapes_sim_u_socket.batch_size) == int(train_batch_size) assert int(cfg.data.valid_dataloader_params.pushshapes_sim_u_socket.batch_size) == int(valid_batch_size) if family == "paper_dp": - assert experiment == "sim/pusht/planar_v2_usocket_dp_paper" + assert experiment == "arc_sim/pusht/planar_v2_usocket_dp_paper" assert cfg.name == "planar_v2_usocket_dp_paper_h16" elif family == "unite": - assert experiment == "sim/pusht/unite_usocket_register_sweep_val01_h16" + assert experiment == "arc_sim/pusht/unite_usocket_register_sweep_val01_h16" assert cfg.name == "unite_usocket_register_sweep_h16" assert int(cfg.evaluator.energy_score_max_batches_per_rank) == 1 assert int(cfg.evaluator.energy_score_validation_view.world_size) == world_size @@ -401,7 +401,7 @@ elif family == "unite": else: assert bool(cfg.evaluator.unite_diagnostics.enabled) is False elif family == "action_latent_vfm": - assert experiment == "sim/pusht/action_latent_vfm_usocket_val01_h16" + assert experiment == "arc_sim/pusht/action_latent_vfm_usocket_val01_h16" assert cfg.name == "action_latent_vfm_usocket_h16" assert cfg.model._target_ == "egomimic.pl_utils.pl_model.ModelWrapper" assert cfg.model.training_behavior._target_.endswith( diff --git a/scripts/integration/matrix.yaml b/scripts/integration/matrix.yaml index 11750b465f..fdf68d198c 100644 --- a/scripts/integration/matrix.yaml +++ b/scripts/integration/matrix.yaml @@ -21,38 +21,38 @@ cases: hpt-eva: suite: hpt config: train_zarr_cartesian - overrides: [model=hpt_bc_flow_eva, data=eva, evaluator=eval_hpt] + overrides: [model=hpt/bc_flow_eva, data=eva, evaluator=eval_cartesian_camera] sources: {eva_bimanual: eva} hpt-aria: suite: hpt config: train_zarr_cartesian - overrides: [model=hpt_bc_flow_human, data=aria, evaluator=eval_hpt] + overrides: [model=hpt/bc_flow_human, data=aria, evaluator=eval_cartesian_camera] sources: {human_bimanual: aria} hpt-mecka: suite: hpt config: train_zarr_cartesian - overrides: [model=hpt_bc_flow_mecka, data=mecka, evaluator=eval_hpt] + overrides: [model=hpt/bc_flow_mecka, data=mecka, evaluator=eval_cartesian_camera] sources: {human_bimanual: mecka} hpt-scale: suite: hpt config: train_zarr_cartesian - overrides: [model=hpt_bc_flow_scale, data=scale, evaluator=eval_hpt] + overrides: [model=hpt/bc_flow_scale, data=scale, evaluator=eval_cartesian_camera] sources: {human_bimanual: scale} hpt-yam: suite: hpt config: train_zarr_cartesian - overrides: [+experiment=abc/yam_fstshirt_hpt, evaluator=eval_hpt] + overrides: [model=hpt/yam_bimanual, data=abc_arc/abc_fstshirt_bc, evaluator=eval_cartesian_camera] sources: {yam_bimanual: yam} inference_identity: 7 hpt-cotrain: suite: hpt config: train_zarr_cartesian - overrides: [model=hpt_cotrain_flow_seperate_head, data=cotrain_pi_base, evaluator=eval_hpt] + overrides: [model=hpt/cotrain_flow_seperate_head, data=cotrain_pi_base, evaluator=eval_cartesian_camera] sources: {eva_bimanual: eva, human_bimanual: aria} pi-eva: suite: pi config: train_zarr_cartesian - overrides: [model=pi0.5_bc_eva, data=eva, evaluator=eval_pi] + overrides: [model=pi/bc_eva_legacy, data=eva, evaluator=eval_cartesian_camera_prompt] sources: {eva_bimanual: eva} runtime_overrides: &pi_runtime model.pipeline.stages.0.policy.gradient_checkpointing: true @@ -62,31 +62,31 @@ cases: pi-aria: suite: pi config: train_zarr_cartesian - overrides: [model=pi0.5_bc_aria, data=aria, evaluator=eval_pi] + overrides: [model=pi/bc_aria_legacy, data=aria, evaluator=eval_cartesian_camera_prompt] sources: {human_bimanual: aria} runtime_overrides: *pi_runtime pi-mecka: suite: pi config: train_zarr_cartesian - overrides: [model=pi0.5_bc_mecka, data=mecka, evaluator=eval_pi] + overrides: [model=pi/bc_mecka_legacy, data=mecka, evaluator=eval_cartesian_camera_prompt] sources: {human_bimanual: mecka} runtime_overrides: *pi_runtime pi-scale: suite: pi config: train_zarr_cartesian - overrides: [model=pi0.5_bc_mecka, data=scale, evaluator=eval_pi] + overrides: [model=pi/bc_mecka_legacy, data=scale, evaluator=eval_cartesian_camera_prompt] sources: {human_bimanual: scale} runtime_overrides: *pi_runtime pi-cotrain: suite: pi config: train_zarr_cartesian - overrides: [model=pi0.5_cotrain_eva_aria, data=cotrain_pi_base, evaluator=eval_pi] + overrides: [model=pi/cotrain_eva_aria_legacy, data=cotrain_pi_base, evaluator=eval_cartesian_camera_prompt] sources: {eva_bimanual: eva, human_bimanual: aria} runtime_overrides: *pi_runtime hpt-ddp: suite: ddp config: train_zarr_cartesian - overrides: [model=hpt_bc_flow_eva, data=eva, evaluator=eval_hpt] + overrides: [model=hpt/bc_flow_eva, data=eva, evaluator=eval_cartesian_camera] sources: {eva_bimanual: eva} trainer: devices: 2 diff --git a/scripts/pi05/debug_mecka_fold_2g.sbatch b/scripts/pi05/debug_mecka_fold_2g.sbatch index 2d62f32b60..4b9fca611c 100644 --- a/scripts/pi05/debug_mecka_fold_2g.sbatch +++ b/scripts/pi05/debug_mecka_fold_2g.sbatch @@ -40,8 +40,8 @@ NORM_PATH="${NORM_PATH-/storage/project/r-dxu345-0/agao81/norm_stats/mecka_all_6 ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/mecka_fold_flagship_pi_6d - model=pi05/pi0.5_bc_mecka_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_mecka_6d + evaluator=eval_cartesian_wrist_6d_prompt launch_params.gpus_per_node=2 trainer.check_val_every_n_epoch=10 # norm stats from the full mecka set (superset, same distribution) — avoids diff --git a/scripts/pi05/eval_aria_fold_grip.sbatch b/scripts/pi05/eval_aria_fold_grip.sbatch index ce53ea4be8..77cca943fb 100644 --- a/scripts/pi05/eval_aria_fold_grip.sbatch +++ b/scripts/pi05/eval_aria_fold_grip.sbatch @@ -28,8 +28,8 @@ CKPT="${CKPT-/storage/project/r-dxu345-0/agao81/EgoVerse/logs/pi05_mecka_all_6d/ "$PY" -u egomimic/trainHydra.py \ --config-name=train_zarr_cartesian_pi \ data=pi05/aria_fold_pi_6d_grip \ - model=pi05/pi0.5_ft_aria_fold_grip \ - evaluator=pi05/eval_pi_wristframe_6d_grip \ + model=pi/ft_aria_fold_grip \ + evaluator=eval_cartesian_wrist_6d_gripper_prompt \ launch_params.gpus_per_node=1 \ ++mode=eval \ "ckpt_path='$CKPT'" \ diff --git a/scripts/pi05/norm_abc_eva_6d.sbatch b/scripts/pi05/norm_abc_eva_6d.sbatch index 28a6b917d8..326e934be0 100644 --- a/scripts/pi05/norm_abc_eva_6d.sbatch +++ b/scripts/pi05/norm_abc_eva_6d.sbatch @@ -24,7 +24,7 @@ mkdir -p "$OUT" "$PY" -u scripts/data/precompute_norm_stats.py \ --data pi05/abc_eva_pi_6d \ - --model pi05/pi0.5_bc_abc_eva_6d \ + --model pi/bc_abc_eva_6d \ --sample-frac 0.1 \ --max-samples 2000000 \ --num-workers 22 \ diff --git a/scripts/pi05/norm_abc_eva_foldviz_debug.sbatch b/scripts/pi05/norm_abc_eva_foldviz_debug.sbatch index 8e4641b1c9..a4f09db8a5 100644 --- a/scripts/pi05/norm_abc_eva_foldviz_debug.sbatch +++ b/scripts/pi05/norm_abc_eva_foldviz_debug.sbatch @@ -24,7 +24,7 @@ mkdir -p "$OUT" "$PY" -u scripts/data/precompute_norm_stats.py \ --data pi05/abc_eva_pi_6d_foldviz \ - --model pi05/pi0.5_bc_abc_eva_6d \ + --model pi/bc_abc_eva_6d \ --sample-frac 0.005 \ --max-samples 50000 \ --num-workers 22 \ diff --git a/scripts/pi05/norm_abc_stationery_rl2yam.sbatch b/scripts/pi05/norm_abc_stationery_rl2yam.sbatch index 26851116ca..4b8f96bcab 100644 --- a/scripts/pi05/norm_abc_stationery_rl2yam.sbatch +++ b/scripts/pi05/norm_abc_stationery_rl2yam.sbatch @@ -28,7 +28,7 @@ mkdir -p "$OUT" echo "=== commit=$(git rev-parse --short HEAD) node=$SLURMD_NODENAME $(date) ===" "$PY" -u scripts/data/precompute_norm_stats.py \ --data pi05/abc_stationery_pi_6d \ - --model pi05/pi0.5_bc_abc_eva_6d \ + --model pi/bc_abc_eva_6d \ --sample-frac 0.2 \ --max-samples 2000000 \ --num-workers 22 \ diff --git a/scripts/pi05/norm_aria_fold_grip.sbatch b/scripts/pi05/norm_aria_fold_grip.sbatch index bad70e1bae..d959847c67 100644 --- a/scripts/pi05/norm_aria_fold_grip.sbatch +++ b/scripts/pi05/norm_aria_fold_grip.sbatch @@ -24,7 +24,7 @@ mkdir -p "$OUT" "$PY" -u scripts/data/precompute_norm_stats.py \ --data pi05/aria_fold_pi_6d_grip \ - --model pi05/pi0.5_ft_aria_fold_grip \ + --model pi/ft_aria_fold_grip \ --sample-frac 0.1 \ --max-samples 2000000 \ --num-workers 22 \ diff --git a/scripts/pi05/norm_mecka_all_6d.sbatch b/scripts/pi05/norm_mecka_all_6d.sbatch index 2024abcc78..8e168eab66 100644 --- a/scripts/pi05/norm_mecka_all_6d.sbatch +++ b/scripts/pi05/norm_mecka_all_6d.sbatch @@ -25,7 +25,7 @@ export OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 "$PY" scripts/data/precompute_norm_stats.py \ --data pi05/mecka_all_pi_6d \ - --model pi05/pi0.5_bc_mecka_6d \ + --model pi/bc_mecka_6d \ --sample-frac 0.1 \ --num-workers 22 \ --out /storage/project/r-dxu345-0/agao81/norm_stats/mecka_all_6d diff --git a/scripts/pi05/smoke_abc_eva.sbatch b/scripts/pi05/smoke_abc_eva.sbatch index 75c1514d26..84ef2dea7c 100644 --- a/scripts/pi05/smoke_abc_eva.sbatch +++ b/scripts/pi05/smoke_abc_eva.sbatch @@ -29,8 +29,8 @@ INIT_CKPT=/storage/project/r-dxu345-0/agao81/EgoVerse/logs/pi05_mecka_all_6d/pi0 "$PY" -u egomimic/trainHydra.py \ --config-name=train_zarr_cartesian_pi \ data=pi05/abc_eva_pi_6d \ - model=pi05/pi0.5_bc_abc_eva_6d \ - evaluator=pi05/eval_pi_wristframe_6d \ + model=pi/bc_abc_eva_6d \ + evaluator=eval_cartesian_wrist_6d_prompt \ trainer=debug \ launch_params.gpus_per_node=1 \ evaluator.viz_every_n_epochs=4 \ diff --git a/scripts/pi05/smoke_abc_stationery_pi.sbatch b/scripts/pi05/smoke_abc_stationery_pi.sbatch index 6a974fbd14..c3d69631bf 100644 --- a/scripts/pi05/smoke_abc_stationery_pi.sbatch +++ b/scripts/pi05/smoke_abc_stationery_pi.sbatch @@ -32,8 +32,8 @@ echo "=== commit=$(git rev-parse --short HEAD) node=$SLURMD_NODENAME $(date) === "$PY" -u egomimic/trainHydra.py \ --config-name=train_zarr_cartesian_pi \ data=pi05/abc_stationery_pi_6d \ - model=pi05/pi0.5_bc_abc_eva_6d \ - evaluator=pi05/eval_pi_wristframe_6d \ + model=pi/bc_abc_eva_6d \ + evaluator=eval_cartesian_wrist_6d_prompt \ trainer=debug \ launch_params.gpus_per_node=1 \ evaluator.viz_every_n_epochs=4 \ diff --git a/scripts/pi05/smoke_aria_fold_grip.sbatch b/scripts/pi05/smoke_aria_fold_grip.sbatch index 157aeb98ef..c7045446db 100644 --- a/scripts/pi05/smoke_aria_fold_grip.sbatch +++ b/scripts/pi05/smoke_aria_fold_grip.sbatch @@ -28,8 +28,8 @@ INIT_CKPT=/storage/project/r-dxu345-0/agao81/EgoVerse/logs/pi05_mecka_all_6d/pi0 "$PY" -u egomimic/trainHydra.py \ --config-name=train_zarr_cartesian_pi \ data=pi05/aria_fold_pi_6d_grip \ - model=pi05/pi0.5_ft_aria_fold_grip \ - evaluator=pi05/eval_pi_wristframe_6d_grip \ + model=pi/ft_aria_fold_grip \ + evaluator=eval_cartesian_wrist_6d_gripper_prompt \ trainer=debug \ launch_params.gpus_per_node=1 \ evaluator.viz_every_n_epochs=4 \ diff --git a/scripts/pi05/smoke_mecka_abc_6d.sbatch b/scripts/pi05/smoke_mecka_abc_6d.sbatch index 74298b2c30..d8d1198c76 100644 --- a/scripts/pi05/smoke_mecka_abc_6d.sbatch +++ b/scripts/pi05/smoke_mecka_abc_6d.sbatch @@ -34,8 +34,8 @@ NORM_PATH="${NORM_PATH-/storage/project/r-dxu345-0/agao81/norm_stats/mecka_abc_6 "$PY" -u egomimic/trainHydra.py \ --config-name=train_zarr_cartesian_pi \ data=pi05/mecka_abc_pi_6d \ - model=pi05/pi0.5_bc_mecka_abc_6d \ - evaluator=pi05/eval_pi_wristframe_6d \ + model=pi/bc_mecka_abc_6d \ + evaluator=eval_cartesian_wrist_6d_prompt \ trainer=debug \ launch_params.gpus_per_node=1 \ evaluator.viz_every_n_epochs=4 \ diff --git a/scripts/pi05/smoke_mecka_all_6d.sbatch b/scripts/pi05/smoke_mecka_all_6d.sbatch index d23eb3d93e..cbffe2559a 100644 --- a/scripts/pi05/smoke_mecka_all_6d.sbatch +++ b/scripts/pi05/smoke_mecka_all_6d.sbatch @@ -34,8 +34,8 @@ NORM_PATH="${NORM_PATH-}" ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/mecka_all_pi_6d - model=pi05/pi0.5_bc_mecka_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_mecka_6d + evaluator=eval_cartesian_wrist_6d_prompt trainer=debug launch_params.gpus_per_node=1 evaluator.viz_every_n_epochs=4 diff --git a/scripts/pi05/train_abc_eva_ft.sbatch b/scripts/pi05/train_abc_eva_ft.sbatch index fe3721b38d..0f3407092d 100644 --- a/scripts/pi05/train_abc_eva_ft.sbatch +++ b/scripts/pi05/train_abc_eva_ft.sbatch @@ -60,8 +60,8 @@ fi ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/abc_eva_pi_6d - model=pi05/pi0.5_bc_abc_eva_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_abc_eva_6d + evaluator=eval_cartesian_wrist_6d_prompt launch_params.gpus_per_node=8 "model.optimizer.lr=$LR" "norm_stats.precomputed_norm_path=$NORM_PATH" diff --git a/scripts/pi05/train_abc_stationery_pi.sbatch b/scripts/pi05/train_abc_stationery_pi.sbatch index ffbf17eba2..d4ee9df66b 100644 --- a/scripts/pi05/train_abc_stationery_pi.sbatch +++ b/scripts/pi05/train_abc_stationery_pi.sbatch @@ -25,9 +25,9 @@ # data=pi05/abc_stationery_pi_6d wristframe-6D actions, rl2yam extrinsics, # ABC's dense sub-task annotation spans (stock # annotation pipeline, no injected task prompt) -# model=pi05/pi0.5_bc_abc_eva_6d eva_bimanual domain (the mirror zarrs still say +# model=pi/bc_abc_eva_6d eva_bimanual domain (the mirror zarrs still say # eva_bimanual), 20D native, no embodiment label -# evaluator=pi05/eval_pi_wristframe_6d + the train_viz second head +# evaluator=eval_cartesian_wrist_6d_prompt + the train_viz second head # init = the DEFAULT pi0.5 base checkpoint (pi05_base_pytorch) and nothing # else — BC from scratch, no mecka/human pretrain in the lineage # norm = norm_stats/abc_stationery_rl2yam (this task, this extrinsics key — @@ -80,8 +80,8 @@ echo "=== commit=$(git rev-parse --short HEAD) node=$SLURMD_NODENAME $(date) === ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/abc_stationery_pi_6d - model=pi05/pi0.5_bc_abc_eva_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_abc_eva_6d + evaluator=eval_cartesian_wrist_6d_prompt launch_params.gpus_per_node=4 "trainer.max_epochs=$MAX_EPOCHS" trainer.min_epochs=0 diff --git a/scripts/pi05/train_aria_fold_grip.sbatch b/scripts/pi05/train_aria_fold_grip.sbatch index 4e0bfe8c50..ced679ddec 100644 --- a/scripts/pi05/train_aria_fold_grip.sbatch +++ b/scripts/pi05/train_aria_fold_grip.sbatch @@ -56,8 +56,8 @@ fi ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/aria_fold_pi_6d_grip - model=pi05/pi0.5_ft_aria_fold_grip - evaluator=pi05/eval_pi_wristframe_6d_grip + model=pi/ft_aria_fold_grip + evaluator=eval_cartesian_wrist_6d_gripper_prompt launch_params.gpus_per_node=4 "model.optimizer.lr=$LR" "norm_stats.precomputed_norm_path=$NORM_PATH" diff --git a/scripts/pi05/train_aria_fold_grip_evahuman.sbatch b/scripts/pi05/train_aria_fold_grip_evahuman.sbatch index 1bb94142d1..a1d1ce400c 100644 --- a/scripts/pi05/train_aria_fold_grip_evahuman.sbatch +++ b/scripts/pi05/train_aria_fold_grip_evahuman.sbatch @@ -56,8 +56,8 @@ fi ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/aria_fold_pi_6d_grip - model=pi05/pi0.5_ft_aria_fold_grip - evaluator=pi05/eval_pi_wristframe_6d_grip + model=pi/ft_aria_fold_grip + evaluator=eval_cartesian_wrist_6d_gripper_prompt launch_params.gpus_per_node=4 "model.optimizer.lr=$LR" "norm_stats.precomputed_norm_path=$NORM_PATH" diff --git a/scripts/pi05/train_mecka_all_6d.sbatch b/scripts/pi05/train_mecka_all_6d.sbatch index ca673cfb50..56d559210c 100644 --- a/scripts/pi05/train_mecka_all_6d.sbatch +++ b/scripts/pi05/train_mecka_all_6d.sbatch @@ -76,8 +76,8 @@ fi ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/mecka_all_pi_6d - model=pi05/pi0.5_bc_mecka_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_mecka_6d + evaluator=eval_cartesian_wrist_6d_prompt # Second eval head: spot-check metrics + videos on TRAIN data each val # (train_viz/ prefix, videos_train_viz/). Loader defined in the data config. launch_params.gpus_per_node=8 diff --git a/scripts/pi05/train_mecka_all_6d_4g.sbatch b/scripts/pi05/train_mecka_all_6d_4g.sbatch index 470a703c3b..0c6ab7240c 100644 --- a/scripts/pi05/train_mecka_all_6d_4g.sbatch +++ b/scripts/pi05/train_mecka_all_6d_4g.sbatch @@ -61,8 +61,8 @@ fi ARGS=( --config-name=train_zarr_cartesian_pi data=pi05/mecka_all_pi_6d - model=pi05/pi0.5_bc_mecka_6d - evaluator=pi05/eval_pi_wristframe_6d + model=pi/bc_mecka_6d + evaluator=eval_cartesian_wrist_6d_prompt launch_params.gpus_per_node=4 "model.optimizer.lr=$LR" "norm_stats.precomputed_norm_path=$NORM_PATH" diff --git a/scripts/train/action_flow_run_child.sh b/scripts/train/action_flow_run_child.sh index 8e816f738e..9bc3074583 100644 --- a/scripts/train/action_flow_run_child.sh +++ b/scripts/train/action_flow_run_child.sh @@ -1,5 +1,5 @@ # The explicit joint route may never inherit the historical step-wide save transport. -if [[ "${AF_EXPERIMENT:-}" = sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 || "${AF_EXPERIMENT:-}" = sim/libero/action_flow_libero_spatial_sharpa_mot_s42 ]]; then +if [[ "${AF_EXPERIMENT:-}" = arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 || "${AF_EXPERIMENT:-}" = arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42 ]]; then test "${AF_CHECKPOINT_TRANSPORT:-}" = trainer-pidfd || { printf '%s\n' 'joint child requires audited trainer-pidfd checkpoint transport' >&2; exit 2; } diff --git a/scripts/train/launch_action_flow_usocket.sbatch b/scripts/train/launch_action_flow_usocket.sbatch index fc2af7b732..badff85d0b 100644 --- a/scripts/train/launch_action_flow_usocket.sbatch +++ b/scripts/train/launch_action_flow_usocket.sbatch @@ -64,12 +64,12 @@ done # Select only an existing current recipe in the exact bound source checkout. # Parent templates and historical recipes are not fresh-launch entry points. case "$AF_EXPERIMENT" in - sim/pusht/parents/*|pusht_historical/*) die "select a current PushT recipe, not a parent or historical config" ;; - pusht/*|libero/*|libero_historical/*|oat/*) die "retired experiment selector; use the sim/ menu for new launches" ;; + arc_sim/pusht/parents/*|pusht_historical/*) die "select a current PushT recipe, not a parent or historical config" ;; + arc_sim/*|pusht/*|libero/*|libero_historical/*|oat/*) die "retired experiment selector; use the arc_sim/ menu for new launches" ;; *..*|/*) die "invalid experiment selector" ;; esac case "$AF_EXPERIMENT" in - sim/pusht/*) test -f "$AF_REPO/egomimic/hydra_configs/experiment/$AF_EXPERIMENT.yaml" || die "recipe is absent or retired in the selected source: $AF_EXPERIMENT" ;; + arc_sim/pusht/*) test -f "$AF_REPO/egomimic/hydra_configs/experiment/$AF_EXPERIMENT.yaml" || die "recipe is absent or retired in the selected source: $AF_EXPERIMENT" ;; esac AF_LAUNCH_MODE=${AF_LAUNCH_MODE:-preflight} @@ -127,7 +127,7 @@ if test "$AF_LAUNCH_MODE" = run; then required AF_EXPECTED_TIME_LIMIT fi # Explicit unequal-width joint route reuses the maintained phase/child adapters. -if [[ "$AF_EXPERIMENT" = sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 || "$AF_EXPERIMENT" = sim/libero/action_flow_libero_spatial_sharpa_mot_s42 ]]; then +if [[ "$AF_EXPERIMENT" = arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42 || "$AF_EXPERIMENT" = arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42 ]]; then test "$AF_LAUNCH_MODE" != schema || die "joint route uses typed preflight, not historical native schema" test -z "${AF_INITIAL_CHECKPOINT:-}" || die "joint fresh launch cannot use historical resume adapter" if test -z "${SLURM_STEP_ID:-}" || test "$SLURM_STEP_ID" = batch; then @@ -183,17 +183,17 @@ AF_H696_EULER=false AF_NATIVE_LIBERO=false AF_NATIVE_RECIPE_BASE=$AF_EXPERIMENT case "$AF_EXPERIMENT" in - sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42|sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42|sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42|sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42) + arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42|arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42|arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42|arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42) AF_NATIVE_RECIPE_BASE=${AF_EXPERIMENT/_80k_av0_/_80k_} ;; esac case "$AF_NATIVE_RECIPE_BASE" in - sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42|sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42|sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42|sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) + arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42|arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42|arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42|arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) AF_NATIVE_LIBERO=true AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 - AF_EXPECTED_CONFIG_NAME=${AF_EXPERIMENT#sim/libero_historical/} + AF_EXPECTED_CONFIG_NAME=${AF_EXPERIMENT#arc_sim/libero_historical/} AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 AF_FULL_MAX_STEPS=80000 AF_FULL_VALIDATE_EVERY=15000 @@ -217,7 +217,7 @@ case "$AF_NATIVE_RECIPE_BASE" in done case "$AF_NATIVE_PHASE" in preflight|smoke|full) ;; *) die "invalid native phase" ;; esac case "$AF_NATIVE_RECIPE_BASE:$AF_SEED" in - sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42:42|sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42:43|sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42:42|sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42:43|sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42:42|sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42:42) ;; + arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42:42|arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42:43|arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42:42|arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42:43|arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42:42|arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42:42) ;; *) die "unsupported native suite/training seed" ;; esac AF_DATASET_VALIDATOR=$AF_NATIVE_DATASET_VALIDATOR @@ -226,7 +226,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_SMOKE_VERIFIER=$AF_NATIVE_SMOKE_VERIFIER ;; - sim/pusht/action_flow_chain_native4_points6_manual4919_h384_150k_s42) + arc_sim/pusht/action_flow_chain_native4_points6_manual4919_h384_150k_s42) AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -246,7 +246,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_PAIRED_CHAIN_MANUAL4919=true AF_WANDB_TAGS="[pushshapes,chain-gripper,native4,points6,manual4919,cotrain,action-flow,private-codecs,shared-field,h384d12h12,h16,z8x16,k14,dopri5,cfg4,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_cotrain_usocket_chain_obstacle_unite_arch_denoiser90m_sum14_adamw_lr1e5_s42) + arc_sim/pusht/action_flow_cotrain_usocket_chain_obstacle_unite_arch_denoiser90m_sum14_adamw_lr1e5_s42) AF_METHOD=latent_fm_stopgrad_cotrain AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -261,7 +261,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_SECOND_SOURCE=pushshapes_sim_chain_gripper AF_WANDB_TAGS="[pushshapes,u-socket,chain-gripper,obstacle,cotrain,action-flow,route-local-codecs,shared-field,h16,z8x16,k14,adamw,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42) + arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42) AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -277,7 +277,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_EXPECTED_PARAMETER_COUNT=97956613 AF_WANDB_TAGS="[pushshapes,chain-gripper,action-flow,unite-h384,sum14,cfg4,val8x32,h16,z8x16,k14,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42) + arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42) AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -291,7 +291,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_EXPECTED_PARAMETER_COUNT=155626932 AF_WANDB_TAGS="[pushshapes,u-socket,action-flow,unite-denoiser90m,codec-h384,sum14,cfg4,smoke-val1x32,full-no-val,h16,z8x16,k14,no-checkpointing,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_usocket_latent_fm_sg_unite_arch_denoiser90m_sum14_cfg4_adamw_lr1e5_finalval1_s42) + arc_sim/pusht/action_flow_usocket_latent_fm_sg_unite_arch_denoiser90m_sum14_cfg4_adamw_lr1e5_finalval1_s42) AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -305,7 +305,7 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_EXPECTED_PARAMETER_COUNT=155626932 AF_WANDB_TAGS="[pushshapes,u-socket,action-flow,unite-architecture-only,unite-denoiser90m,codec-h384,adamw-lr1e-5,sum14,cfg4,smoke-val1x32,full-no-val,h16,z8x16,k14,no-checkpointing,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_chain_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42) + arc_sim/pusht/action_flow_chain_latent_fm_sg_unite_denoiser90m_sum14_cfg4_finalval1_s42) AF_METHOD=latent_fm_stopgrad_unite AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 @@ -321,28 +321,28 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_EXPECTED_PARAMETER_COUNT=155627445 AF_WANDB_TAGS="[pushshapes,chain-gripper,action-flow,unite-denoiser90m,codec-h384,sum14,cfg4,smoke-val1x32,full-no-val,ckpt20k,h16,z8x16,k14,no-checkpointing,single-gpu,$AF_RUN_KIND]" ;; - sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42) + arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42) AF_METHOD=latent_fm_stopgrad AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_latent_fm_sg_recon1_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42) + arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42) AF_METHOD=latent_fm_stopgrad AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42) + arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42) AF_METHOD=latent_fm_stopgrad AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42) + arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42) AF_METHOD=gaussian_bridge_likelihood # Transport placeholders only; no FM/reconstruction flags enter this model. AF_RECONSTRUCTION_WEIGHT=0 @@ -350,32 +350,32 @@ case "$AF_NATIVE_RECIPE_BASE" in AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_bridge_likelihood_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_graph_section_s42) + arc_sim/pusht/action_flow_bc_usocket_graph_section_s42) AF_METHOD=graph_section_diagnostic AF_RECONSTRUCTION_WEIGHT=0 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_graph_section_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_recon1_s42) + arc_sim/pusht/action_flow_bc_usocket_recon1_s42) AF_RECONSTRUCTION_WEIGHT=1 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_recon1_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_recon10_s42) + arc_sim/pusht/action_flow_bc_usocket_recon10_s42) AF_RECONSTRUCTION_WEIGHT=10 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_recon10_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=0 ;; - sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42) + arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42) AF_RECONSTRUCTION_WEIGHT=10 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_recon10_warmup10k_s42 AF_FULL_RECONSTRUCTION_ONLY_WARMUP_STEPS=10000 ;; - sim/pusht/action_flow_bc_usocket_recon100_s42) + arc_sim/pusht/action_flow_bc_usocket_recon100_s42) AF_RECONSTRUCTION_WEIGHT=100 AF_FLOW_WEIGHT=1 AF_EXPECTED_CONFIG_NAME=action_flow_bc_usocket_recon100_s42 @@ -437,10 +437,10 @@ fi if test "$AF_METHOD" != action_flow_joint || [[ "$AF_EXPERIMENT" = *warmup10k* ]]; then AF_COMPARISON_EXPERIMENT= -elif test "$AF_EXPERIMENT" = sim/pusht/action_flow_bc_usocket_recon1_s42; then - AF_COMPARISON_EXPERIMENT=sim/pusht/action_flow_bc_usocket_recon10_s42 +elif test "$AF_EXPERIMENT" = arc_sim/pusht/action_flow_bc_usocket_recon1_s42; then + AF_COMPARISON_EXPERIMENT=arc_sim/pusht/action_flow_bc_usocket_recon10_s42 else - AF_COMPARISON_EXPERIMENT=sim/pusht/action_flow_bc_usocket_recon1_s42 + AF_COMPARISON_EXPERIMENT=arc_sim/pusht/action_flow_bc_usocket_recon1_s42 fi [[ "$AF_EXPECTED_HEAD" =~ ^[0-9a-f]{40}$ ]] || \ @@ -456,16 +456,16 @@ fi if test "$AF_NATIVE_LIBERO" = true; then # Suite corpus identities are immutable; destination receipt paths may differ. case "$AF_NATIVE_RECIPE_BASE" in - sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42) + arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42) native_logical_sha=3ed2b86cf97eaa27045af81e6d6c4834c8659d15eda69a2394d835e602014e1d native_historical_split_sha=c588c831ef0deab61a59a21840ea9e11dafe92a170d9023a87e0acee831df236 ;; - sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42) + arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42) native_logical_sha=ee28d02e8c6cd28fb0a235fab2ccc649b662e6f3e10eb096d89be082d2871860 native_historical_split_sha=15640eada8102c7071dcfcd153ef438cb98c61835474a2fb2c3c9415cf6d714e ;; - sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42) + arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42) native_logical_sha=c190385dfc8a7d05b966a4b5f94651826b8a906b1e63d8b348edc70df52c1f60 native_historical_split_sha=207c24115104bf0545e7f165abb91cf052eb99f6568446f11e835f9a3e4c6af8 ;; - sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) + arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) native_logical_sha=40b5e3ccf4febcbae470690755adc8aa6a39471a4fcb7fa585a9bdb8e59c325d native_historical_split_sha=5ade2a86fad7552457e093944fd60efec8b51fbe9dfd20831f627d1c56e9b5d7 ;; *) die "unsupported native corpus identity profile" ;; @@ -659,10 +659,10 @@ export ICE_EXPECTED_RUN_ID=$AF_WANDB_RUN_ID export ICE_EXPECTED_SPLIT_SHA256=$AF_EXPECTED_SPLIT_MANIFEST_SHA256 if test "$AF_NATIVE_LIBERO" = true; then case "$AF_NATIVE_RECIPE_BASE" in - sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42) export LIBERO10_REPLAY_ROOT=$AF_DATASET_DIR ;; - sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42) export LIBERO_SPATIAL_REPLAY_ROOT=$AF_DATASET_DIR ;; - sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42) export LIBERO_GOAL_REPLAY_ROOT=$AF_DATASET_DIR ;; - sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) export LIBERO_OBJECT_REPLAY_ROOT=$AF_DATASET_DIR ;; + arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42) export LIBERO10_REPLAY_ROOT=$AF_DATASET_DIR ;; + arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42) export LIBERO_SPATIAL_REPLAY_ROOT=$AF_DATASET_DIR ;; + arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42) export LIBERO_GOAL_REPLAY_ROOT=$AF_DATASET_DIR ;; + arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42) export LIBERO_OBJECT_REPLAY_ROOT=$AF_DATASET_DIR ;; *) die "unsupported native replay environment" ;; esac elif test "$AF_SOURCE" = pushshapes_sim_u_socket; then @@ -801,7 +801,7 @@ else ) fi -if test "$AF_EXPERIMENT" = sim/pusht/action_flow_cotrain_uc_multiplier_interpolation; then +if test "$AF_EXPERIMENT" = arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation; then BASE_OVERRIDES+=(model.pipeline.stages.6.inference_method=euler model.pipeline.stages.6.num_inference_steps=50 ++run_provenance.validation_inference_sampler=euler ++run_provenance.validation_inference_steps=50) fi if test "$AF_SPEED_DIAGNOSTIC" = true; then diff --git a/scripts/train/launch_synthetic_manifold.sbatch b/scripts/train/launch_synthetic_manifold.sbatch index 18686e9a23..19a8ff50a8 100755 --- a/scripts/train/launch_synthetic_manifold.sbatch +++ b/scripts/train/launch_synthetic_manifold.sbatch @@ -20,8 +20,8 @@ PYTHON=/coc/flash7/paphiwetsa3/projects/EgoVerse7/.venv/bin/python test "$(git -C "$SOURCE" rev-parse HEAD)" = "$EXPECTED_SOURCE_COMMIT" test -z "$(git -C "$SOURCE" status --porcelain)" case "$SLURM_ARRAY_TASK_ID" in - 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus.json ;; - 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus.json ;; + 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus.json ;; + 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus.json ;; *) exit 2 ;; esac cd "$SOURCE" diff --git a/scripts/train/launch_synthetic_manifold_codec10k.sbatch b/scripts/train/launch_synthetic_manifold_codec10k.sbatch index 91c4f7a761..a42902b2bf 100644 --- a/scripts/train/launch_synthetic_manifold_codec10k.sbatch +++ b/scripts/train/launch_synthetic_manifold_codec10k.sbatch @@ -18,10 +18,10 @@ PYTHON=/coc/flash7/paphiwetsa3/projects/EgoVerse7/.venv/bin/python test "$(git -C "$SOURCE" rev-parse HEAD)" = "$EXPECTED_SOURCE_COMMIT" test -z "$(git -C "$SOURCE" status --porcelain)" case "$SLURM_ARRAY_TASK_ID" in - 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent2_codec10k.json ;; - 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json ;; - 2) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4_codec10k.json ;; - 3) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json ;; + 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent2_codec10k.json ;; + 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent2_codec10k.json ;; + 2) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4_codec10k.json ;; + 3) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4_codec10k.json ;; *) exit 2 ;; esac cd "$SOURCE" diff --git a/scripts/train/launch_synthetic_manifold_latent4.sbatch b/scripts/train/launch_synthetic_manifold_latent4.sbatch index f04c10d8cc..3d8803c0b1 100644 --- a/scripts/train/launch_synthetic_manifold_latent4.sbatch +++ b/scripts/train/launch_synthetic_manifold_latent4.sbatch @@ -18,8 +18,8 @@ PYTHON=/coc/flash7/paphiwetsa3/projects/EgoVerse7/.venv/bin/python test "$(git -C "$SOURCE" rev-parse HEAD)" = "$EXPECTED_SOURCE_COMMIT" test -z "$(git -C "$SOURCE" status --porcelain)" case "$SLURM_ARRAY_TASK_ID" in - 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/unite_gaussian_torus_latent4.json ;; - 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/sim/synthetic/vfm_gaussian_torus_latent4.json ;; + 0) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/unite_gaussian_torus_latent4.json ;; + 1) CONFIG=$SOURCE/egomimic/hydra_configs/experiment/arc_sim/synthetic/vfm_gaussian_torus_latent4.json ;; *) exit 2 ;; esac cd "$SOURCE" diff --git a/scripts/train/libero_native_launch_contract.py b/scripts/train/libero_native_launch_contract.py index c16c3fd297..6926329eaf 100644 --- a/scripts/train/libero_native_launch_contract.py +++ b/scripts/train/libero_native_launch_contract.py @@ -8,7 +8,7 @@ import sys from pathlib import Path -PROFILE = "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" +PROFILE = "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" def sha(path): @@ -890,7 +890,7 @@ def main(): ) profile_name = inputs.get("profile_name", SPATIAL_HAND_JOINT_PROFILE.name) - if os.environ.get("AF_EXPERIMENT") != "sim/libero/" + profile_name: + if os.environ.get("AF_EXPERIMENT") != "arc_sim/libero/" + profile_name: raise ValueError("joint exact factory experiment/environment mismatch") for key, envkey in { "source_root": "AF_REPO", diff --git a/scripts/train/planar_v2_dp_cotrain_clean_skynet_v1.sbatch b/scripts/train/planar_v2_dp_cotrain_clean_skynet_v1.sbatch index a8a8364c60..0b4ea84ccb 100644 --- a/scripts/train/planar_v2_dp_cotrain_clean_skynet_v1.sbatch +++ b/scripts/train/planar_v2_dp_cotrain_clean_skynet_v1.sbatch @@ -41,7 +41,7 @@ CHAIN_DATA=/coc/flash7/paphiwetsa3/datasets/Tsim_v2/chain_gripper_3000_v2 EXP_ROOT=/coc/flash7/paphiwetsa3/experiments/planar_v2_dp_cotrain_clean_20260903 WORLD_SIZE=2 PARAMETER_COUNT=51506981 -EXPERIMENT=sim/pusht/planar_v2_cotrain_clean_dp_standard +EXPERIMENT=arc_sim/pusht/planar_v2_cotrain_clean_dp_standard WANDB_ID=planar-v2-dp-cotrain-clean-smoke-${SLURM_JOB_ID} SLURM_BIN=/opt/slurm/Ubuntu-20.04/24.11.0/bin export PATH=$SLURM_BIN:$PATH diff --git a/scripts/train/standard_dp_launch.py b/scripts/train/standard_dp_launch.py index 64162a3111..9233d2910a 100644 --- a/scripts/train/standard_dp_launch.py +++ b/scripts/train/standard_dp_launch.py @@ -61,8 +61,8 @@ def recipe(): "supported DP_COTRAIN_STANDARD profile. Historical runs keep their recorded source." ) if multiplier(): - return "sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier" - return "sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked" + return "arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier" + return "arc_sim/pusht/planar_usocket_chain_manual4919_standard_dp_retimed_masked" def layout(): diff --git a/scripts/train/validate_multiplier_config.py b/scripts/train/validate_multiplier_config.py index 855d5e533a..92eea47ec7 100644 --- a/scripts/train/validate_multiplier_config.py +++ b/scripts/train/validate_multiplier_config.py @@ -14,7 +14,7 @@ from hydra.core.hydra_config import HydraConfig from omegaconf import OmegaConf, open_dict -EXPERIMENT = "sim/pusht/action_flow_cotrain_uc_multiplier_interpolation" +EXPERIMENT = "arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation" SOURCE = "344dd3baf5ba942d947e0a0e136c8990025d91e5" REFERENCE = 84.11600368466028 ROUTES = ("pushshapes_sim_u_socket", "pushshapes_sim_chain_gripper") diff --git a/scripts/train/verify_action_flow_training_smoke.py b/scripts/train/verify_action_flow_training_smoke.py index 947714e3a8..15c11df87a 100644 --- a/scripts/train/verify_action_flow_training_smoke.py +++ b/scripts/train/verify_action_flow_training_smoke.py @@ -66,10 +66,10 @@ CODEC98K_EXPERIMENT = "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_codec98k_s42" CODEC98K_PARAMETER_COUNT = 50_801_685 SCALED_MUON_EXPERIMENT = ( - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42" + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42" ) SCALED_ADAMW_EXPERIMENT = ( - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42" + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42" ) SCALED_200M_EXPERIMENTS = {SCALED_MUON_EXPERIMENT, SCALED_ADAMW_EXPERIMENT} SCALED_200M_PARAMETER_COUNT = 199_754_837 @@ -109,7 +109,7 @@ 1.0, 1.0, ), - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": ( "action_flow_bc_usocket_latent_fm_sg_recon1_s42", 1.0, 1.0, @@ -129,27 +129,27 @@ 1.0, 1.0, ), - "sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42": ( "action_flow_bc_usocket_bridge_likelihood_s42", 0.0, 0.0, ), - "sim/pusht/action_flow_bc_usocket_graph_section_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_graph_section_s42": ( "action_flow_bc_usocket_graph_section_s42", 0.0, 1.0, ), - "sim/pusht/action_flow_bc_usocket_recon1_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_recon1_s42": ( "action_flow_bc_usocket_recon1_s42", 1.0, 1.0, ), - "sim/pusht/action_flow_bc_usocket_recon10_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_recon10_s42": ( "action_flow_bc_usocket_recon10_s42", 10.0, 1.0, ), - "sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_recon10_warmup10k_s42": ( "action_flow_bc_usocket_recon10_warmup10k_s42", 10.0, 1.0, @@ -159,7 +159,7 @@ 10.0, 0.01, ), - "sim/pusht/action_flow_bc_usocket_recon100_s42": ( + "arc_sim/pusht/action_flow_bc_usocket_recon100_s42": ( "action_flow_bc_usocket_recon100_s42", 100.0, 1.0, @@ -2456,8 +2456,8 @@ def verify_smoke( expected_preflight_sha256: str | None = None, ) -> dict[str, Any]: if experiment in { - "sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", - "sim/libero/action_flow_libero_spatial_sharpa_mot_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42", }: from scripts.train.verify_libero_native_action_flow_smoke import ( verify_joint_smoke, @@ -2473,14 +2473,14 @@ def verify_smoke( shared=sys.modules[__name__], ) if experiment in ( - "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", ): from scripts.train.verify_libero_native_action_flow_smoke import ( verify_native_smoke, @@ -2639,16 +2639,16 @@ def _parser() -> argparse.ArgumentParser: dest="experiment", choices=( *tuple(APPROVED_EXPERIMENTS), - "sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", - "sim/libero/action_flow_libero_spatial_sharpa_mot_s42", - "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", - "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", - "sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42", + "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_spatial_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_goal_h240_euler50_dithalf_80k_av0_s42", + "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_s42", + "arc_sim/libero_historical/action_flow_libero_object_h240_euler50_dithalf_80k_av0_s42", ), required=True, ) @@ -2675,8 +2675,8 @@ def main(argv: Sequence[str] | None = None) -> int: _require( args.experiment in { - "sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", - "sim/libero/action_flow_libero_spatial_sharpa_mot_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_mot_s42", }, "separate verifier source requires explicit joint route", ) diff --git a/scripts/train/verify_libero_native_action_flow_smoke.py b/scripts/train/verify_libero_native_action_flow_smoke.py index 95b3a264cf..03e8c52976 100644 --- a/scripts/train/verify_libero_native_action_flow_smoke.py +++ b/scripts/train/verify_libero_native_action_flow_smoke.py @@ -8,7 +8,7 @@ import json from pathlib import Path -PROFILE = "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" +PROFILE = "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" PARAMETER_COUNT = 39750391 SOURCE = "libero_panda" diff --git a/scripts/train/verify_scaled_action_flow_training_smoke.py b/scripts/train/verify_scaled_action_flow_training_smoke.py index 84f46b84fc..1db4437282 100755 --- a/scripts/train/verify_scaled_action_flow_training_smoke.py +++ b/scripts/train/verify_scaled_action_flow_training_smoke.py @@ -19,7 +19,7 @@ SCHEMA_VERSION = 1 EXPERIMENTS = { - "sim/pusht/action_flow_cotrain_uc_multiplier_interpolation": { + "arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation": { "config_name": "action_flow_cotrain_uc_multiplier_interpolation", "parameter_count": 471_549_402, "sources": { diff --git a/tests/test_abc_arc_experiment_layout.py b/tests/test_abc_arc_experiment_layout.py index cbf6008685..93abb0f221 100644 --- a/tests/test_abc_arc_experiment_layout.py +++ b/tests/test_abc_arc_experiment_layout.py @@ -10,7 +10,7 @@ ROOT = Path(__file__).resolve().parents[1] REAL_LEAVES = sorted( - f"real/{task}_{mode}" + f"arc_real/{task}_{mode}" for task in ("stationary", "earbuds", "towels") for mode in ("time", "duration", "velocity") ) @@ -23,13 +23,13 @@ def test_real_is_exactly_the_nine_task_leaves(): found = sorted( p.relative_to(CONFIGS / "experiment").with_suffix("").as_posix() - for p in (CONFIGS / "experiment/real").rglob("*.yaml") + for p in (CONFIGS / "experiment/arc_real").rglob("*.yaml") ) assert found == REAL_LEAVES def test_retired_menus_are_gone(): - for folder in ("abc_arc/cotrain", "yam_arc_grid", "real/arcdur_abstime"): + for folder in ("abc_arc/cotrain", "yam_arc_grid", "arc_real/arcdur_abstime"): assert not list((CONFIGS / "experiment" / folder).rglob("*.yaml")) pools = {p.stem for p in (CONFIGS / "data/abc_arc").glob("*.yaml")} assert {"stationary", "earbuds", "towels", "stationery_yam_loader"} <= pools @@ -37,11 +37,9 @@ def test_retired_menus_are_gone(): p for p in pools if p.startswith(("stationery_s", "earbuds_", "towels_", "organize_")) + and not p.endswith("_arc") } - shared = { - p.stem for p in (CONFIGS / "experiment/_shared/frozen_yam").glob("*.yaml") - } - assert shared == {"base", "stattempo", "stattempo_openloop", "hpt300_openloop"} + assert not list((CONFIGS / "experiment/_shared").rglob("*.yaml")) def test_migration_targets_exist_or_are_retired(): @@ -53,7 +51,7 @@ def test_migration_targets_exist_or_are_retired(): (CONFIGS / f"experiment/{target}{suffix}").is_file() for suffix in (".yaml", ".json") ), target - assert sum(v is None for v in mapping.values()) == 18 + assert sum(v is None for v in mapping.values()) >= 18 @pytest.mark.parametrize("leaf", REAL_LEAVES + ABC_LEAVES) diff --git a/tests/test_abc_yam.py b/tests/test_abc_yam.py index 51f5353d54..a50b475478 100644 --- a/tests/test_abc_yam.py +++ b/tests/test_abc_yam.py @@ -34,6 +34,8 @@ ) config_graph = importlib.util.module_from_spec(_SPEC) _SPEC.loader.exec_module(config_graph) +_MODEL_YAML = _REPO / "egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml" +_GRAPH_OVERRIDES = ["data=abc_arc/abc_fstshirt_bc", "~evaluator"] # -- embodiment registration ------------------------------------------------ @@ -192,18 +194,17 @@ def test_action_target_builder_is_excluded_from_the_inference_graph(): @pytest.mark.parametrize("mode", ["train", "inference"]) -def test_abc_experiment_graph_lints_clean(mode): - graph = config_graph.build_graph( - _REPO / "egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml", mode=mode - ) +def test_abc_model_graph_lints_clean(mode): + graph = config_graph.build_graph(_MODEL_YAML, mode=mode, overrides=_GRAPH_OVERRIDES) assert graph["lint"] == [] assert graph["seed_keys_by_source"]["yam_bimanual"] def test_abc_train_graph_has_the_five_dp_stages_and_a_loss(): graph = config_graph.build_graph( - _REPO / "egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml", + _MODEL_YAML, mode="train", + overrides=_GRAPH_OVERRIDES, ) assert [node["t"] for node in graph["nodes"]] == [ "FusedObsEncoder", @@ -219,8 +220,9 @@ def test_abc_train_graph_has_the_five_dp_stages_and_a_loss(): def test_abc_inference_graph_drops_the_training_only_stages(): graph = config_graph.build_graph( - _REPO / "egomimic/hydra_configs/experiment/abc/yam_fstshirt_dp.yaml", + _MODEL_YAML, mode="inference", + overrides=_GRAPH_OVERRIDES, ) skipped = {entry["t"] for entry in graph["skipped_stages"]} assert skipped == { diff --git a/tests/test_action_flow_av_norm.py b/tests/test_action_flow_av_norm.py index 80f8e26275..2de0d7b38c 100644 --- a/tests/test_action_flow_av_norm.py +++ b/tests/test_action_flow_av_norm.py @@ -168,7 +168,7 @@ def test_cotrain_av_norm_is_opt_in_and_hydra_reachable(monkeypatch): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/pusht/parents/action_flow_cotrain_uc_manual4919", + "+experiment=arc_sim/pusht/parents/action_flow_cotrain_uc_manual4919", "callbacks=action_flow_avln_cotrain", "hydra/launcher=basic", "++paths.root_dir=.", diff --git a/tests/test_action_flow_configs.py b/tests/test_action_flow_configs.py index 73162d366d..7083f1b0fb 100644 --- a/tests/test_action_flow_configs.py +++ b/tests/test_action_flow_configs.py @@ -18,7 +18,7 @@ def _compose(row: str): with initialize_config_dir(version_base=None, config_dir=str(CONFIG_DIR.resolve())): cfg = compose( config_name="train_zarr_cartesian", - overrides=[f"+experiment=sim/pusht/{row}", "++paths.root_dir=."], + overrides=[f"+experiment=arc_sim/pusht/{row}", "++paths.root_dir=."], ) return cfg diff --git a/tests/test_action_flow_likelihood.py b/tests/test_action_flow_likelihood.py index 751f277825..a5da812549 100644 --- a/tests/test_action_flow_likelihood.py +++ b/tests/test_action_flow_likelihood.py @@ -401,7 +401,7 @@ def test_full_hydra_likelihood_graph_has_unblocked_inference_plan(): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42", + "+experiment=arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42", "++paths.root_dir=.", ], ) diff --git a/tests/test_action_latent_vfm.py b/tests/test_action_latent_vfm.py index 7a1cc35183..28b7edafa9 100644 --- a/tests/test_action_latent_vfm.py +++ b/tests/test_action_latent_vfm.py @@ -29,7 +29,7 @@ ROOT = Path(__file__).resolve().parents[1] EXPERIMENT = ( ROOT - / "egomimic/hydra_configs/experiment/sim/pusht/action_latent_vfm_usocket_val01_h16.yaml" + / "egomimic/hydra_configs/experiment/arc_sim/pusht/action_latent_vfm_usocket_val01_h16.yaml" ) diff --git a/tests/test_arc_clock_contract.py b/tests/test_arc_clock_contract.py index f0f969d6ed..e6d2bc26b0 100644 --- a/tests/test_arc_clock_contract.py +++ b/tests/test_arc_clock_contract.py @@ -264,7 +264,7 @@ def test_recipe_data_evaluator_and_declared_rollout_share_contract(clock, stream recipe = ( CONFIGS - / "experiment/real" + / "experiment/arc_real" / ( "stationary_velocity.yaml" if clock == "velocity" diff --git a/tests/test_config_cleanup_layout.py b/tests/test_config_cleanup_layout.py new file mode 100644 index 0000000000..618a110785 --- /dev/null +++ b/tests/test_config_cleanup_layout.py @@ -0,0 +1,59 @@ +"""Protect the compact config menu without removing distinct protocols.""" + +import json + +from omegaconf import OmegaConf + +from scripts.audit_hydra_configs import CONFIGS + + +def test_deleted_experiment_menus_stay_deleted(): + for folder in ( + "abc", + "abc_arc/human_bc", + "arc_real/arcdur_abstime", + "real", + "sim", + "_shared", + ): + assert not list((CONFIGS / "experiment" / folder).rglob("*.yaml")) + + +def test_real_recipes_never_inherit_another_experiment(): + paths = list((CONFIGS / "experiment/arc_real").rglob("*.yaml")) + assert len(paths) == 9 + for path in paths: + config = OmegaConf.to_container(OmegaConf.load(path), resolve=False) + assert all( + not isinstance(entry, str) or not entry.startswith("/experiment/") + for entry in config["defaults"] + ), path + + +def test_models_are_grouped_and_have_reusable_bases(): + assert not list((CONFIGS / "model").glob("*.yaml")) + for family in ("base", "hpt", "pi"): + assert list((CONFIGS / "model" / family).glob("*.yaml")) + assert not list((CONFIGS / "model/pi05").glob("*.yaml")) + + +def test_evaluator_names_describe_behavior_not_models(): + for path in (CONFIGS / "evaluator").rglob("*.yaml"): + assert not path.stem.startswith(("eval_hpt", "eval_pi")) + + +def test_every_selector_migration_points_to_a_retained_config(): + receipt = CONFIGS.parents[1] / "docs/config_cleanup_migration.json" + mapping = json.loads(receipt.read_text())["renamed"] + for old, new in mapping.items(): + assert not (CONFIGS / f"{old}.yaml").exists() + assert (CONFIGS / f"{new}.yaml").is_file() + + +def test_no_identical_yaml_config_bodies(): + bodies = {} + for path in sorted(CONFIGS.rglob("*.yaml")): + value = OmegaConf.to_container(OmegaConf.load(path), resolve=False) + body = json.dumps(value, sort_keys=True) + assert body not in bodies, f"Duplicate configs: {bodies.get(body)}, {path}" + bodies[body] = path diff --git a/tests/test_consolidated_core_configs.py b/tests/test_consolidated_core_configs.py index d3dd7995ef..65d5f16634 100644 --- a/tests/test_consolidated_core_configs.py +++ b/tests/test_consolidated_core_configs.py @@ -10,7 +10,11 @@ def _compose(row, *extra): with initialize_config_dir(version_base=None, config_dir=str(CONFIG_DIR.resolve())): return compose( config_name="train_zarr_cartesian", - overrides=[f"+experiment=sim/pusht/{row}", "++paths.root_dir=.", *extra], + overrides=[ + f"+experiment=arc_sim/pusht/{row}", + "++paths.root_dir=.", + *extra, + ], ) diff --git a/tests/test_deployment_data_requirements.py b/tests/test_deployment_data_requirements.py index cec95c54d2..dea2c1bef1 100644 --- a/tests/test_deployment_data_requirements.py +++ b/tests/test_deployment_data_requirements.py @@ -15,24 +15,21 @@ from scripts.audit_hydra_configs import CONFIGS, compose_for_audit PI_CASES = [ - ("pi0.5_bc_eva", "eva_pi_lang"), - ("pi0.5_bc_abc_eva_6d", "pi05/abc_eva_pi_6d"), - ("pi0.5_ft_abc_eva_6d", "pi05/abc_eva_pi_6d"), - ("pi0.5_bc_mecka_6d", "pi05/mecka_all_pi_6d"), - ("pi0.5_bc_mecka_abc_6d", "pi05/mecka_abc_pi_6d"), - ("pi0.5_cotrain_eva_aria_6d", "pi05/cotrain_pi_lang_6d"), - ("pi0.5_ft_aria_fold_grip", "pi05/aria_fold_pi_6d_grip"), + ("pi/bc_eva_legacy", "eva_pi_lang"), + ("pi/bc_abc_eva_6d", "pi05/abc_eva_pi_6d"), + ("pi/bc_mecka_6d", "pi05/mecka_all_pi_6d"), + ("pi/bc_mecka_abc_6d", "pi05/mecka_abc_pi_6d"), + ("pi/cotrain_eva_aria_6d", "pi05/cotrain_pi_lang_6d"), + ("pi/ft_aria_fold_grip", "pi05/aria_fold_pi_6d_grip"), ] EXPERIMENTS = [ - "abc/yam_fstshirt_dp", - "abc/yam_fstshirt_hpt", "abc_arc/robot_bc/abc_towels_hpt180_baseline_visual_openloop", "abc_arc/robot_bc/multitask/arc_dur", - "real/stationary_time", - "real/stationary_velocity", - "real/stationary_duration", - "real/earbuds_duration", - "real/towels_duration", + "arc_real/stationary_time", + "arc_real/stationary_velocity", + "arc_real/stationary_duration", + "arc_real/earbuds_duration", + "arc_real/towels_duration", ] SOURCE_IDENTITIES = {"human_bimanual": "3", "eva_bimanual": "6", "yam_bimanual": "7"} @@ -83,7 +80,7 @@ def test_pi_selected_frame_rotation_and_stride(model, data): cfg = compose( config_name="train_zarr_cartesian_pi", overrides=[ - f"model=pi05/{model}", + f"model={model}", f"data={data}", "pi05.pretrained_weights=/unused/no-weight-download", ], @@ -97,6 +94,16 @@ def test_yam_arc_and_tempo_selected_contracts(experiment): check_selected_contract(cfg) +@pytest.mark.parametrize("model", ["abc/yam_bimanual_dp", "hpt/yam_bimanual"]) +def test_reusable_yam_model_data_contracts(model): + with initialize_config_dir(version_base=None, config_dir=str(CONFIGS)): + cfg = compose( + config_name="train_zarr_cartesian", + overrides=[f"model={model}", "data=abc_arc/abc_fstshirt_bc"], + ) + check_selected_contract(cfg) + + def test_every_shipped_ready_model_declares_data_semantics(): for path in sorted((CONFIGS / "model").rglob("*.yaml")): with compose_for_audit(path) as cfg: diff --git a/tests/test_dp_action_flow_observation_parity.py b/tests/test_dp_action_flow_observation_parity.py index c4eb511fb8..a81b573594 100644 --- a/tests/test_dp_action_flow_observation_parity.py +++ b/tests/test_dp_action_flow_observation_parity.py @@ -33,7 +33,7 @@ def config(monkeypatch): config_name="train_zarr_cartesian", overrides=[ "hydra/launcher=basic", - "+experiment=sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier", + "+experiment=arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier", ], ) diff --git a/tests/test_evaluator_config_cleanup.py b/tests/test_evaluator_config_cleanup.py new file mode 100644 index 0000000000..b0b71b771c --- /dev/null +++ b/tests/test_evaluator_config_cleanup.py @@ -0,0 +1,88 @@ +"""Evaluator names describe predictions/frames, not a policy architecture.""" + +from pathlib import Path + +import pytest +from hydra import compose, initialize_config_dir +from hydra.utils import instantiate +from omegaconf import OmegaConf, open_dict + +CONFIGS = Path(__file__).resolve().parents[1] / "egomimic/hydra_configs" + + +def evaluator_config(name): + with initialize_config_dir(version_base=None, config_dir=str(CONFIGS)): + cfg = compose(config_name=None, overrides=[f"+evaluator={name}"]) + with open_dict(cfg): + cfg.paths = {"output_dir": "/tmp/evaluator-config-test"} + cfg.model = { + "pipeline": {"stages": [{"policy": {"config": {"reverse_kl_samples": 8}}}]} + } + return cfg.evaluator + + +def test_model_named_evaluator_and_renderer_aliases_are_removed(): + root = CONFIGS / "evaluator" + assert not list(root.glob("eval_hpt*.yaml")) + assert not list(root.glob("eval_pi*.yaml")) + assert not list((root / "pi05").glob("*.yaml")) + assert not list((root / "viz").glob("pi_*.yaml")) + for name in ("cartesian_wrist", "cartesian_wrist_lang", "cotrain_lang"): + assert not (root / "viz" / f"{name}.yaml").exists() + + +def test_open_loop_and_chunk_metrics_share_native_pose_bindings(): + execution = evaluator_config("eval_open_loop_sim") + chunks = evaluator_config("eval_bimanual_cartesian") + for key in ( + "action_key", + "obs_pose_key", + "image_key", + "revert_transforms", + "viz_func", + "video_chunk_frames", + "max_episode_frames", + ): + assert OmegaConf.to_container(OmegaConf.create({key: execution[key]})) == ( + OmegaConf.to_container(OmegaConf.create({key: chunks[key]})) + ) + evaluator = instantiate(execution) + assert evaluator.distance_dtw_enabled + assert evaluator.execute_steps == 30 + assert evaluator.data_requirements().complete_episodes + assert evaluator.data_requirements().ordered + assert evaluator.arc_execution_cap_mode == "waypoints" + + +@pytest.mark.parametrize( + "name,suffix,max_batches", + [ + ( + "eval_cartesian_wrist_prompt", + "cartesian_revert_eef_frame_transform_list", + None, + ), + ("eval_cartesian_camera_6d_prompt", "cartesian_revert_6d_transform_list", None), + ( + "eval_cartesian_wrist_6d_prompt", + "cartesian_revert_6d_wristframe_transform_list", + 10, + ), + ( + "eval_cartesian_wrist_6d_gripper_prompt", + "cartesian_revert_6d_wristframe_grip_transform_list", + 10, + ), + ], +) +def test_prompt_presets_preserve_frame_and_sampling_contract(name, suffix, max_batches): + cfg = evaluator_config(name) + assert cfg.revert_transforms.human_bimanual._target_.endswith(suffix) + assert cfg.viz_every_n_epochs == 100 + assert cfg.get("viz_max_batches") == max_batches + assert cfg.rkl_samples == 8 + assert cfg.group_options.train_viz.limit_batches == 50 + assert cfg.group_options.train_viz.rkl_samples == 1 + assert cfg.viz_func.human_bimanual.annotation_key == "sampled_prompt" + assert cfg.viz_func.eva_bimanual.fallback_intrinsics[0][0] == 436.26 + instantiate(cfg) diff --git a/tests/test_hpt_configs.py b/tests/test_hpt_configs.py index ac5cd20916..1353b1afb7 100644 --- a/tests/test_hpt_configs.py +++ b/tests/test_hpt_configs.py @@ -6,11 +6,13 @@ import hydra import pytest import torch +import torchvision from hydra import compose, initialize_config_dir _REPO = Path(__file__).resolve().parents[1] _CONFIGS = _REPO / "egomimic/hydra_configs" -_EXPERIMENT = "abc/yam_fstshirt_hpt" +_MODEL = "hpt/yam_bimanual" +_DATA = "abc_arc/abc_fstshirt_bc" _SPEC = importlib.util.spec_from_file_location( "config_graph_hpt", _REPO / "tools/config_graph.py" @@ -18,14 +20,29 @@ config_graph = importlib.util.module_from_spec(_SPEC) _SPEC.loader.exec_module(config_graph) -_EXPERIMENT_YAML = _CONFIGS / "experiment" / "abc" / "yam_fstshirt_hpt.yaml" +_MODEL_YAML = _CONFIGS / "model" / "hpt" / "yam_bimanual.yaml" +_GRAPH_OVERRIDES = [f"data={_DATA}", "~evaluator"] + + +@pytest.fixture(scope="module", autouse=True) +def offline_resnet_weights(): + """Exercise the real architecture without downloading ImageNet weights.""" + real_resnet18 = torchvision.models.resnet18 + + def offline_resnet18(*args, **kwargs): + kwargs["weights"] = None + return real_resnet18(*args, **kwargs) + + with pytest.MonkeyPatch.context() as patch: + patch.setattr(torchvision.models, "resnet18", offline_resnet18) + yield def _cfg(): with initialize_config_dir(version_base=None, config_dir=str(_CONFIGS)): return compose( config_name="train_zarr_cartesian", - overrides=[f"+experiment={_EXPERIMENT}", "++paths.root_dir=."], + overrides=[f"model={_MODEL}", *_GRAPH_OVERRIDES, "++paths.root_dir=."], ) @@ -91,11 +108,18 @@ def test_action_key_matches_what_the_yam_transform_list_emits(cfg): @pytest.mark.parametrize("mode", ["train", "inference"]) def test_the_configured_graph_lints_clean(mode): - assert config_graph.build_graph(_EXPERIMENT_YAML, mode=mode)["lint"] == [] + assert ( + config_graph.build_graph(_MODEL_YAML, mode=mode, overrides=_GRAPH_OVERRIDES)[ + "lint" + ] + == [] + ) def test_inference_drops_the_training_only_stages(): - graph = config_graph.build_graph(_EXPERIMENT_YAML, mode="inference") + graph = config_graph.build_graph( + _MODEL_YAML, mode="inference", overrides=_GRAPH_OVERRIDES + ) assert {e["t"] for e in graph["skipped_stages"]} == { "ActionTargetBuilder", "FlowNoisingStage", @@ -106,7 +130,9 @@ def test_inference_drops_the_training_only_stages(): def test_the_encoder_survives_into_inference(): """Stems and trunk must NOT be mode-restricted: sampling needs them.""" - graph = config_graph.build_graph(_EXPERIMENT_YAML, mode="inference") + graph = config_graph.build_graph( + _MODEL_YAML, mode="inference", overrides=_GRAPH_OVERRIDES + ) names = [n["t"] for n in graph["nodes"]] assert names[:2] == ["HPTStemStage", "HPTTrunkStage"] @@ -158,11 +184,15 @@ def test_inference_samples_an_action_chunk(cfg, algo): def test_the_dp_counterpart_still_uses_diffusion(cfg): - """The two abc experiments must stay distinct architectures.""" + """The two YAM model configurations must stay distinct architectures.""" with initialize_config_dir(version_base=None, config_dir=str(_CONFIGS)): dp = compose( config_name="train_zarr_cartesian", - overrides=["+experiment=abc/yam_fstshirt_dp", "++paths.root_dir=."], + overrides=[ + "model=abc/yam_bimanual_dp", + *_GRAPH_OVERRIDES, + "++paths.root_dir=.", + ], ) dp_stages = [s._target_.rsplit(".", 1)[-1] for s in dp.model.pipeline.stages] hpt_stages = [s._target_.rsplit(".", 1)[-1] for s in cfg.model.pipeline.stages] diff --git a/tests/test_inference_contract_boundaries.py b/tests/test_inference_contract_boundaries.py index 4f65b6b4eb..2aa83a8ed0 100644 --- a/tests/test_inference_contract_boundaries.py +++ b/tests/test_inference_contract_boundaries.py @@ -17,6 +17,11 @@ from egomimic.robot.graph_policy import configure_profile_controls CONFIGS = Path(__file__).parents[1] / "egomimic/hydra_configs" +YAM_DP_OVERRIDES = [ + "model=abc/yam_bimanual_dp", + "data=abc_arc/abc_fstshirt_bc", + "~evaluator", +] def compose_recipe(**kwargs): @@ -27,7 +32,7 @@ def compose_recipe(**kwargs): def test_resampled_yam_contract_restores_native_duration(): cfg = compose_recipe( config_name="train_zarr_cartesian", - overrides=["+experiment=abc/yam_fstshirt_dp"], + overrides=YAM_DP_OVERRIDES, ) artifact = build_inference_config(cfg) assert artifact["status"] == "ready" @@ -47,7 +52,7 @@ def test_chunk_mean_timing_is_non_deployable_even_with_same_model_shape(): cfg = compose_recipe( config_name="train_zarr_cartesian", overrides=[ - "+experiment=abc/yam_fstshirt_dp", + *YAM_DP_OVERRIDES, "model.inference.native_output.timing.kind=chunk_mean", "model.inference.native_output.timing.temporally_resolved=false", ], @@ -70,7 +75,7 @@ def test_pi_mixed_source_contract_selects_exact_width_and_time( cfg = compose_recipe( config_name="train_zarr_cartesian_pi", overrides=[ - "model=pi05/pi0.5_bc_mecka_abc_6d", + "model=pi/bc_mecka_abc_6d", "pi05.pretrained_weights=/unused/weights", f"model.deployment_domain={domain}", ], diff --git a/tests/test_libero_action_flow_boundary.py b/tests/test_libero_action_flow_boundary.py index 4526ab2fcf..81319a2776 100644 --- a/tests/test_libero_action_flow_boundary.py +++ b/tests/test_libero_action_flow_boundary.py @@ -59,7 +59,7 @@ def test_libero_action_flow_recipe_selects_effective_train_and_validation_data( cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/libero_historical/action_flow_libero10_avln_80k_s42" + "+experiment=arc_sim/libero_historical/action_flow_libero10_avln_80k_s42" ], ) assert "data" not in cfg.data diff --git a/tests/test_libero_arc_global.py b/tests/test_libero_arc_global.py index a7eba89ed6..24c4e6b2af 100644 --- a/tests/test_libero_arc_global.py +++ b/tests/test_libero_arc_global.py @@ -197,7 +197,7 @@ def test_matched_configs_and_real_network_dimensions(): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - f"+experiment=sim/oat/libero_arc_global_{basis}", + f"+experiment=arc_sim/oat/libero_arc_global_{basis}", "benchmark.dataset=/unused-for-construction", "hydra/launcher=basic", ], diff --git a/tests/test_libero_benchmark.py b/tests/test_libero_benchmark.py index dc7bf93854..bf57014435 100644 --- a/tests/test_libero_benchmark.py +++ b/tests/test_libero_benchmark.py @@ -276,7 +276,7 @@ def test_complete_benchmark_recipes_compose(experiment): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - f"+experiment=sim/oat/{experiment}", + f"+experiment=arc_sim/oat/{experiment}", f"benchmark.suite={suite}", "benchmark.dataset=/tmp/libero.zarr", ], diff --git a/tests/test_mot_action_codec.py b/tests/test_mot_action_codec.py index d0650e6ab4..25e468a079 100644 --- a/tests/test_mot_action_codec.py +++ b/tests/test_mot_action_codec.py @@ -413,7 +413,7 @@ def test_phase_cli_rejects_wrong_experiment_before_model_or_composition( monkeypatch.setenv("SLURM_STEP_ID", "0") monkeypatch.setenv( "AF_EXPERIMENT", - "sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", + "arc_sim/libero/action_flow_libero_spatial_sharpa_gripper_keypoints_s42", ) monkeypatch.setattr( sys, diff --git a/tests/test_oat_diffusion.py b/tests/test_oat_diffusion.py index a5567968ad..8e64620b5e 100644 --- a/tests/test_oat_diffusion.py +++ b/tests/test_oat_diffusion.py @@ -106,7 +106,7 @@ def test_released_config_and_parameter_counts(reference): ): cfg = compose( config_name="train_zarr_cartesian", - overrides=["+experiment=sim/oat/libero_arc_oat_dp_policy"], + overrides=["+experiment=arc_sim/oat/libero_arc_oat_dp_policy"], ) denoiser = cfg.model.pipeline.stages[3].policy model = denoiser.model @@ -187,10 +187,10 @@ def test_training_launcher_selects_backbone_and_keeps_legacy_default(): args = ("arc_stk", "libero_spatial", "data.zarr", "out", "full", 5001) matched = training_arguments(*args, gpus=4, arc_backbone="oat_dp") - assert "+experiment=sim/oat/libero_arc_oat_dp_policy" in matched + assert "+experiment=arc_sim/oat/libero_arc_oat_dp_policy" in matched assert "trainer.devices=4" in matched assert "benchmark.batch_size=256" in matched assert "trainer.accumulate_grad_batches=1" in matched - assert "+experiment=sim/oat/libero_arc_stk_policy" in training_arguments(*args) + assert "+experiment=arc_sim/oat/libero_arc_stk_policy" in training_arguments(*args) with pytest.raises(ValueError, match="Unknown ARC backbone"): training_arguments(*args, arc_backbone="invalid") diff --git a/tests/test_oat_training.py b/tests/test_oat_training.py index 5aab85a243..512be1180f 100644 --- a/tests/test_oat_training.py +++ b/tests/test_oat_training.py @@ -16,7 +16,7 @@ def config_for(experiment, path, output): return compose( config_name="train_zarr_cartesian", overrides=[ - f"+experiment=sim/oat/{experiment}", + f"+experiment=arc_sim/oat/{experiment}", f"benchmark.dataset={path}", "benchmark.batch_size=2", "benchmark.horizon=8", diff --git a/tests/test_pi05_graph.py b/tests/test_pi05_graph.py index 1e2401fedc..30d346d65c 100644 --- a/tests/test_pi05_graph.py +++ b/tests/test_pi05_graph.py @@ -21,6 +21,12 @@ from egomimic.rldb.zarr.zarr_dataset_multi import MultiDataset CONFIGS = Path(__file__).parents[1] / "egomimic/hydra_configs" +PI_EVALUATORS = ( + "eval_cartesian_wrist_prompt", + "eval_cartesian_camera_6d_prompt", + "eval_cartesian_wrist_6d_prompt", + "eval_cartesian_wrist_6d_gripper_prompt", +) def config(overrides=()): @@ -34,13 +40,18 @@ def config(overrides=()): @pytest.mark.parametrize( "group,name", [ - (group, path.stem) - for group in ["model", "data", "evaluator"] - for path in sorted((CONFIGS / group / "pi05").glob("*.yaml")) - ], + ("model", f"pi/{path.stem}") + for path in sorted((CONFIGS / "model/pi").glob("*.yaml")) + if not path.stem.endswith("_legacy") + ] + + [ + ("data", f"pi05/{path.stem}") + for path in sorted((CONFIGS / "data/pi05").glob("*.yaml")) + ] + + [("evaluator", name) for name in PI_EVALUATORS], ) def test_pi_configs_resolve_without_openpi(group, name): - cfg = config([f"{group}=pi05/{name}"]) + cfg = config([f"{group}={name}"]) instantiate(cfg.model.pipeline) # backend remains deferred instantiate(cfg.evaluator) for ds in cfg.data.train_datasets.values(): @@ -50,6 +61,19 @@ def test_pi_configs_resolve_without_openpi(group, name): instantiate(ds.resolver.transform_list) +def test_modern_and_legacy_pi_eva_keep_distinct_encodings(): + modern = config(["model=pi/bc_eva"]) + legacy = config(["model=pi/bc_eva_legacy"]) + assert ( + modern.model.pipeline.stages[0].policy.action_encoding + == "cartesian_ypr_raw_rot6d" + ) + assert ( + legacy.model.pipeline.stages[0].policy.action_encoding + == "legacy_normalized_ypr_rot6d" + ) + + @pytest.fixture def tiny_openpi(monkeypatch): modules = {} diff --git a/tests/test_pipeline_arc.py b/tests/test_pipeline_arc.py index c0a367b8fd..a62f4bd041 100644 --- a/tests/test_pipeline_arc.py +++ b/tests/test_pipeline_arc.py @@ -18,7 +18,7 @@ _REPO = Path(__file__).resolve().parents[1] _EXPERIMENT = ( _REPO - / "egomimic/hydra_configs/experiment/sim/pusht/planar_v2_usocket_arc_graph_tok.yaml" + / "egomimic/hydra_configs/experiment/arc_sim/pusht/planar_v2_usocket_arc_graph_tok.yaml" ) _SPEC = importlib.util.spec_from_file_location( "config_graph_arc", _REPO / "tools/config_graph.py" diff --git a/tests/test_planar_configs.py b/tests/test_planar_configs.py index 496ab1bae9..c97a8eba8e 100644 --- a/tests/test_planar_configs.py +++ b/tests/test_planar_configs.py @@ -61,7 +61,10 @@ def test_planar_row_composes_without_pipeline_routing_metadata(row, expected): with initialize_config_dir(version_base=None, config_dir=str(config_dir.resolve())): cfg = compose( config_name="train_zarr_cartesian", - overrides=[f"+experiment=sim/pusht/planar_v2_{row}", "++paths.root_dir=."], + overrides=[ + f"+experiment=arc_sim/pusht/planar_v2_{row}", + "++paths.root_dir=.", + ], ) for section in (cfg.planar, cfg.model, cfg.data): OmegaConf.resolve(section) @@ -166,17 +169,17 @@ def test_dp_rows_inherit_model_neutral_domain_base(row, domain_base): / "egomimic" / "hydra_configs" / "experiment" - / "sim" + / "arc_sim" / "pusht" ) experiment = OmegaConf.load(experiment_dir / f"planar_v2_{row}.yaml") defaults = OmegaConf.to_container(experiment.defaults) - assert defaults[0] == f"/experiment/sim/pusht/planar_v2_{domain_base}" + assert defaults[0] == f"/experiment/arc_sim/pusht/planar_v2_{domain_base}" assert all("direct_bc" not in str(item) for item in defaults) base = OmegaConf.load(experiment_dir / f"planar_v2_{domain_base}.yaml") base_defaults = OmegaConf.to_container(base.defaults) - assert base_defaults[0] == "/experiment/sim/pusht/planar_v2_base" + assert base_defaults[0] == "/experiment/arc_sim/pusht/planar_v2_base" assert base_defaults[1] == { "override /data": f"pusht/planar_v2_{DOMAIN_BASE_DATA[domain_base]}" } @@ -189,7 +192,7 @@ def test_standard_dp_clean_cotrain_composes_both_domains_without_obstacles(): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/pusht/planar_v2_cotrain_clean_dp_standard", + "+experiment=arc_sim/pusht/planar_v2_cotrain_clean_dp_standard", "++paths.root_dir=.", ], ) @@ -237,7 +240,7 @@ def test_duration_arc_explicitly_separates_raw_and_token_horizons(): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", + "+experiment=arc_sim/pusht/planar_v2_usocket_arc_paper_uniform_D40_M16_R24deg", "++paths.root_dir=.", ], ) @@ -256,7 +259,7 @@ def test_ddp_device_count_is_per_node_without_eval_resolver(): cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "+experiment=sim/pusht/planar_v2_cotrain_clean_dp_standard", + "+experiment=arc_sim/pusht/planar_v2_cotrain_clean_dp_standard", "launch_params.gpus_per_node=2", "launch_params.nodes=3", "++paths.root_dir=.", diff --git a/tests/test_real_task_recipes.py b/tests/test_real_task_recipes.py index d52c182d46..bcb31475dd 100644 --- a/tests/test_real_task_recipes.py +++ b/tests/test_real_task_recipes.py @@ -10,7 +10,7 @@ from egomimic.rldb.zarr.arc_length_tokenizer import TokenizeBimanualArcLengthCartesian from scripts.audit_hydra_configs import CONFIGS, compose_for_audit -REAL = CONFIGS / "experiment/real" +REAL = CONFIGS / "experiment/arc_real" TASKS = {"stationary": 0.22, "earbuds": 0.22, "towels": 0.33} MODES = ("time", "duration", "velocity") POOLS = {"stationary": (417, 24), "earbuds": (339, 15), "towels": (462, 17)} @@ -59,9 +59,9 @@ def test_the_three_leaves_of_a_task_share_graph_widths_pool_and_schedule(task): @pytest.mark.parametrize("task", TASKS) -def test_time_leaf_is_the_frozen_yam_time_baseline(task): +def test_time_leaf_is_the_yam_time_baseline(task): with compose_for_audit(REAL / f"{task}_time.yaml") as cfg: - assert cfg.frozen_yam.variant == "time" + assert cfg.yam.variant == "time" assert "abc" not in cfg assert cfg.evaluator.action_mode == "baseline" for split in ("train_datasets", "valid_datasets"): diff --git a/tests/test_retained_hpt_variants.py b/tests/test_retained_hpt_variants.py index 8b1d457763..bdc885d829 100644 --- a/tests/test_retained_hpt_variants.py +++ b/tests/test_retained_hpt_variants.py @@ -22,9 +22,9 @@ @pytest.mark.parametrize( "name", [ - "hpt_cotrain_flow_shared_head", - "hpt_cotrain_flow_seperate_head", - "hpt_cotrain_enc_dec_base", + "hpt/cotrain_flow_shared_head", + "hpt/cotrain_flow_seperate_head", + "base/hpt_cotrain_enc_dec", ], ) def test_both_domains_train_and_decode(name): diff --git a/tests/test_retained_language.py b/tests/test_retained_language.py index 41e7831c04..e41e6887d0 100644 --- a/tests/test_retained_language.py +++ b/tests/test_retained_language.py @@ -47,10 +47,10 @@ def __call__(self, prompts, **kwargs): config = compose( config_name="train_zarr_cartesian", overrides=[ - f"model=hpt_bc_pickplace_qwen_{variant}", + f"model=hpt/bc_pickplace_qwen_{variant}", "data=bc_pickplace_eva_qwen", - "evaluator=eval_hpt", - "evaluator/viz@evaluator.viz_func=cotrain_lang", + "evaluator=eval_cartesian_camera", + "evaluator/viz@evaluator.viz_func=cartesian_annotations", ], ) for index in range(3): diff --git a/tests/test_retained_pi_parity.py b/tests/test_retained_pi_parity.py index 9b3133336a..6e3b948b4d 100644 --- a/tests/test_retained_pi_parity.py +++ b/tests/test_retained_pi_parity.py @@ -39,7 +39,7 @@ def test_retained_pi_schedules_match_pinned_source_and_base_fails_closed(): value = OmegaConf.to_container(value, resolve=True) assert value == original[field] artifact = build_inference_config(cfg) - if name == "pi0.5_base": + if name == "base/pi05": assert artifact["status"] == "unsupported" with pytest.raises(Exception, match="incomplete base fragment"): instantiate(cfg.model.pipeline) @@ -54,9 +54,9 @@ def test_cotrain_keeps_normalization_isolated_and_renders_prompt( cfg = compose( config_name="train_zarr_cartesian", overrides=[ - "model=pi0.5_cotrain_eva_aria", + "model=pi/cotrain_eva_aria_legacy", "data=cotrain_pi_base", - "evaluator=eval_pi", + "evaluator=eval_cartesian_camera_prompt", ], ) datasets = {} diff --git a/tests/test_retained_recipe_steps.py b/tests/test_retained_recipe_steps.py index 4f04877594..49c207d6bf 100644 --- a/tests/test_retained_recipe_steps.py +++ b/tests/test_retained_recipe_steps.py @@ -25,7 +25,12 @@ def recipe(vendor, family): - model = f"hpt_bc_flow_{vendor}" if family == "hpt" else f"pi0.5_bc_{vendor}" + if family == "hpt": + source = "human" if vendor == "aria" else vendor + model = f"hpt/bc_flow_{source}" + else: + source = "mecka" if vendor == "scale" else vendor + model = f"pi/bc_{source}_legacy" with initialize_config_dir(version_base=None, config_dir=str(CONFIGS)): return compose( config_name="train_zarr_cartesian", diff --git a/tests/test_retiming_multiplier_condition.py b/tests/test_retiming_multiplier_condition.py index a2ae743093..dc63c1e13c 100644 --- a/tests/test_retiming_multiplier_condition.py +++ b/tests/test_retiming_multiplier_condition.py @@ -155,7 +155,7 @@ def test_recipe_preserves_compression_and_explicit_rollout_contract(monkeypatch) config_name="train_zarr_cartesian", overrides=[ "hydra/launcher=basic", - "+experiment=sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", + "+experiment=arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", ], ) assert ( diff --git a/tests/test_shared_robot_components.py b/tests/test_shared_robot_components.py index 479cbf7390..8556c383db 100644 --- a/tests/test_shared_robot_components.py +++ b/tests/test_shared_robot_components.py @@ -17,7 +17,7 @@ def test_camera_mse_preserves_source_semantics_at_angle_wrap(): prediction = target.clone() target[..., 3] = torch.pi - 0.01 prediction[..., 3] = -torch.pi + 0.01 - for recipe in ("eval_hpt", "eval_pi"): + for recipe in ("eval_cartesian_camera", "eval_cartesian_camera_prompt"): with initialize_config_dir(version_base=None, config_dir=str(CONFIGS)): cfg = compose( config_name="train_zarr_cartesian", overrides=[f"evaluator={recipe}"] diff --git a/tests/test_speed_config.py b/tests/test_speed_config.py index 523032efe0..d621764885 100644 --- a/tests/test_speed_config.py +++ b/tests/test_speed_config.py @@ -15,7 +15,7 @@ def test_pair_config_diff_is_only_encoding(monkeypatch): config_name="train_zarr_cartesian", overrides=[ "hydra/launcher=basic", - "+experiment=sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", + "+experiment=arc_sim/pusht/action_flow_cotrain_uc_multiplier_interpolation", f"speed_diagnostic.encoding={encoding}", ], ) @@ -70,7 +70,7 @@ def __init__(self, config): def test_measured_speed_recipes_are_not_shipped(): root = Path(__file__).resolve().parents[1] / "egomimic/hydra_configs" for name in ( - "experiment/sim/pusht/action_flow_cotrain_uc_speed_interpolation.yaml", + "experiment/arc_sim/pusht/action_flow_cotrain_uc_speed_interpolation.yaml", "experiment/e1/yam_human_keypoints_speed_h816_private512_s42.yaml", "data/e1/yam_human_keypoints_speed_proportional_val01.yaml", ): diff --git a/tests/test_three_source_reconciliation.py b/tests/test_three_source_reconciliation.py index 0f9d34c64f..d1e0827262 100644 --- a/tests/test_three_source_reconciliation.py +++ b/tests/test_three_source_reconciliation.py @@ -198,7 +198,7 @@ def test_offline_audit_inputs_are_scoped_and_do_not_become_launch_defaults(monke monkeypatch.setenv("MATCHED_SOURCE_COMMIT", "human-owned-value") monkeypatch.delenv("DP_CHAIN_SPLIT_MANIFEST", raising=False) for relative in ( - "experiment/sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml", + "experiment/arc_sim/pusht/planar_uc_manual4919_dp_261m_af_obs_multiplier.yaml", ): with compose_for_audit(CONFIGS / relative) as cfg: OmegaConf.to_container(cfg.model, resolve=True, throw_on_missing=True) diff --git a/tests/test_time_retimer_controls.py b/tests/test_time_retimer_controls.py index c2c374c70e..d7cf388e6d 100644 --- a/tests/test_time_retimer_controls.py +++ b/tests/test_time_retimer_controls.py @@ -8,7 +8,7 @@ def test_time_recipe_speed_controls_bind_to_the_deployed_retimer_and_act(): - with compose_for_audit(CONFIGS / "experiment/real/earbuds_time.yaml") as cfg: + with compose_for_audit(CONFIGS / "experiment/arc_real/earbuds_time.yaml") as cfg: profile = build_inference_config(cfg)["inference_graph"]["profiles"]["default"] decoder = instantiate(profile["adapter"]["decoder"]) assert type(decoder).__name__ == "TimeChunkRetimer" diff --git a/tests/test_unite_register.py b/tests/test_unite_register.py index 527446fcf9..692a488e2c 100644 --- a/tests/test_unite_register.py +++ b/tests/test_unite_register.py @@ -427,7 +427,7 @@ def test_four_rows_compose_with_supported_data_and_evaluator_schema( config_name="train_zarr_cartesian", overrides=[ f"model=bf/{row}", - "+experiment=sim/pusht/unite_usocket_register_sweep_val01_h16", + "+experiment=arc_sim/pusht/unite_usocket_register_sweep_val01_h16", "++paths.root_dir=.", ], ) @@ -507,7 +507,7 @@ def test_ice_launcher_explicitly_wires_unite_diagnostics(): experiment = yaml.safe_load( ( CONFIG_DIR - / "experiment/sim/pusht/unite_usocket_register_sweep_val01_h16.yaml" + / "experiment/arc_sim/pusht/unite_usocket_register_sweep_val01_h16.yaml" ).read_text() ) assert experiment["trainer"]["max_steps"] == 240_000 diff --git a/tests/test_validate_checkpoint_sweep.py b/tests/test_validate_checkpoint_sweep.py index 0d89a85805..bfa8d60115 100644 --- a/tests/test_validate_checkpoint_sweep.py +++ b/tests/test_validate_checkpoint_sweep.py @@ -80,7 +80,7 @@ def test_builds_arc_eval_command_with_checkpoint_step_and_shared_wandb_id(tmp_pa command = validation_command( python="python", - experiment="real/stationary_duration", + experiment="arc_real/stationary_duration", checkpoint=checkpoint, output_dir=tmp_path / "results", action_mode="arc", @@ -95,7 +95,7 @@ def test_builds_arc_eval_command_with_checkpoint_step_and_shared_wandb_id(tmp_pa assert command[:3] == [ "python", "egomimic/trainHydra.py", - "+experiment=real/stationary_duration", + "+experiment=arc_real/stationary_duration", ] assert "mode=eval" in command assert "eval_logger_enabled=true" in command @@ -121,7 +121,7 @@ def test_builds_baseline_eval_command_with_explicit_action_mode(tmp_path): command = validation_command( python="python", - experiment="real/stationary_time", + experiment="arc_real/stationary_time", checkpoint=checkpoint, output_dir=tmp_path / "results", action_mode="baseline", @@ -136,7 +136,7 @@ def test_builds_baseline_eval_command_with_explicit_action_mode(tmp_path): assert "evaluator.action_mode=baseline" in command assert "evaluator.execute_fraction=0.3" in command assert "trainer.limit_val_batches=1.0" in command - assert "+experiment=real/stationary_time" in command + assert "+experiment=arc_real/stationary_time" in command def test_builds_video_only_command_without_metric_output(tmp_path): @@ -148,7 +148,7 @@ def test_builds_video_only_command_without_metric_output(tmp_path): command = validation_command( python="python", - experiment="real/stationary_duration", + experiment="arc_real/stationary_duration", checkpoint=checkpoint, output_dir=tmp_path / "output", action_mode="arc", diff --git a/tests/test_wrist6d_roundtrip.py b/tests/test_wrist6d_roundtrip.py index 3c8298dd82..66da4d0fe1 100644 --- a/tests/test_wrist6d_roundtrip.py +++ b/tests/test_wrist6d_roundtrip.py @@ -265,7 +265,7 @@ def test_human_wristframe_6d_pipeline_round_trips_to_headframe(fix_left): torch.testing.assert_close(a_un, a_t, atol=1e-5, rtol=0) torch.testing.assert_close(o_un, o_t, atol=1e-5, rtol=0) - # (4) evaluator revert (batched, like eval_pi) lands back in head frame + # (4) evaluator revert (batched, like eval_cartesian_camera_prompt) lands back in head frame rev = _build_human_cartesian_revert_6d_wristframe_transform_list() out = Embodiment.apply_transform( {"actions_cartesian": a_un, "observations.state.ee_pose": o_un}, rev @@ -416,7 +416,7 @@ def test_split_keys_rejects_width_mismatch(): def test_ypr_revert_on_6d_batch_fails_loudly(): - """The evaluator/data-config mismatch the guard is for: eval_pi.yaml's + """The evaluator/data-config mismatch the guard is for: eval_cartesian_camera_prompt.yaml's ypr revert applied to a cartesian_wristframe_6d batch.""" from egomimic.rldb.embodiment.human import ( _build_human_cartesian_revert_eef_frame_transform_list, diff --git a/tests/test_yam_data_pools.py b/tests/test_yam_data_pools.py new file mode 100644 index 0000000000..06cd1fedab --- /dev/null +++ b/tests/test_yam_data_pools.py @@ -0,0 +1,28 @@ +"""All three action representations retain the current task's pinned pool.""" + +import pytest +from omegaconf import OmegaConf + +from scripts.audit_hydra_configs import CONFIGS, compose_for_audit + + +@pytest.mark.parametrize( + "task,train,val", + [("stationary", 417, 24), ("earbuds", 339, 15), ("towels", 462, 17)], +) +def test_modes_preserve_pinned_episode_membership(task, train, val): + signatures = [] + for mode in ("time", "duration", "velocity"): + with compose_for_audit( + CONFIGS / f"experiment/arc_real/{task}_{mode}.yaml" + ) as cfg: + data = OmegaConf.to_container(cfg.data, resolve=True) + signature = [] + for split, count in (("train_datasets", train), ("valid_datasets", val)): + dataset = data[split]["yam_bimanual"] + assert dataset["expected_train_episode_count"] == count + digest = dataset["expected_train_episode_names_sha256"] + assert len(digest) == 64 + signature.append((dataset["filters"], count, digest)) + signatures.append(signature) + assert signatures[0] == signatures[1] == signatures[2] diff --git a/tools/libero_maintained_dispatch_v1.py b/tools/libero_maintained_dispatch_v1.py index 6c7aa92349..034c921397 100644 --- a/tools/libero_maintained_dispatch_v1.py +++ b/tools/libero_maintained_dispatch_v1.py @@ -14,7 +14,7 @@ profile_for_experiment, ) -PROFILE = "sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" +PROFILE = "arc_sim/libero_historical/action_flow_libero10_h240_euler50_dithalf_80k_s42" SOURCE = "libero_panda" EXPECTED_PARAMETER_COUNT = 39750391 SEED_BANK_SHA = "88657b829905d4374823db145ded19b99cec4735f76694734473bcee068bb5b6" diff --git a/tools/libero_oat_pair_config.py b/tools/libero_oat_pair_config.py index 03b6f97626..2f153e1e9c 100644 --- a/tools/libero_oat_pair_config.py +++ b/tools/libero_oat_pair_config.py @@ -18,7 +18,7 @@ def current_experiment(family, suite): if suite not in SUITES or family not in ("dp", "action_flow"): raise ValueError("unsupported current LIBERO family/suite") suffix = "oat_batch16_keep_steps_s42" if family == "dp" else "oat_dp_matched_s42" - return f"sim/libero/{family}_{suite}_{suffix}" + return f"arc_sim/libero/{family}_{suite}_{suffix}" def validate(cfg, family, suite, *, recipe_contract="current"): diff --git a/tools/validate_action_flow_config.py b/tools/validate_action_flow_config.py index 0792d635c6..a8a7393cca 100644 --- a/tools/validate_action_flow_config.py +++ b/tools/validate_action_flow_config.py @@ -120,18 +120,18 @@ # Serialized checkpoint identities, not a menu of selectable training recipes. CHECKPOINT_METHOD_CONTRACTS = { "pusht/" + HISTORICAL_COMPAT_CONFIG_NAME: STOPGRAD_UNITE_METHOD, - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": STOPGRAD_METHOD, + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42": STOPGRAD_METHOD, "pusht/action_flow_bc_usocket_latent_fm_sg_recon1_codec98k_s42": STOPGRAD_METHOD, - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42": STOPGRAD_METHOD, - "sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42": STOPGRAD_METHOD, + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_muon_lr1e5_s42": STOPGRAD_METHOD, + "arc_sim/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_200m_adamw_lr1e5_s42": STOPGRAD_METHOD, "pusht/action_flow_usocket_latent_fm_sg_unite_h384_s42": STOPGRAD_UNITE_METHOD, "pusht/action_flow_usocket_latent_fm_sg_unite_h384_sum14_cfg4_val8_s42": STOPGRAD_UNITE_METHOD, "pusht/action_flow_usocket_latent_fm_sg_unite_h384_sum14_cfg4_val8_deterministic_s42": STOPGRAD_UNITE_METHOD, "pusht/action_flow_usocket_latent_fm_sg_unite_h512d14h16_sum14_cfg4_val10k_s42": STOPGRAD_UNITE_METHOD, "pusht/action_flow_chain_points6_latent_fm_sg_unite_h512d14h16_sum14_cfg4_val10k_s42": STOPGRAD_UNITE_METHOD, "pusht/action_flow_cotrain_uc_latent_fm_sg_unite_h512d14h16_sum14_cfg4_val10k_s42": STOPGRAD_UNITE_METHOD, - "sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42": LIKELIHOOD_METHOD, - "sim/pusht/action_flow_bc_usocket_graph_section_s42": GRAPH_METHOD, + "arc_sim/pusht/action_flow_bc_usocket_bridge_likelihood_s42": LIKELIHOOD_METHOD, + "arc_sim/pusht/action_flow_bc_usocket_graph_section_s42": GRAPH_METHOD, } LIKELIHOOD_STAGE_TARGETS = EXPECTED_STAGE_TARGETS[:3] + tuple( f"egomimic.pipeline.stages_action_flow_likelihood.{name}" @@ -492,7 +492,7 @@ def compose_experiment( "invalid experiment selector", ) _require( - not experiment.startswith(("sim/pusht/parents/", "pusht_historical/")), + not experiment.startswith(("arc_sim/pusht/parents/", "pusht_historical/")), "parent/historical PushT configs are not fresh training entry points; use the checkpoint's recorded source", ) _require( diff --git a/training.md b/training.md index 1e9de2bead..c2288e3beb 100644 --- a/training.md +++ b/training.md @@ -13,7 +13,21 @@ Its model, data, and evaluator defaults are intentionally null: every run must select a complete experiment instead of inheriting a historical policy by accident. -A direct composition has this shape: +Prefer a task-first experiment. Each public real recipe links all components +needed by the training entry point: + +```bash +python egomimic/trainHydra.py \ + +experiment=arc_real/stationary_velocity --cfg job --resolve +``` + +This prints the configuration without training. Substitute `stationary_duration` +or `stationary_time`, or select a task from the +[experiment menu](egomimic/hydra_configs/experiment/README.md). +See [CONFIGURATION.md](docs/CONFIGURATION.md) for shared parents, model families +and behavior-named evaluators. + +For advanced component selection, a direct composition has this shape: ```bash python egomimic/trainHydra.py \ @@ -62,6 +76,20 @@ not inferred, with `train_only` or its mirror `inference_only`. ## Action tokenization +### Cartesian bimanual ARC + +Real ARC experiments select `abc.arc_clock_mode=velocity|duration` and +`abc.arc_streams=full_separation|translation_rotation|translation|projection`. +Velocity stores start delay followed by incoming per-waypoint rates; duration +stores absolute waypoint seconds, including start delay in row zero. Shapes +are M×20, M×18 or M×16 according to stream separation. The tokenizer and +first-moving-stream decoder share this contract. See +[ARC_CLOCK_CONTRACT.md](docs/ARC_CLOCK_CONTRACT.md) for geometry, timing, +holds and checkpoint compatibility. + +### Planar ARC + +The following is the separate planar codec, not the Cartesian clock API above. Arc-length tokenization can run either in the loader's `transform_list` or as graph nodes. The node form makes the boundary visible and lintable: @@ -99,8 +127,8 @@ episode's 30 Hz frames and the tokenizer, and `arc_dt` is the true capture period. A loader-side arc path that resamples before tokenizing would understate arc length and make that constant wrong. -Compare `experiment/sim/pusht/planar_v2_usocket_arc_bc` (tokenized in the loader) -with `experiment/sim/pusht/planar_v2_usocket_arc_graph_tok` (tokenized in the +Compare `experiment/arc_sim/pusht/planar_v2_usocket_arc_bc` (tokenized in the loader) +with `experiment/arc_sim/pusht/planar_v2_usocket_arc_graph_tok` (tokenized in the graph). Both are valid; only the second shows the tokenizer in `gt`-style graph output and in `tools/config_graph.py`. @@ -123,14 +151,16 @@ They support annotation and embedding utilities and do not select a policy. ## Slurm -After a configuration composes and passes its required smoke test, add `-m` to -use the configured Submitit launcher: +After a configuration composes and passes its required smoke test, select the +reviewed Submitit launcher explicitly and add `-m`, even for one run: ```bash -python egomimic/trainHydra.py --config-name=train_zarr_cartesian \ - model= data= \ - evaluator= -m +python egomimic/trainHydra.py -m \ + +experiment=arc_real/stationary_velocity \ + hydra/launcher= ``` -Always inspect the fully resolved config saved with the run before treating a -launch as reproducible. +This command submits a job; run it only after launch authorization and +cluster-specific preflight. Verify GPU/task layout and effective global batch +in the resolved launcher configuration. Always retain the fully resolved +config, source commit and normalization identity with the run.