Skip to content

tests: keep the suites that protect codec, data, graph, training, eval and rollout functions; drop launcher, cluster, contract and diagnostic pins - #278

Closed
aidang3019 wants to merge 1 commit into
graphite-base/278from
aidan/trim-tests-20261022
Closed

aidang3019 wants to merge 1 commit into
graphite-base/278from
aidan/trim-tests-20261022

Conversation

@aidang3019

Copy link
Copy Markdown

Aidan (2026-10-22): "remove all of the unnecessary tests except for the ones that maintain all
essential functions and proper training/eval/rollouts". 82 of 247 test files go (56.3k -> 37.7k
lines); 165 stay. The line: a test stays when it checks a function that real or sim training,
evaluation or rollouts run through (tokenizers and codecs, data loading, model graphs and stages,
Lightning training / checkpoint / resume / requeue, evaluators and metrics, robot runtime, teleop,
IK and dashboards, inference contracts, composition of the maintained experiment menu). A test
goes when its subject is a launcher or cluster script, a scheduler / profile / receipt contract, a
historical-parity or retired-path guard, a doc or migration pin, a diagnostic or visualization
tool, or an opt-in benchmark harness.

Removed, by family:

  • ICE / Slurm / OSMO tooling: ice_checkpoint_archive_inventory, ice_checkpoint_mirror,
    ice_checkpoint_mirror_pool, ice_dp_campaign_contract, ice_gpu_probe, ice_planar_bc_launcher,
    ice_requeue_runner, ice_wandb_resume, yam_gpu_uuid_contract, capture_runtime_lock,
    sync_runtime_script, batch_signal_relay, validate_slurm_job_contract, osmo_gate,
    unite_ice_contract, unite_fixed_step_requeue_boundary, check_checkpoint_storage,
    local_cpu_storage_guard, local_cpu_validation_guard, installed_package, pi_source_patch,
    wandb_health.
  • LIBERO native-launch contracts and cluster tooling: libero_cluster, libero_performance (+ the
    libero_ddp_resume_worker helper), libero_multi_embodiment_native, libero_native_binding_parser,
    libero_native_half_factory_identity, libero_native_launch_contract,
    libero_native_scheduler_contract, libero_native_smoke_contract, libero_native_suite_profiles,
    libero_native_velocity_ablation, libero_native_whole_launcher, libero_saved_hydra_context,
    libero_action_flow_artifacts, libero_active_recipes, libero_arc_sweep,
    libero_basis_scaling_audit, current_family_profiles, native_final_validation_lifecycle.
  • Action-flow launch / verifier / candidate gates: action_flow_launcher,
    action_flow_smoke_schedule, verify_action_flow_training_smoke, validate_action_flow_config,
    action_flow_candidate_{codec,configs,gates,stages}, action_flow_affine_configs,
    action_flow_manual4919_config, original_size_action_flow_cotrain_config,
    multiplier_training_launch, standard_dp_launch_args, oat_step_budget.
  • Historical parity / retired-path guards: c12_compatibility, alignment_parity,
    legacy_data_metric_parity, legacy_hpt_trunk_parity, likelihood_training_behavior_parity,
    imported_recipe_framework_parity, measured_speed_removed, pipeline_only_surface,
    pusht_parent_cleanup, offline_constructor_inputs, offline_static_buffer_audit.
  • Diagnostics / visualization: action_flow_diagnostics, action_flow_jacobian_checkpoint_performance,
    token_diagnostics, unite_diagnostics, speed_retiming_diagnostic, arc_tsne, config_graph,
    rank_synthetic_trajectory_nn.
  • Synthetic manifold toy: synthetic_{action_adapter_flow,multi_action_adapter_flow,
    shared_latent_flow,trainer_resume,trainer_signals,trajectory_eval}, gaussian_paraboloid_dataset,
    gaussian_torus_dataset, validate_planar_dataset.
  • Fixtures only those tests read: legacy_pipeline/, legacy_hpt_trunk/, legacy_alignment_reference,
    legacy_language_schedule, libero_native_scontrol_408787, libero_native_suite_corpus_authority,
    pusht_before_real_sim_cleanup, pusht_config_before_parent_cleanup, stattempo_contracts (unused).

Kept in full: every test_pipeline*, test_hpt*, test_arc*, test_robot*, test_rollout*, test_yam_,
test_gello
, test_oat_, test_unite_ (functional), test_libero_{arc,arc_global,arc_timed,benchmark,
evaluate,replay,decoded_replay,training_checks,tuned_configs,native_checkpoint_format,
saved_native_state,action_flow_boundary}, test_planar_, test_pi05_ / test_pi_, the data-module
and resolver tests, the Slurm requeue / signal tests that trainHydra depends on
(slurm_save_only_checkpoint, train_hydra_slurm_environment, trainer_pid_signal,
standard_dp_pid_transport), and the config-menu tests (real_task_recipes,
abc_arc_experiment_layout, recursive_config_audit, consolidated_core_configs, abc_
_configs).

The remaining suite collects 2128 tests; the four collection errors (diffusers, av, mujoco not
installed locally) are identical on the untouched tree. Stale mentions of removed tests remain in
docs/integration/parity-manifest.json and a few docs; the code those tests covered is untouched.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5.1 noreply@anthropic.com

…l and rollout functions; drop launcher, cluster, contract and diagnostic pins

Aidan (2026-10-22): "remove all of the unnecessary tests except for the ones that maintain all
essential functions and proper training/eval/rollouts". 82 of 247 test files go (56.3k -> 37.7k
lines); 165 stay. The line: a test stays when it checks a function that real or sim training,
evaluation or rollouts run through (tokenizers and codecs, data loading, model graphs and stages,
Lightning training / checkpoint / resume / requeue, evaluators and metrics, robot runtime, teleop,
IK and dashboards, inference contracts, composition of the maintained experiment menu). A test
goes when its subject is a launcher or cluster script, a scheduler / profile / receipt contract, a
historical-parity or retired-path guard, a doc or migration pin, a diagnostic or visualization
tool, or an opt-in benchmark harness.

Removed, by family:
- ICE / Slurm / OSMO tooling: ice_checkpoint_archive_inventory, ice_checkpoint_mirror,
  ice_checkpoint_mirror_pool, ice_dp_campaign_contract, ice_gpu_probe, ice_planar_bc_launcher,
  ice_requeue_runner, ice_wandb_resume, yam_gpu_uuid_contract, capture_runtime_lock,
  sync_runtime_script, batch_signal_relay, validate_slurm_job_contract, osmo_gate,
  unite_ice_contract, unite_fixed_step_requeue_boundary, check_checkpoint_storage,
  local_cpu_storage_guard, local_cpu_validation_guard, installed_package, pi_source_patch,
  wandb_health.
- LIBERO native-launch contracts and cluster tooling: libero_cluster, libero_performance (+ the
  libero_ddp_resume_worker helper), libero_multi_embodiment_native, libero_native_binding_parser,
  libero_native_half_factory_identity, libero_native_launch_contract,
  libero_native_scheduler_contract, libero_native_smoke_contract, libero_native_suite_profiles,
  libero_native_velocity_ablation, libero_native_whole_launcher, libero_saved_hydra_context,
  libero_action_flow_artifacts, libero_active_recipes, libero_arc_sweep,
  libero_basis_scaling_audit, current_family_profiles, native_final_validation_lifecycle.
- Action-flow launch / verifier / candidate gates: action_flow_launcher,
  action_flow_smoke_schedule, verify_action_flow_training_smoke, validate_action_flow_config,
  action_flow_candidate_{codec,configs,gates,stages}, action_flow_affine_configs,
  action_flow_manual4919_config, original_size_action_flow_cotrain_config,
  multiplier_training_launch, standard_dp_launch_args, oat_step_budget.
- Historical parity / retired-path guards: c12_compatibility, alignment_parity,
  legacy_data_metric_parity, legacy_hpt_trunk_parity, likelihood_training_behavior_parity,
  imported_recipe_framework_parity, measured_speed_removed, pipeline_only_surface,
  pusht_parent_cleanup, offline_constructor_inputs, offline_static_buffer_audit.
- Diagnostics / visualization: action_flow_diagnostics, action_flow_jacobian_checkpoint_performance,
  token_diagnostics, unite_diagnostics, speed_retiming_diagnostic, arc_tsne, config_graph,
  rank_synthetic_trajectory_nn.
- Synthetic manifold toy: synthetic_{action_adapter_flow,multi_action_adapter_flow,
  shared_latent_flow,trainer_resume,trainer_signals,trajectory_eval}, gaussian_paraboloid_dataset,
  gaussian_torus_dataset, validate_planar_dataset.
- Fixtures only those tests read: legacy_pipeline/, legacy_hpt_trunk/, legacy_alignment_reference,
  legacy_language_schedule, libero_native_scontrol_408787, libero_native_suite_corpus_authority,
  pusht_before_real_sim_cleanup, pusht_config_before_parent_cleanup, stattempo_contracts (unused).

Kept in full: every test_pipeline*, test_hpt*, test_arc*, test_robot*, test_rollout*, test_yam_*,
test_gello*, test_oat_*, test_unite_* (functional), test_libero_{arc,arc_global,arc_timed,benchmark,
evaluate,replay,decoded_replay,training_checks,tuned_configs,native_checkpoint_format,
saved_native_state,action_flow_boundary}, test_planar_*, test_pi05_* / test_pi_*, the data-module
and resolver tests, the Slurm requeue / signal tests that trainHydra depends on
(slurm_save_only_checkpoint, train_hydra_slurm_environment, trainer_pid_signal,
standard_dp_pid_transport), and the config-menu tests (real_task_recipes,
abc_arc_experiment_layout, recursive_config_audit, consolidated_core_configs, abc_*_configs).

The remaining suite collects 2128 tests; the four collection errors (diffusers, av, mujoco not
installed locally) are identical on the untouched tree. Stale mentions of removed tests remain in
docs/integration/parity-manifest.json and a few docs; the code those tests covered is untouched.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@aidang3019

Copy link
Copy Markdown
Author

Folded into #277 together with the config cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant