Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,29 @@ Open [the collection planner](http://127.0.0.1:8270/coverage) in the same local

The [episode tape recorder](docs/episode-tape.md) preserves the existing simulated arm's measured state, requested/applied commands, contacts and sensor ages. Recording needs the optional arm extra; validation and inspection need only Python's standard library. These tools are available from the current checkout and do not train a new policy or change the older Hugging Face pilot.

The [Skillspace improvement pilot](docs/improvement-studio.md) now connects a
measured failure to a frozen training-data experiment and the next investigation.
Its [first executed comparison](docs/skillspace-improvement-pilot-20261015.md)
found no task-completion gain from the targeted addition. The local mobile-ready
viewer keeps that negative result visible and distinguishes a recording request
from a controller investigation. This is a simulation research tool, not a
production account feature or a human-video-to-robot training bridge.

The [grip/recovery follow-up](docs/grip-recovery-diagnostic-20261015.md) ran 48
native episodes on the shared failure. Increased jaw capability plus a longer
deadline completed the case; the original conditions still fail. The viewer
keeps these diagnostic outcomes separate from qualification. The
[video-to-robot requirements](docs/video-to-robot-skill-requirements.md) describe
action capture, simulation calibration, coverage and physical validation needed
to turn a Skillspace's recordings into a tested robot-specific skill.

The [controlled-recovery experiment](docs/controlled-recovery-experiment-20261015.md)
then tested lowering to support and holding through actual jaw clearance at the
original force limits and 18-second deadline. Two frozen revisions completed
136 episodes. The sampled release handoff improved; completion stayed 15/16
with no paired completion regressions. Both revisions and their limits are
visible in the local improvement viewer; no controller was promoted.

## Review what is visible in a video

The new [local observation lab](docs/observations.md) uses optional CPU object and
Expand Down
42 changes: 42 additions & 0 deletions benchmarks/controlled-recovery-20261015/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Controlled recovery receipts — 15 October 2026

These are byte-identical protocol and result copies from two completed local
simulation experiments. The second experiment explicitly references the first
protocol and report hashes. Both plans and their source snapshots were frozen
before running.

| Receipt folder | Intervention | Native episodes | Complete wall clock |
| --- | --- | ---: | ---: |
| `phase-one` | Lower to settled table support before commanding the jaws open | 68 | 66.698673 s |
| `phase-two` | Also hold the hand still until measured jaw clearance and settled support | 68 | 70.969774 s |
| Total | Same 16 previously inspected cases, two revisions | **136** | **137.668447 s** |

Each phase compares baseline and candidate across all 16 original student cases,
plus the teacher on the known failing case. Every run has its own recorded-arm,
open-jaw control: 34 controller episodes and 34 controls per phase. All attempts
completed, all controls failed, all 17 baseline trajectories exactly reproduced
their originals, and all 17 candidate action prefixes matched before recovery.

The student still completes 15/16 cases in both revisions and both baselines.
There are no task-completion gains or regressions. The remaining case still
fails the original 18-second deadline. The second revision removes unsupported
opening commands in the sampled target recovery and waits for physical jaw
clearance before withdrawal. That is a bounded recovery-behavior improvement,
not a qualified robot skill or a demonstrated task-completion improvement.

The frozen confirmation gate was not met. No fresh confirmation suite was run,
no model was retrained and no production controller was changed. These repeated,
previously inspected cases are not independent generalization trials.

Read the [measured report](../../docs/controlled-recovery-experiment-20261015.md)
for exact settings, timing, observations and limits. External inference spend
was $0; local compute is unpriced. Full synthetic traces, replay HTML, model
weights and source snapshots remain in ignored `runs/` directories. These
compact receipts include no human video, credentials or private account data.

| Artifact | SHA256 |
| --- | --- |
| Phase-one protocol | `d35eec1f71c0154f75078b313338a0627487731057a3582c4afb04e9377bfe8d` |
| Phase-one report | `8d9d5162fd9904e3d4770ff335d6156414247267ad15d500bfcdc1f236b1b2d3` |
| Phase-two protocol | `e4736ec7895ce2f445dfece37eb943470b628c2287e7fd92af1a8ab11619f882` |
| Phase-two report | `acd006c62adc9749d2e038c117714ecdd5b1d79eed7c55287dd226d7f193a659` |
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"attempts":[{"case_id":"scene-15","policy":"student","variant":"baseline"},{"case_id":"scene-15","policy":"teacher","variant":"baseline"},{"case_id":"scene-15","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-15","policy":"teacher","variant":"controlled_lowering"},{"case_id":"scene-00","policy":"student","variant":"baseline"},{"case_id":"scene-00","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-01","policy":"student","variant":"baseline"},{"case_id":"scene-01","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-02","policy":"student","variant":"baseline"},{"case_id":"scene-02","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-03","policy":"student","variant":"baseline"},{"case_id":"scene-03","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-04","policy":"student","variant":"baseline"},{"case_id":"scene-04","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-05","policy":"student","variant":"baseline"},{"case_id":"scene-05","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-06","policy":"student","variant":"baseline"},{"case_id":"scene-06","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-07","policy":"student","variant":"baseline"},{"case_id":"scene-07","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-08","policy":"student","variant":"baseline"},{"case_id":"scene-08","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-09","policy":"student","variant":"baseline"},{"case_id":"scene-09","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-10","policy":"student","variant":"baseline"},{"case_id":"scene-10","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-11","policy":"student","variant":"baseline"},{"case_id":"scene-11","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-12","policy":"student","variant":"baseline"},{"case_id":"scene-12","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-13","policy":"student","variant":"baseline"},{"case_id":"scene-13","policy":"student","variant":"controlled_lowering"},{"case_id":"scene-14","policy":"student","variant":"baseline"},{"case_id":"scene-14","policy":"student","variant":"controlled_lowering"}],"benchmark_protocol_sha256":"ec842fcbf4a99c0144a48463cef6d203149531066d4089f2f4fb0146ef98c9b2","benchmark_report_sha256":"1bfb368fa7393c35c8b808f5f7b96ac2c314c9d4a6ae7617f9da7368b0dc90ce","capsule_file_sha256":"32c62628ed40064de57e3bcb3e32c041b16b268a48dd59307c32f778c3b5ec4d","cases":[{"group":"nominal","id":"scene-00","layout_sha256":"789d56d498c92cfc1980664db28d3bc7a083ee0cbc812bcfb9c8c49014eaf61f","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.2,"object_mass":0.04,"object_position":[0.459453,0.013509,0.31],"object_size":[0.045,0.04,0.04],"seed":381527,"target_position":[0.373487,0.157965,0.31]},"scene_sha256":"1086739a0d3be3d0ebcd61bf42818fe424c53cd355ab8c68e65e4e1d3c38f4e3"},{"group":"nominal","id":"scene-01","layout_sha256":"071dd0b0f3d534c76e8ed9dbd2ab22928ba665b074b42d72ad8b3c062bd7ac23","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.2,"object_mass":0.08,"object_position":[0.527017,0.067855,0.3075],"object_size":[0.05,0.035,0.035],"seed":381528,"target_position":[0.362924,0.157239,0.3075]},"scene_sha256":"f5c959bbe841604e23b93ac1d4c99c1c027f2d4b5e315ad02f08126bb08103d2"},{"group":"nominal","id":"scene-02","layout_sha256":"db58cb81a3405c7cc89e03503267939c958fa917d9bf0c33945ac852cbd98fe5","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.4,"object_mass":0.02,"object_position":[0.343031,-0.080238,0.31],"object_size":[0.05,0.045,0.04],"seed":381529,"target_position":[0.296122,0.068203,0.31]},"scene_sha256":"6ab036b5aeb31981bd16025815def9f708afc5f4181c257a720802ab964bf116"},{"group":"nominal","id":"scene-03","layout_sha256":"1292f5291f53417a73280099c2badda159326d09b765ef25bfd0e014152220b8","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.2,"object_mass":0.04,"object_position":[0.446442,-0.035939,0.305],"object_size":[0.03,0.03,0.03],"seed":381530,"target_position":[0.388333,0.15314,0.305]},"scene_sha256":"591f8091aef5d9a704d8ab2214a8afb14d7b98c4d8afc1a8d6baa153f260aad3"},{"group":"nominal","id":"scene-04","layout_sha256":"7924c9a3300366254f90747174d7d1c1a3bbe05392490624e5d86237f3af0fbd","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.4,"object_mass":0.06,"object_position":[0.271314,0.1105,0.315],"object_size":[0.045,0.035,0.05],"seed":381531,"target_position":[0.463894,-0.005711,0.315]},"scene_sha256":"e92e9f6406d2ee4e158d26c79b2a58b5f7796832fe298fbec13b88a322868ded"},{"group":"nominal","id":"scene-05","layout_sha256":"f9e97e44eeee2ec78d9ed2b6bef686a67dc675a8715f8caef8ec67ed24b7e244","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.5,"object_mass":0.04,"object_position":[0.518036,-0.132181,0.3125],"object_size":[0.05,0.04,0.045],"seed":381532,"target_position":[0.383342,-0.112624,0.3125]},"scene_sha256":"b7b02f0964fb17a47d722c061305f4372eb59044a5130e5154ca1d0d1a0f2b27"},{"group":"nominal","id":"scene-06","layout_sha256":"209fb86e5f6b15e6c690afe41f36fe0b9f56cf00a51b5696c56fa2c257a92998","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.8,"object_mass":0.08,"object_position":[0.431451,-0.03774,0.315],"object_size":[0.045,0.035,0.05],"seed":381533,"target_position":[0.454662,-0.148527,0.315]},"scene_sha256":"58d95e82b69a4f1e12ff8bcde67f8d22797d2a3008c14bf331a87b65f601ec80"},{"group":"nominal","id":"scene-07","layout_sha256":"aa78d3be9f01f3058db6787c139548cb649337009ecf88ff96139e7b263a2c95","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.8,"object_mass":0.02,"object_position":[0.277889,-0.162546,0.31],"object_size":[0.045,0.05,0.04],"seed":381534,"target_position":[0.4694,0.165679,0.31]},"scene_sha256":"b7f1632e2983bc8bab2e30a32372057b4375fd34764f7c8f9a13f298bbec9999"},{"group":"nominal","id":"scene-08","layout_sha256":"01ec5d4dfbd59aee896912efe9163a1aaa8ddbbd9fa63b0250430cd18885779d","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.2,"object_mass":0.02,"object_position":[0.52549,0.079728,0.31],"object_size":[0.04,0.03,0.04],"seed":381535,"target_position":[0.375258,0.125991,0.31]},"scene_sha256":"e61927d0666191a72058a3ccc2448b7a0b90da79a8146001475cfb17d5243aef"},{"group":"nominal","id":"scene-09","layout_sha256":"62d79f85f44f901a0a02726e79d6c463a173b0dec1a18ded51923bb0f0985ab2","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.8,"object_mass":0.08,"object_position":[0.276059,-0.055274,0.3125],"object_size":[0.04,0.05,0.045],"seed":381536,"target_position":[0.329581,0.084425,0.3125]},"scene_sha256":"e250458d5b8be5556e4cc9db717946e6e856a09d175c36deb798bc7c1aa6c65e"},{"group":"nominal","id":"scene-10","layout_sha256":"bb371dff8341503216066d0068c64c38a145892c4bec40e2ae4b1f1ab1a6cf33","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":0.4,"object_mass":0.06,"object_position":[0.407735,-0.084183,0.31],"object_size":[0.035,0.04,0.04],"seed":381537,"target_position":[0.44571,0.063405,0.31]},"scene_sha256":"7d4db022b4a12fa154868ef14e9d3f06337cbee501e62604fecaecb9cccb5fd3"},{"group":"nominal","id":"scene-11","layout_sha256":"61a0df352254f2851dd1b702e68864f8b1007667ad810cab44bac1e7a80e6968","scene":{"jaw_force_limit":15.0,"joint_torque_limit":40.0,"object_friction":1.5,"object_mass":0.08,"object_position":[0.282079,0.116322,0.305],"object_size":[0.045,0.035,0.03],"seed":381538,"target_position":[0.488661,0.050336,0.305]},"scene_sha256":"788216e2089c14fadcf5735495af27510f9f5d95e84260536962dede05982f0d"},{"group":"actuator_stress","id":"scene-12","layout_sha256":"3f8a9c923e055c28fde4475f43d05e3745d44cc2be80b0364f807d8116262beb","scene":{"jaw_force_limit":0.3,"joint_torque_limit":5.0,"object_friction":1.2,"object_mass":0.02,"object_position":[0.437015,0.128491,0.3075],"object_size":[0.045,0.03,0.035],"seed":381539,"target_position":[0.383355,0.010512,0.3075]},"scene_sha256":"19d57bedda61dcbcf71ef6c563d175308c73d0b4c74b577d18924ee0025bfa6f"},{"group":"actuator_stress","id":"scene-13","layout_sha256":"b8cb868707e1f5e8a6520d020215a241c6d6b0efb206dd73a6d1a9613edb9963","scene":{"jaw_force_limit":0.3,"joint_torque_limit":5.0,"object_friction":0.4,"object_mass":0.02,"object_position":[0.38773,-0.102516,0.315],"object_size":[0.05,0.03,0.05],"seed":381540,"target_position":[0.305714,0.094967,0.315]},"scene_sha256":"5cceae37e963e35dec839d7eb2503b2b5db9377c70fd68fee5b2c8668811a3dd"},{"group":"actuator_stress","id":"scene-14","layout_sha256":"78d751dbeb1227087d814711b904f299620f5731a579a84bf2e8361c53745e3a","scene":{"jaw_force_limit":0.3,"joint_torque_limit":5.0,"object_friction":1.5,"object_mass":0.06,"object_position":[0.295324,0.167677,0.315],"object_size":[0.05,0.05,0.05],"seed":381541,"target_position":[0.293713,0.058456,0.315]},"scene_sha256":"e9ec1b555e0183ac8ba9ea56d743b707d277e3dc08e85a2026d890c63ebabb44"},{"group":"actuator_stress","id":"scene-15","layout_sha256":"55376e0c86f0c5fbc20bd4d66b3b0022da5815da9472504ebce4f77f297862e1","scene":{"jaw_force_limit":0.3,"joint_torque_limit":5.0,"object_friction":1.2,"object_mass":0.06,"object_position":[0.329349,0.167046,0.31],"object_size":[0.03,0.05,0.04],"seed":381542,"target_position":[0.491828,-0.153256,0.31]},"scene_sha256":"81910996057200e1e36777ddb5c2e1159e9e871ffa94ed414cd8ffe99d08a474"}],"confirmation_gate":{"allowed_control_successes":0,"allowed_errors":0,"allowed_regressions":0,"confirmation_required_before_promotion":true,"strict_task_success_gain":true},"kind":"dvidia.controlled-recovery-experiment","limits":["One experiment-only controller revision on 16 previously inspected simulation cases; regression, not held-out generalization.","Same original force caps, model weights, object states/material and 18-second deadline; no retraining or predicate relaxation.","Supported-opening command evidence is separate from task completion; holding until timeout is not a task success.","Support and speed instrumentation is sampled at 50 Hz; it does not measure impact force or prove continuous native contact.","Simplified rigid box, privileged simulator state, authored phases and collision-excluded links; no physical-robot qualification.","No automatic promotion. Fresh confirmation requires a strict task-completion gain, zero regressions/errors and failed open-jaw controls."],"original_run_sha256":{"student-scene-00":"e19c51162e03c135891bf29547d3772cf2c674a5d978b82703991a13cdbd6a22","student-scene-01":"f3ebbe1b4173e5dff92af365d401b8808b5ce17c3000e608e86575e71f723b61","student-scene-02":"9ea50d31293aaed0251d89df3b5ead40dc7e678d501cf9ea6df6584bc378bd13","student-scene-03":"88a89d2a2e84c7a41dbd3fd730643106703fa3306cc5b4450e6cc45aeba26c4e","student-scene-04":"67cd09c4dadadd1c8791ca1cfa7cb50fa555764fe9fee0acfcd9c82ba2f5ca2c","student-scene-05":"0513b38be6786cfa242538e77d936f6a82469d2cc42a76286f586b899d2e364e","student-scene-06":"5ae42d74e92fc3addc3cbc72ea363d45d3030e445c00ccd8a7b7bdaf0d399f9a","student-scene-07":"6437d038209835777085a8c5e2f2ac1ec457fba0a7bec116d4b01abbffd57b0c","student-scene-08":"134ff645644e183fe8e20a998e126c37fb72844e091cd3485d8d98c2b34f7dce","student-scene-09":"962fe36ce1a3f2f1951dd249b97f36ee5520323628749452bb41a96f6fdb0f04","student-scene-10":"8c2e0a743bd20aac519d40e00ec0371ff7f57bc1f314cab94f2cd8edcaaf4c90","student-scene-11":"1d620db6fbaae5977ac4b1e423ba86bb13b5e92a00477005c5aa221b8e9d9670","student-scene-12":"d4f96f40c6709902ea9df0ee1e63ee11eb43cae2aa49003cc3437166b0554a54","student-scene-13":"d0d4b3cb9e8e86b59b36af8f014e5b7397810c6986fb7a9cdba31b48a1987287","student-scene-14":"8217946c65100b0e8a1446a96659b684f3deec4e3b33fa38a8940b8704cde291","student-scene-15":"ee5d3b4c61555cd5a2c01028d54813871bcf4b964b874ef67117b85d1668680b","teacher-scene-15":"2662da17cbdb627deaf4c0ab58e3b3cecdf1855e9ee8b5ca7aa0fbed46659cb8"},"physical_robot_ready":false,"planned_native_episodes":68,"runtime":{"action_contract":"six joint targets in radians; symmetric gripper width in metres","dependencies":{"mujoco":"3.15.0","numpy":"2.5.3"},"files_sha256":{"arm_distill.py":"14640f91ec9c91d10a405fa56eb21bda8b55b285854ce2b5859a71d1ffd06ea1","arm_env.py":"e4af9b75364a568284b1e6d96469deacf498dd87a60b067fa5e310fe15c54c00","arm_policy.py":"d6d53d87703fb8608390092309285afb6a2c8fd3103a7abd77668be85469eab3","arm_runner.py":"affb5812b6a395f6cca9cb4e900af2ef28162e7ff1e52a51bd29b789daa98df1","capsule_installer.py":"60bb1652cae830fd03ae6eaa5e10150c957be13e1dc630a5631c1a2692ae37ec","cli.py":"192ec6a0852c7fd5dd3ae912815dae2fc7c9cfb93b97c85d528060e00a975b17","env.py":"929c1d8c3c4cafe0054c633ee5f3e51c551dd6b26ae014ab1ea9219b58eca8e4","installer.py":"d3a2c5a4913920c52beef89df9954494b89fd0becb08bb4847264dae48d3761c","skill_capsule.py":"ecb7dbd60d3a11f40bf65c4520f024b449c0d63f9ae27c601c9c9fe17b32e6a0","skillspace.py":"34b651609358ae9a58b9a610545ce7b85b83fb3d6c699613414ff8e0ea457b3c"},"kind":"hybrid-distilled-placement-runtime"},"schema_version":1,"scope":"simulation-only","settings":{"force_caps":"each original case unchanged","horizon_seconds":18.0,"lower_goal_cap_m":0.003,"lower_timeout_seconds":4.0,"movement_mapping":"same frozen student head / authored teacher IK; no fallback","phase_budgets":"normal phases unchanged; explicit four-second recovery-lowering budget within 18-second horizon","support_angular_speed_rad_s":0.3,"support_height_tolerance_m":0.008,"support_linear_speed_m_s":0.035,"support_samples":5},"source_sha256":{"arm_distill.py":"14640f91ec9c91d10a405fa56eb21bda8b55b285854ce2b5859a71d1ffd06ea1","arm_env.py":"e4af9b75364a568284b1e6d96469deacf498dd87a60b067fa5e310fe15c54c00","arm_policy.py":"d6d53d87703fb8608390092309285afb6a2c8fd3103a7abd77668be85469eab3","arm_runner.py":"affb5812b6a395f6cca9cb4e900af2ef28162e7ff1e52a51bd29b789daa98df1","capsule_installer.py":"60bb1652cae830fd03ae6eaa5e10150c957be13e1dc630a5631c1a2692ae37ec","cli.py":"192ec6a0852c7fd5dd3ae912815dae2fc7c9cfb93b97c85d528060e00a975b17","controlled_recovery_experiment.py":"731f83da85888ed5b22bd8fcbe7451d506a828a42cd485f2dec2d410bf7a700e","controlled_recovery_policy.py":"7447eb2569fa01107b6db1686c584fd619b5e78c41fe6f5ed6e33fb4075bda7b","env.py":"929c1d8c3c4cafe0054c633ee5f3e51c551dd6b26ae014ab1ea9219b58eca8e4","grip_diagnostic.py":"29c2ac53ed60873e4c6b8470818f467e1bedc258a5801eb1bb6feb245c61c39c","installer.py":"d3a2c5a4913920c52beef89df9954494b89fd0becb08bb4847264dae48d3761c","skill_capsule.py":"ecb7dbd60d3a11f40bf65c4520f024b449c0d63f9ae27c601c9c9fe17b32e6a0","skillspace.py":"34b651609358ae9a58b9a610545ce7b85b83fb3d6c699613414ff8e0ea457b3c"},"variants":["baseline","controlled_lowering"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
d35eec1f71c0154f75078b313338a0627487731057a3582c4afb04e9377bfe8d

Large diffs are not rendered by default.

Loading
Loading