Skip to content

Measure recording counts through installed simulation qualification - #1

Merged
Tarzelf merged 1 commit into
mainfrom
codex/frozen-sample-count-benchmark
Oct 8, 2026
Merged

Tarzelf merged 1 commit into
mainfrom
codex/frozen-sample-count-benchmark

Conversation

@Tarzelf

@Tarzelf Tarzelf commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Training candidates previously had one-scene evidence and no reproducible way to compare recording counts against installed execution. Add dvidia-training-benchmark: nested whole-group subsets keep development/test footage fixed; protocol and all candidate bytes freeze before scene evaluation; native installation, exact-scene qualification and matched open-jaw controls retain failures and scoped timing.

The published CPU pilot compares 2/4/7 groups across eight nominal and four actuator-stress layouts. Nominal qualifications are 8/8, 7/8 and 8/8; stress qualifications are 3/4 for each, with no successful open-jaw controls. Exact source-layout exclusions are bound to native declarations. This is one composition order, with privileged simulator observations and authored gripper/phase logic, not a minimum video count or human-video/hardware transfer claim.

Validation: 131 local native-enabled tests pass, 17 focused benchmark tests pass, isolated package build and installed entrypoint help pass. The actual offline benchmark completes all 84 teacher/student/control episodes without execution errors. Protocol, models, candidate capsules, results and an evidence archive manifest are included; source runtime/qualification files are unchanged. CI adds base benchmark tests and entrypoint checks across the existing platform matrix.

Also records anonymous verification of the already-public Hugging Face pilot artifacts. Full native traces/replays will be attached as a separate prerelease evidence asset after merge; this does not replace frozen v0.1.0 assets.

@Tarzelf
Tarzelf merged commit 4de53c9 into main Oct 8, 2026
16 checks passed
@Tarzelf
Tarzelf deleted the codex/frozen-sample-count-benchmark branch October 8, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant