Skip to content

Separate logical RLTT loss and plan actor prefix groups - #24

Closed
0z5a wants to merge 1 commit into
test/native-qualification-chainfrom
feat/rltt-logical-step
Closed

0z5a wants to merge 1 commit into
test/native-qualification-chainfrom
feat/rltt-logical-step

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

The MCore loss includes a microbatch multiplier that direct prefix backward must not inherit. Extract the unchanged logical RLTT objective, keep the old MCore adapter's scaling, and apply optimizer scaling once to suffix losses before injecting their already-scaled boundary adjoints.

Add an Ouro logical-step plan over actual iterator indices, preserving sample/GRPO group identity and global token/response weights across prompt grouping and suffix waves. Native rollout transport carries the original group IDs. This provides B1/B2 components; it does not yet replace the real learner schedule.

Stacked on #23 (test/native-qualification-chain). Validation: 170 CPU tests pass, including 12 FP32 two-update cases against independent dense and actual MCore loss/backward functions. Maximum gradient relative L2 is 5.38e-7; maximum parameter-delta error is 1.20e-7. Twelve CPU BF16 cases remain strict expected numerical failures under unchanged tolerances; all 24 BF16 update deltas and six later gradients fail qualification. Full DDP/optimizer-wrapper and CUDA validation remain open.

docs/rltt_logical_step.md provides numerical and required-speedup tables. Raw logs/XML, both-update measurements, retained failures and hashes are in benchmarks/results/rltt-logical-step/. No GPU speed, memory reduction or reward-convergence claim is made.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant