Skip to content

Add an opt-in Ouro prefix schedule to the MCore learner step - #25

Closed
0z5a wants to merge 1 commit into
feat/rltt-logical-stepfrom
feat/rltt-prefix-schedule
Closed

0z5a wants to merge 1 commit into
feat/rltt-logical-stepfrom
feat/rltt-prefix-schedule

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Add an explicit --rltt-prefix-wave-size route to the actual MCore learner step. It creates each prompt prefix through DDP forward, accumulates packed suffix and prefix gradients in the existing buffers, finalizes once, and leaves optimizer/scheduler/publication/checkpoint ownership in the existing flow. The iterator advances only after the prefix update succeeds. The default remains the ordinary MCore schedule.

Stacked on #24 (feat/rltt-logical-step). The candidate is limited to one actual learner rank, FP32 Ouro RLTT, eager execution and unsharded non-overlapped gradients. Mixed precision and joint prefix rematerialization are not enabled.

Validation: broad regression has 195 CPU passes, 2 CUDA skips and the 12 known BF16 numerical failures; a final focused run after guard/fixture review has 25 passes and 2 CUDA skips. These runs overlap. Provider entry gradients match exactly, native argument checks pass, and raw logs/XML plus hashes are retained in benchmarks/results/rltt-prefix-schedule/.

The prepared CUDA test calls actual MCore DDP, FP32Optimizer/AdamW, scheduler and train_one_step for two updates, capturing finalized gradients and checking moments/cursor/cleanup. It has not executed: the full requested environment and device window are still pending. docs/rltt_prefix_schedule.md records scope and comparison tables. Official-model learner qualification, GPU speed/memory and reward convergence remain open; this draft does not complete B.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant