Skip to content

Rematerialize loop segments and causal token chunks for RLTT - #10

Closed
0z5a wants to merge 1 commit into
perf/packed-recurrent-replayfrom
perf/recurrent-rematerialization
Closed

0z5a wants to merge 1 commit into
perf/packed-recurrent-replayfrom
perf/recurrent-rematerialization

Conversation

@0z5a

@0z5a 0z5a commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Per-layer checkpointing still retains recurrent boundary states and per-loop response inputs. Add opt-in loop segments with readout inside the checkpoint, physical-layer groups, and causal query chunks with full K/V-prefix gradients. Huginn coda remains outside the recurrent feedback state. A finite profile selector keeps workload, timing-scope and memory-metric contracts separate.

Stacked on #9 (perf/packed-recurrent-replay); one incremental commit. The default replay path is unchanged.

Validation: 354 CPU checks pass; six CUDA cases skip and three known missing-Triton factory checks are deselected. Full-parameter gradient comparisons, nonzero updates, and three Tiny native sampling/reward/Adam/publication/fresh-resume cycles pass with rematerialization.

Results and raw data include 864 same-tree observations and 36 unchanged-parent observations across 12 fresh processes. Selected plans reduce the observed forward saved-storage metric by 78.30–83.43%; this is not peak/GPU memory. Clean-parent ratios are 1.119× Ouro, 0.649× Nanbeige and 0.517× Huginn, with all intervals including 1. Slower schedules and Huginn storage regressions remain in the full table. Target CUDA, official-model full RL and reward convergence remain pending.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant