Skip to content

Add recurrent prefix gradient accumulation reference - #11

Closed
0z5a wants to merge 1 commit into
perf/recurrent-rematerializationfrom
feature/differentiable-prefix
Closed

0z5a wants to merge 1 commit into
perf/recurrent-rematerializationfrom
feature/differentiable-prefix

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Repeated prompt groups currently recompute the same actor prefix in every suffix microbatch. This adds a fixed-depth gradient reference that runs prefix forward once, accumulates every loop/layer KV adjoint and direct first-response readout adjoint, then runs prefix backward once before the optimizer step. Ouro, Nanbeige and same-latent Huginn preserve the complete shared-parameter gradient; Huginn keeps coda separate from recurrent feedback and rejects a different latent identity.

Incremental stack on #10 (5f7ce7fa). This is an explicit serial-suffix reference API, not a default MCore/Ray schedule. The ordinary three-phase prefix technique is attributed to its existing work. Packed suffix execution, distinct-latent Huginn prelude sharing, joint rematerialization, distributed integration and target-GPU/full-RL validation remain pending.

Validation: 402 CPU checks pass, six CUDA checks skip, and three known missing-Triton factory checks are deselected. Thirty-six added full-gradient/update configurations have maximum relative L2 error 1.819e-7; separate FP64 attention and all Q/K/V gradients agree within 1e-12. Original overly strict FP64-weight failures and mixed-precision diagnostics are retained: native normalization still reduces in FP32.

The result report includes all speed-comparison tables and raw data. All 648 same-tree and 108 untouched-parent observations match loss, full gradients and nonzero SGD updates. The serial-suffix reference is slower: clean-parent P24/D4/G32 ratios are 0.404× Ouro, 0.326× Nanbeige and 0.471× Huginn. The parent batches suffix projections; this reference currently does not. No GPU memory saving, end-to-end RL improvement or reward convergence is claimed. All 688 parent blobs and provider import paths are checked; full parent gradient/update oracle vectors are retained.

Accumulate every loop and stage boundary across logical suffix microbatches, including direct first-response readout gradients and Huginn latent identity. Retain full-gradient checks and negative CPU comparisons against packed replay.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant