Repository navigation
Conversation
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Huginn samples can have the same prompt and different recorded latents. The prefix learner now groups their actual latent identity and uses complete replay for singleton latent groups, preserving the original behavior scores, reference and logical loss. The actual Huginn provider enters through the opt-in FP32 MCore schedule and records shared/full-replay sample counts after backward finalization.
Stacked on #29. The related CPU regression passes 216 checks, skips 12 CUDA cases and retains 12 existing strict Ouro BF16 xfails. Twenty-four Huginn configurations compare two nonzero AdamW updates with complete packed replay, including all parameter gradients and Adam moments, a distinct reference, both reductions, ragged/empty responses, reordered microbatches and loss scales 1/8.
Detailed results and reproduction include the baseline comparison table, unchanged numeric gates, raw positive/negative logs and SHA manifests. Independent, mixed and shared arrangements are explicit fixed-trace fixtures; online sharing frequency is not established. Twelve actual CUDA DDP/Adam cases are prepared. Official Huginn CUDA, GPU memory/speed, scaled online RL and reward convergence remain pending; Huginn joint prefix rematerialization is unsupported.