Skip to content

Prepare resident native reward qualification and signal audit - #18

Closed
0z5a wants to merge 1 commit into
data/rltt-source-contractfrom
test/resident-reward-qualification
Closed

0z5a wants to merge 1 commit into
data/rltt-source-contractfrom
test/resident-reward-qualification

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

The short native recipe defaults to SGD and 48-token responses, so it cannot establish the intended Adam-based online-RL qualification. This adds a frozen, reward-independent 12-train/8-held-out packet and continuous/split/resume commands for GRPO and RLTT, with explicit optimizer, sampling, response-length and checkpoint settings.

The saved-evidence audit requires mixed-reward groups, nonzero masked advantages and finite nonzero gradients together. Version increments or nonzero gradients alone do not pass. Parameter/Adam-state changes, fresh-worker recovery, held-out equality and convergence remain separate required gates.

Stacked on #17 (67da3243); this incremental diff changes no production runtime. Validation: 136 CPU tests pass, and all 20 prompts match the official tokenizer through the actual VIME Dataset. Raw inputs, the corrected input-audit failure and final receipts are retained with SHA256 manifests. The report includes the comparison table and explicit unmeasured cells.

Actual Ray/CUDA execution, the full framework parser, controller execute path, GPU memory/speed and reward convergence have not run for this profile. Revision flags declare intended inputs; actual weights and worker sources still require verification before execution.

Report: native_reward_qualification.md.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant