Skip to content

Support a resident single-GPU native training profile - #14

Closed
0z5a wants to merge 1 commit into
feature/native-heldout-evaluationfrom
feature/native-resident-colocation
Closed

0z5a wants to merge 1 commit into
feature/native-heldout-evaluationfrom
feature/native-resident-colocation

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

The native backend rejected colocation, and its rollout worker requested the entire 1-CPU/1-GPU placement bundle. Removing the guard alone would leave the learner's additional 0.4 CPU/GPU reservation unplaceable and retain the wrong rollout device offset.

This adds an explicit resident GRPO/RLTT profile: one learner and one rollout engine share a bundle, with rollout requesting 0.5 CPU/GPU and reporting offset zero. The recipe disables both offload paths. Model state remains resident in separate workers; Ray fractions do not partition GPU memory. Dedicated placement, training math, publication barriers and held-out evaluation remain unchanged.

Stacked on #13. Author/committer and DCO: 0z5a.

Validation

580 distinct CPU tests pass; six CUDA tests skip and three known missing-Triton factory tests are deselected. This includes the existing numerical/update/recovery contracts, actual argument resolution, captured deployment/learner resource options, and family/algorithm/resume recipe cases. The initial one-case fixture failure is retained; the fixture omitted a field normally resolved by common validation. Ruff, formatting and source diff checks pass.

Scope Parent #13 Resident candidate Parent/current speed ratio
Native single-GPU resident profile Rejected CPU placement/argument checks pass Undefined until actual runs
Matched official full RL NOT_RUN NOT_RUN —
Whole-device peak memory NOT_RUN NOT_RUN —
Held-out reward time-to-quality NOT_RUN NOT_RUN —

Actual shared-device Ray/CUDA startup, capacity, complete training and fresh-worker recovery remain unqualified. A compatible existing environment and full resource handoff are required for those runs. Critic, offload, multi-rank learner, release-train and recovery are outside this initial resident profile; bare --colocate still selects offload defaults and is rejected.

Report: docs/native_colocation_results.md. Tested source hashes and raw receipts are retained in benchmarks/results/native-colocation-cpu/.

Fit both CPU and GPU scheduling reservations into one placement bundle and report the shared device offset. Expose an explicit actor-only resident recipe and retain complete CPU validation receipts.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant