[0z5a]

Incremental review: Files changed.

Base: 0z5a-branch at 791f101e20db377b28e15f12a00863baaea2605f.

Purpose

Integrate Ouro's physical shared layers with VIME's existing Megatron GRPO optimizer, loss, distributed checkpoint and HF conversion paths. Rollouts use the companion engine's public generation and transactional full-weight publication contract for local and real prefill/decode engines.

The provider replays full-depth prompt prefill and each decode token's actual execution depth, including the last-exited KV state at deeper planes. Physical parameters retain their checkpoint names across MCore/DDP wrappers; recurrent loops do not duplicate parameter storage. examples/ouro/run.sh provides a multi-iteration launch and fresh-process resume recipe.

Target: 0z5a-branch at 791f101e20db377b28e15f12a00863baaea2605f. The submitted diff contains the implementation and user recipe: 17 files, 870 additions and 28 deletions. Added tests, validation harnesses, raw logs and coordination artifacts have been removed. Their results are summarized here.

Coordination and dependency

Companion ThinkFlowLab/vllm-rlt#78 remains a draft. Following Minhao's PR 1 design, VIME uses full four-loop prompt prefill, actual decode-depth replay, selected-token processed scores and returned effective seeds. The reduced-prefill prototype is superseded.

The reference currently exposes log_probs and per-request logprobs_mode. The proposed upstream API uses logprobs and engine-level raw_logprobs/processed_logprobs, with weight publication in PR 2. Adapting to and testing Minhao's exact implementation remain pending availability of that branch.

Validation

Check	Thor local engine	Two A100 GPUs, real PD
Full Ouro-1.4B training and fresh-process resume	Five continuous eager updates vs four graph updates plus resumed fifth: all 272 checkpoint tensors and Adam metadata exact	Three continuous eager updates vs two graph updates plus resumed third: all 272 exact
Generic HF export	All 269 physical tensors exact	All 269 physical tensors exact
Training through inference features	Early exit, delayed async and self-speculation each complete two full-model updates	Delayed async, NIXL transfers, committed worker versions and cooperative shutdown

After porting to the target branch, the mixed-depth probability/recomputed-gradient CPU check and physical-parameter HF round trip pass on the unchanged Thor runtime. Changed Python files pass Ruff, Black and isort checks. Five full-model K=2/3/4/early/delayed cases exactly replay tokens and selected scores from returned entropy seeds.

Actual-depth replay reduces early-exit actor/inference score MAE from 3.622689 to 0.011142; fixed K=4 still has MAE 0.026607. Training/inference score equality is not asserted for the full model.

Speed comparisons

Full Ouro-1.4B warmed rollout medians, five alternating-order pairs, two prompts, 32 response tokens, temperature 0.9 and fixed seeds; initialization and capture excluded:

Rollout	Eager (s)	Graphs (s)	Speedup	Time saved
K=2	1.0689	0.9939	1.076×	7.02%
K=3	1.5190	1.4207	1.069×	6.47%
K=4	1.9414	1.8234	1.065×	6.08%
Self-speculative K=4 target / K=2 draft	1.4952	1.3741	1.088×	8.11%

Real full-model GRPO component	Eager median (s)	Graph median (s)	Speedup
Thor rollout	1.8644	1.8612	1.002×
Thor training	10.9735	11.7205	0.936×
Thor publication	0.0287	0.0285	1.007×
Thor whole step	12.8710	13.5940	0.947×
A100 PD whole step	62.1185	62.2978	0.997×

Thor component medians use matched updates 1–3 at K=3/4/2, four prompts × four samples and 16-token responses. A100 medians include first-use capture and fresh resume under existing GPU contention. Whole-training acceleration and reward convergence are not demonstrated.

The A100 runtime's default UCX GPU IPC probe failed; passing PD runs use command-scoped UCX_TLS=tcp,cuda_copy with host-staged transport. BF16 bitwise comparisons require matched one-slot batches; differing dynamic batch shapes fail strict parity. TP/PP/CP=1 is required by this recipe.

Launch, parameter conversion and resume instructions. Completed task-owned model/checkpoint/export files are cleaned after checks. No process termination, environment updates or lcpu NFS changes were performed.

Historical layer review: provider #3 → budget recipe #4 → public contract/conversion #5. This upstream PR is the consolidated diff against 0z5a-branch.

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #34876111 | 📁 Comparing a13f24a against latest (791f101)

  🔍 Preview build  

46 files changed · + 1 added · ± 45 modified

+ Added

• _examples_synced/ouro/README.html

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/fault-tolerance.html
• advanced/low-precision.html
• advanced/megatron-config.html
• advanced/observability.html
• and 35 more...

[gemini-code-assist]

Code Review

This pull request introduces Ouro shared-layer training documentation, validation reports, and associated utility scripts for model weight conversion, checkpoint comparison, and cleanup. The feedback on the newly added validation scripts focuses on improving portability and compatibility, specifically by avoiding hardcoded absolute paths and virtual environment python paths, and ensuring Path objects are explicitly converted to strings when passed to PyTorch distributed checkpoint APIs.

[gemini-code-assist]

The path root is hardcoded to an absolute path /workspace/0z5a/work/ouro-vime-contract-20261001. This makes the script non-portable and prone to failure in other environments. Consider dynamically resolving the repository root relative to the script's location.

Suggested change

	root = Path("/workspace/0z5a/work/ouro-vime-contract-20261001")
	root = Path(__file__).resolve().parents[3]

[gemini-code-assist]

dcp.FileSystemReader expects a string path. Passing a Path object directly may cause compatibility issues depending on the PyTorch version. It is safer to convert the Path object to a string.

Suggested change

	metadata = dcp.FileSystemReader(folder).read_metadata()
	metadata = dcp.FileSystemReader(str(folder)).read_metadata()

[gemini-code-assist]

The checkpoint_id argument expects a string path. Passing a Path object directly may cause compatibility issues depending on the PyTorch version. It is safer to convert the Path object to a string.

Suggested change

	dcp.load(state, checkpoint_id=folder, planner=planner)
	dcp.load(state, checkpoint_id=str(folder), planner=planner)

[gemini-code-assist]

The root path is hardcoded to /workspace/0z5a/work/ouro-vime-contract-20261001. This makes the script non-portable. Consider dynamically resolving the repository root relative to the script's location.

Suggested change

	root=/workspace/0z5a/work/ouro-vime-contract-20261001
	root="$(cd "$(dirname "${BASH_SOURCE[0]}")"/../../.. && pwd)"

[gemini-code-assist]

The virtualenv python path /venv/main/bin/python is hardcoded. This will fail on systems where the virtualenv is located elsewhere or named differently. Consider using the active environment's python directly.

Suggested change

	/venv/main/bin/python compare_checkpoint.py train-full-pd-eager/checkpoint/iter_0000002 \
	train-full-pd-graph/checkpoint/iter_0000002 full-pd-checkpoint.json >full-pd-compare.log 2>&1
	/venv/main/bin/python check_hf_export.py train-full-pd-graph/checkpoint/iter_0000002 \
	train-full-pd-graph/export full-pd-hf.json >full-pd-hf.log 2>&1
	/venv/main/bin/python cleanup_a100.py full
	python compare_checkpoint.py train-full-pd-eager/checkpoint/iter_0000002 \
	train-full-pd-graph/checkpoint/iter_0000002 full-pd-checkpoint.json >full-pd-compare.log 2>&1
	python check_hf_export.py train-full-pd-graph/checkpoint/iter_0000002 \
	train-full-pd-graph/export full-pd-hf.json >full-pd-hf.log 2>&1
	python cleanup_a100.py full