Sync/validate GitHub main 3dbe2529 - #292
Merged
Merged
Conversation
NINGBENZHE
requested review from
Aurelius84,
Yangruipis and
yxyOo
as code owners
August 28, 2026 13:22
…gentic rollouts - KV-A3 program-aware admission: request-boundary Bypass/Admit/Defer gated by a cluster-wide execution-token budget (single-writer AdmissionBudgetCoordinator), with aging/forced-resume anti-starvation and fail-open to the existing limiter. admission never selects a worker or interrupts in-flight decode. - KV-A2 session lifecycle: top-level session_id on generate + idempotent /close_session fan-out to engines on finalize; requires server session-radix, fail-open (no-op) otherwise. Full replay payload keeps generation correct. - TP-aware /metrics aggregation (max, not sum) so TP-replicated gauges don't deflate usage / inflate the ceiling; usage prefers the token_usage gauge. - In-window running peak/mean KV usage so per-step metrics reflect true peaks. - Admission/lifecycle metrics in the rollout logger. - CPU-only tests covering admission decisions, the budget ledger, defer/resume, close fan-out, fail-open, TP-gauge un-sum, and the peak-usage window. Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit f2ef13b1d41e3d40216a89982be1e8b4c049ee07)
…trypoints
# ✨ Features
Bring the GPU image pipeline's layered design to Ascend/910C so 云效 only
selects code version, params, and scheduling while the Relax repo defines how
images are built.
- **`docker/Dockerfile.npu`**: refactor the monolithic NPU build into multi-stage
`base → train → relax`, mirroring `docker/Dockerfile` (GPU).
- `train` holds the Relax-code-independent heavy deps (torch_npu source build,
MindSpeed/Megatron/MindSpeed-Bridge/Megatron-Bridge, sglang-npu, sgl-kernel-npu)
and only `COPY docker/npu_patch` — a Relax code change no longer rebuilds them.
- `relax` copies the full checkout (incl. `.git` for versioneer) and installs it;
`ray`/`protobuf` pins stay after `pip install -e .` so they still win.
- `BASE_IMAGE` / `SOC_VERSION` / `TRAIN_IMAGE` are now build args (default base
`quay.io/ascend/cann:8.5.1-a3-ubuntu22.04-py3.11`, SOC `ascend910_9391`).
- A plain `docker build -f docker/Dockerfile.npu .` still works via the chained
default `TRAIN_IMAGE=train`.
- **`Makefile`**: add `docker-train-ascend`, `docker-dev-ascend`, and optional
`docker-qs-ascend`, reusing the GPU tag/push/remote-skip logic.
- Tags `ascend-train|dev|qs-YYYYMMDD-<hash8>`; the `ascend-` prefix keeps aarch64
artifacts from overwriting the amd64 `train-/dev-` tags in the same repository.
- `docker-qs-ascend` builds relax-ci's `Dockerfile.qs` (verified pure-python /
arch-independent) on top of the dev image via `ASCEND_QS_DOCKERFILE`; cloning
relax-ci and its credentials stay in CI, not the repo.
# 📝 Docs
- **`docker/npu-training.md`**: document the multi-stage layout, the Makefile build
paths, configurable variables, and the optional QS wrapping.
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 71f7a80c43dadf189c010c3ce5f5748784af5f21)
(cherry picked from commit 00900d0ecae7325bcf6aa5d2e3c3752c2a053361)
# ⭐ Feature ## Train only Qwen3.5 MTP parameters - Add `--mtp-only-training` to enable MTP, freeze non-MTP parameters before DDP and optimizer construction, and bypass the main language-model loss - Validate distributed trainable parameter sets and preserve the MTP auxiliary-loss autograd path - Skip unused rollout weight management for pure MTP-only SFT runs - Add an 8-GPU Qwen3.5-122B-A10B MTP-only SFT recipe with a 3e-5 default learning rate --- # ✅ Tests ## Cover MTP-only argument and training behavior - Test parameter freezing, distributed validation, output-layer bypass, zero-loss anchoring, and weight-management gating - Preserve SFT chunked-logits coverage after moving the runtime gate (cherry picked from commit 3a20b1b03c7aaaace11281223594a52b94ebd44b)
(cherry picked from commit aab61aa8e48f1b016bc509ee93d0555e85bf7a3d)
- **Add `run-qwen35-4B-4xgpu-ppo-colocate.sh`**: colocate (sync) PPO
training script for Qwen3.5-4B on 4 GPUs, adapted from
`run-qwen35-9B-8xgpu-ppo.sh` for fast CE / end-to-end smoke runs.
- Swaps in the `qwen35-4B.sh` model config and scales resources to
`[1, 4]` for actor/critic/rollout with TP=2 (two 2-GPU SGLang engines).
- Preserves the full PPO wiring: GAE, KL-loss path enabled at coef=0,
`--offload-rollout`, critic warmup, and CPU-only advantages `[1, 0]`.
- Updates checkpoint paths, experiment, and log names to the 4B/4xGPU
variant.
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit daf6b8cde0792b7864a3f54bb885467e00469d8f)
…sglang v0.5.15 (cherry picked from commit f97970800ec24134b9e88f1e5806a86f45d07965)
(cherry picked from commit 7320657364ccdca6d5dd7fb4426755589b4d9753)
- Read endpoints and credentials from standard AWS environment variables - Remove private S3 transport flags from Relax argument parsing - Preserve provider-specific model source injection behind the plugin boundary --- - Document standard environment configuration in English and Chinese --- - Verify generic S3 model source parsing and endpoint fallback behavior (cherry picked from commit 9ed5d241d7e4705ed0012d558815992227bc3b03)
- Add RELAX_EAGER_NCCL_INIT env (default on) - Monkey-patch dist.init_process_group to bind the current device to the default NCCL process group, enabling eager NCCL initialization --- - Add _enable_draft_weights_cpu_backup helper deciding backup based on enable_mtp_training and speculative_algorithm (with overrides) - Stop unconditionally enabling enable_draft_weights_cpu_backup in both GenRM and policy rollout server args --- - Add parametrized test for _enable_draft_weights_cpu_backup covering MTP, speculative algorithm, and override combinations (cherry picked from commit 06c38e1f39553eb04d603905fc8e615cd952c042)
# ⭐ Feature ## Add native classification SFT - Support binary, multiclass, and multi-label objectives through the Megatron SFT pipeline - Add classification data validation, task heads, losses, distributed evaluation metrics, and checkpoint handling - Provide Qwen3.5 full-parameter and LoRA training recipes with dataset preparation tooling ## Add classification export and serving - Export Megatron checkpoints with merged adapters and sequence-classification metadata - Add Qwen3.5 SGLang adapters, text-only server launcher, and exact-tokenization request client --- # ✅ Tests ## Cover classification paths - Test dataset processing, transfer fields, head registration, losses, evaluation, export, and SGLang adapters --- # 📝 Documentation ## Document classification SFT design - Describe supported task contracts, distributed behavior, validation datasets, training recipes, export, and serving (cherry picked from commit 9d252f9b255e3749f05e469384426455df5001a2)
- Allow one agent session to export multiple training samples under one shared logical identity. - Preserve the identity across rollout transfer, DP placement, minibatch construction, microbatch packing, loss reduction, and scheduler accounting. - Admit an identity only after all of its physical rows are ready, while allowing sibling rows to cross DP ranks and microbatches within the same optimizer window. - Use the complete identity loss-mask sum as the sample-mean denominator, including CP execution where numerators are local but the denominator remains global to the identity. - Align PP stages with dummy microbatches that contribute no gradients or metric denominator. - Count logical identities rather than physical rows for batch quota and scheduler progress. - Support per-export scalar custom advantage and whole-group filtering without copying task outcomes onto auxiliary samples. - Preserve the original M=1 single-context behavior and keep SFT on its existing sampler and training path. --- - Separate prepare, runtime, reward, and transfer ownership while sharing one resident-group capacity. - Replace prepare-pool parameters with train/eval concurrency and an explicit prelaunch switch. - Launch groups on demand by default and replenish groups dropped during runtime or reward processing. - Preserve valid resident sessions across partial and fully asynchronous rollout steps. - Adapt the existing program-admission and SGLang session-lifecycle implementations to the resident-group and multi-IR SessionForest state machine. - Separate original task outcomes from normalized training rewards so auxiliary samples may omit `Sample.reward`. --- - Budget multimodal generation with processor-expanded training-token lengths while retaining tokenizer-token prefixes for SGLang inference. - Parse explicit JSONL output only on LF so Unicode line separators remain valid inside JSON strings. - Keep partial-rollout backend attempts uniquely addressable and avoid duplicate request IDs after resume. - End empty streaming windows without synthetic dummy batches and fail fixed logical windows that finish below quota. - Own complete agent process groups through the node-local launcher and keep cleanup failures non-fatal to training. - Preserve rollout outcome, metadata, pass-rate, timing, staleness, logical-batch, and performance metrics. - Add Search-R1 as the reference vanilla/multi-agent recipe with role-aware credit, main-only evaluation, retrieval service, and data tooling. - Run WebShop as one cluster-shared server with thin per-session agents and an isolated server environment. - Prepare a reusable node-local DeepEyes V2 environment for host-side Jupyter control. - Harden mini-swe-agent Apptainer cleanup and update NeMo Gym recipes for the current concurrency interface. --- - Explain multi-context export, logical identities, custom advantage, resident capacity, prelaunch, and cross-step retention. - Document how the existing program-admission and SGLang KV-lifecycle features interact with the refactored runtime. - Refresh the bilingual Agentic guides, diagrams, and landing pages. (cherry picked from commit b19f34b48026d8c387c5ff6dc52a963c308e70bf)
# ✨ Feature ## Environment & app - Add `search_qa` agentic environment under `agentic_opd/`: multi-turn search-and-answer agent (`app/agent.py`), env loop (`app/env_search.py`), prompt templates (`app/prompt.py`) and FAISS retrieval client (`app/retrieval_client.py`). - Add `reward_search.py` (exact-match / score reward for the QA task) and `prepare_data.py` for building the Search-R1 prompt set. ## Retriever - Add standalone FAISS retrieval server (`retriever/retrieval_server.py`, `torch_retrieval_server.py`), corpus download (`searchr1_download.py`), launch script (`retrieval_launch.sh`) and a watcher that starts retrieval only after Relax finishes model load (`start_after_relax_model_load.sh`). ## Run scripts - Add GRPO training scripts for Qwen3-1.7B / Qwen3.5-4B and a checkpoint eval script, plus the mopd_agentic GRPO 1.7B launcher. # 📝 Docs - Add `README.md` and `design.md` describing setup, retriever bring-up and the 8-GPU shared-pod eval flow. Internal proxy IPs and `/mnt` paths replaced with placeholders. Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit cf6a4f07338219a76a55003619a4553b02e62aee)
- Add `_QWEN38_PRESERVE_HISTORY_GATE`, the third gate form shipped by Qwen3.8 (model_type qwen3_5) that preserves historical thinking by default (undefined/true keeps it, explicit false drops it) - Recognize the gate as-is without rewriting the template, since it already matches the auto-preserve resolution behavior - Extend the ambiguity guard to report the qwen38 gate count so the RuntimeError no longer trips on the new form --- - Add `scripts/models/qwen38-27B.sh` model args (dense, 64 layers, hybrid attention, gated attention, ZeroCenteredRMSNorm) - Add MTP SFT script on pokemon-gpt4o-captions (8xGPU single-node) - Add DAPO math colocate training script (8xGPU) --- - Assert the preserve-by-default gate is recognized without rewrite - Assert explicit `preserve_thinking=False` disables auto-preserve (cherry picked from commit 6224e8831e7562c680ae5dee71405f9bee2608ba)
- Implement M2PO (Second-Moment Trust Policy Optimization, arXiv:2510.01161, NeurIPS 2025) as a first-class advantage estimator - `compute_m2po_loss` + helpers in ppo_utils: adaptive eps* engages only when the harmful-token second moment M2 exceeds budget, otherwise degenerates to GRPO; floored at miniclip so it is never tighter than GRPO - Wire `--advantage-estimator m2po` through registry, arguments, advantages and the Megatron loss path; add `--m2po-kl2-budget` / `--m2po-miniclip-low` / `--m2po-miniclip-high` and m2po_eps metrics - Add an 8xGPU Qwen3-4B run script (cherry picked from commit dd7ae625f1c5b02cf4a4c0fe7d12997a22523ee5)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3dbe2529d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+3173
to
+3180
| if has_encoder is None: | ||
| logger.warning( | ||
| "Skipped the --lora-scope check for --lora-adapter-mode (HF config unreadable). " | ||
| "If %s is a VL/omni model, pass --lora-scope language: SGLang cannot host a " | ||
| "vision-tower adapter and would silently drop it.", | ||
| hf_checkpoint, | ||
| ) | ||
| return |
There was a problem hiding this comment.
When --hf-checkpoint is an S3 model source, argument parsing has already rewritten args.hf_checkpoint to the raw s3://... URI, which AutoConfig.from_pretrained() cannot inspect. This branch therefore warns and returns, allowing --lora-adapter-mode --lora-scope all for VL/omni models even though this function documents that SGLang silently drops their vision/audio adapters. Such runs train a policy different from the rollout policy and corrupt the on-policy signal; materialize/read the source metadata before this check or fail closed for an uninspectable remote model.
Useful? React with 👍 / 👎.
# ⭐ Feature ## Add Qwen3.5 9B math SFT launch script - Add an 8-GPU Qwen3.5-9B math SFT script for LoRA training with dynamic context parallelism. - Keep dynamic CP enabled by default while allowing `CP=<int>` to force a static context-parallel size. - Configure chunked SFT logits, recompute, TP=2, GBS=64, max context 32768, and LoRA rank 32. # ✅ Tests ## Validate script configuration and training log - Ran `pre-commit run --all-files --show-diff-on-failure`. - Checked the MR diff with `git diff --check origin/dev...HEAD`. - Reviewed the training log and confirmed steps progressed without traceback, OOM, NaN, or Inf. (cherry picked from commit 3b8a719daf29d4a5b3d58e762131f67fd30f7620)
# ⚡ Performance ## Reduce vocab-parallel activation memory - Compute log-probability and entropy through one fused autograd path - Preserve entropy metrics while skipping entropy backward state when its coefficient is zero - Reuse full-vocabulary buffers to avoid redundant logits clones and temporary products --- # 🐛 Bug Fix ## Update Megatron loss call sites - Route policy and chunked SFT paths through the fused API - Remove the superseded standalone log-probability and entropy helpers --- # ✅ Tests ## Cover numerical, gradient, and distributed behavior - Add CPU and Gloo tensor-parallel parity tests - Add CUDA and NCCL parity tests plus Relax entropy-gradient wiring coverage --- # 📝 Documentation ## Add validation requirements - Document runtime, distributed, and memory acceptance criteria for environment testing (cherry picked from commit 89b43c0467114f78fc66a2abf56cea9af0ea2dda)
`save_hf_model` derived strictness from MTP alone: `strict = not
allow_missing_mtp_keys`, which is True for any base without MTP layers. That
breaks every export whose training model structurally cannot emit some weight the
reference declares, because Megatron-Bridge under strict=True refuses each shard
that contains such a key -- the real tensors sharing those shards are lost too,
so the failure is a truncated checkpoint rather than a missing submodule.
A VL base trained text-only hits exactly this. With
`GEMMA4_CONVERSION_MODE=text` the provider builds no vision tower, so the 356
vision-tower tensors of the reference can never appear. Measured on
gemma-4-26B-A4B-it: strict=True wrote 58 of 657 language tensors and raised
`955 tensors from the original checkpoint were not written`.
MTP already had this exact shape of exception, so give vision the same one and
decide strictness from both:
model_has_vision = hasattr(get_model_config(model[0]), "vision_config")
allow_missing_vision_keys = reference_expects_vision(...) and not model_has_vision
strict = not (allow_missing_mtp_keys or allow_missing_vision_keys)
Only the VL providers declare `vision_config` (verified: Gemma4ModelProvider and
Gemma4DenseProvider do not, Gemma4VLModelProvider and Gemma4DenseVLProvider do),
and `Gemma4VLModelProvider.provide()` builds the VL model unconditionally, so the
field's presence -- not its value -- is the structural signal. Reading the model
config also keeps this correct however the conversion mode was selected: env var,
bridge attribute, or the recipes' context manager.
Every other export keeps strict=True. It is the only export-time guard against a
genuine conversion bug silently truncating a checkpoint, so this deliberately
does not widen to "all BF16 exports": with strict off, Bridge logs one non-fatal
`Error:` line and the export still reports success, which is indistinguishable
from a legitimately partial one.
The reconcile call is no longer gated on `allow_missing_mtp_keys`, since ghost
index entries can now arise from the vision case as well; it stays a no-op when
there is nothing to fix. MTP weights are still supplemented from the base model,
vision weights deliberately are not -- a text-only export stays text-only.
Verified on 8xL20Y, gemma-4-26B-A4B-it text-mode SFT: all 657 language tensors
written, index of 657 keys with 0 ghost entries, and both shards byte-identical
to `scripts/tools/convert_torch_dist_to_hf_bridge.py` output from the same
checkpoint (total_size 50466283580).
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 47a265a6bc4e221cd8e0424b0ff80d5f44fcd19e)
- Slice shifted SFT masks with predictor offsets under context parallelism - Keep RL response masks in target-token coordinates - Use the same mask slice for loss reduction and token normalization --- - Verify SFT span-end supervision for THD, padded THD, and BSHD layouts - Verify RL mask semantics remain unchanged (cherry picked from commit 1babcea73c00aac8dbe2680e48fa08d34ddf09b3)
# 🐛 Bug Fix ## Preserve nonempty whitespace content - Accept nonempty whitespace-only string message content - Apply the same validation to text items in list content - Continue rejecting empty strings while preserving assistant tool-call semantics (cherry picked from commit 86a6483e474e3c84c16611fcde6caa379ca2bec3)
# 🐛 Bug Fix ## Restore zero-copy multimodal transport for single-node TP - `_determine_tensor_transport_mode` keyed "cross node" off `dist_init_addr`, but single-node multi-GPU TP must also set it to bootstrap the TP process group - The false positive forced the CPU "default" transport, which makes `wrap_shm_features()` a no-op, so multimodal pixel tensors were pickled and gloo-broadcast per request (measured 2.2-6.5 GB payload, 18-33 s per `recv_requests`) - Key off `nnodes > 1` instead, so only real multi-node falls back to CPU transport - Verified on Qwen3.5-9B MOPD with mmlongbench multi-image data: `recv_requests` total 18-33 s -> 0.31-0.41 s, GB-scale `broadcast_pyobj` gone, cost shifts to the SHM path as expected ## Harden MOPD multi-image recipes against scheduler starvation and teacher timeout - Set `SGLANG_SCHEDULER_MAX_RECV_PER_POLL=4` and propagate it to Ray actors, bounding per-poll SHM feature clone work on the scheduler event loop - Raise `--opd-teacher-timeout-s` default to 1200 s; multi-image teacher logprob fetches re-prefill ~45K-token contexts and timed out at the previous 120 s / 600 s defaults (cherry picked from commit b3a3e8a8383b640bd21fbd066831de942a709669)
# 🐛 Bug Fix ## Preserve partial-rollout protection boundaries - Record threshold hits as pending protection after an abort - Activate session-wide protection on the next rollout resume - Restore agent timeouts before reopening generation and dispatching requests - Keep the generation gate closed when abort handling fails (cherry picked from commit 85f5b1e46e096e6874f6179ecb0f235c6b2bb0a5)
# 🐛 Bug Fix ## Fix multimodal sequence classification samples - Use processor-expanded prompt tokens directly for sequence classification - Keep causal language model loss-mask alignment validation unchanged - Preserve sentinel supervision for multimodal classification samples (cherry picked from commit 2d495e228b6b18a3a314d6eac98f9ffa29380682)
# ⭐ Feature ## Fail-closed NCCL precheck before scale-out weight sync - Add an independent, same-topology NCCL probe (`scale_weight_sync_precheck`) run between the seed engine and each new elastic engine BEFORE touching the real ModelRunner communicator. The probe launches standalone subprocesses on the engine actors via the Ray layer; it never mutates sglang ModelRunner groups and calls no sglang HTTP endpoint. - Compare both sides' transport fingerprints (IB/RoCE vs socket, ifname, `NCCL_IB_*`). An incompatible transport (`wrong type 3!=4`) fails closed so a bad elastic node can never pollute the seed engine or trigger a global restart. Only bind/launch-transient categories are retried (bounded attempts). - Independent bounded rendezvous port window (18000-20000), disjoint from the real direct-sync window, with wrap-around cursor allocation. - Gated by `RELAX_SCALE_WEIGHT_SYNC_PRECHECK` (default on); applies to ray-native scale-out only (external-engine path is skipped). ## Persist precheck-failure artifacts for post-mortem - On failure, dump both sides' transport fingerprints + per-rank NCCL log tails to a durable file (`RELAX_SCALE_WEIGHT_SYNC_PRECHECK_LOG_DIR`) so an incompatible scale-out can be root-caused after the elastic node is reclaimed. Best-effort: persistence never masks the precheck verdict. ## Propagate scale-out failure root cause to error_message - Canonicalize each per-replica failure into a stable, non-sensitive category (drops replica index / traceback / address / secrets) so the monitor TUI can bucket it; aggregate deduped categories (bounded length) into `ScaleOutRequest.error_message` across every failure policy. Each category carries a Chinese comment explaining its meaning and source. --- # 🐛 Bug Fix ## Bound & self-heal scale-out weight-sync port allocation - `get_free_port` / `_get_current_node_ip_and_free_port` take an optional `max_port` so allocation stays inside a bounded window instead of scanning into the OS ephemeral range, raising instead of colliding with sglang's port range. Default `None` preserves existing unbounded behaviour. --- # ✅ Tests - `test_scale_weight_sync_precheck`: transport asymmetry rejection, bounded retry, independent port window, actor-probe device isolation / timeout / unreapable-child join bound / seed-failure lock release, and failure-artifact persistence (both fingerprints + log tails). - `test_weight_sync_port_hardening`, `test_scale_out_registration_order` (reason sanitization / no address leak), plus conftest helpers. (cherry picked from commit 4c92237e6daabb820bf709dbbf36120309a17fc3)
sglang 0.5.15.post1 always routes Qwen3_5AttentionDecoderLayer.self_attention through forward_prepare_cuda_fused when attn_output_gate is set on CUDA. Its fused_qk_gemma_rmsnorm_rope_gate kernel indexes positions as a flat [T] tensor, but multimodal requests on Qwen3.5/Qwen3.6 pass mrope positions shaped [3, T], so only the temporal row is applied across the whole rotary dim. No error is raised -- the output is silently corrupted (degenerate repetition, bbox coords out of range). This is what broke qwen3.5 VL grounding when the image moved from sglang 0.5.12.post1 to 0.5.15.post1 (relax image 20260804 and later). Backport of upstream sgl-project/sglang#31806 (not merged upstream as of 2026-08): route 2-D positions to forward_prepare_fused_gate; text-only (1-D) positions keep the existing CUDA fused fast path. Also fixes the hunk header at line 401 (@@ -1322,6 +1331,20 @@ -> +1331,22). The hunk body carries 22 new lines, so git apply rejected the whole file with "corrupt patch at line 424" and none of v0.5.15.post1.patch could be applied. (cherry picked from commit 465ba553795808a5d6a6d6e6d457d9df519249f5)
- Defer local existence checks for a reference checkpoint that aliases the configured model source - Preserve hierarchical model URIs during path normalization - Reject distinct remote reference URIs with an actionable validation error --- - Verify same-source provider resolution reaches argument validation without local path checks - Preserve local reference and distinct INT4/BF16 checkpoint behavior - Cover URI spelling, malformed authorities, and mismatched remote references (cherry picked from commit 0ae9a127f72fc2a7967d35fab161ad16fd3ebae8)
# 📝 Documentation ## Add a runnable Qwen3-4B elastic rollout path - Add the fully async training script and minimal Autoscaler configuration - Explain the required elastic flags and resource relationships in both languages - Provide copyable launch commands, success checks, and graceful eviction boundaries - Link directly to advanced scaling configuration and troubleshooting (cherry picked from commit 81e577689e15661d839240902b4fe9b296582b5d)
- Replace TidalCache URI examples with a synthetic test scheme - Use a generic provider name for argument parsing coverage (cherry picked from commit a024abcaa7dde33172d27a88e2b157385885695d)
# ⭐ Feature ## Add opd-tuning skill for OPD/MOPD teacher-side tuning - Cover the four OPD-specific areas: teacher SGLang engine knobs, logits memory, student/teacher GPU split, and multimodal caches - Scope it away from `perf-doctor` (actor/training-side perf) and `debug-hang` (hangs), so the three skills do not overlap ## SKILL.md - Encode the dependency-ordered decision flow: logits chunking -> prefill budget -> mem-fraction -> cuda graph -> VLM cache -> GPU re-split, with an explicit note that reordering steps 1 and 2 OOMs - Require hardware and model facts to be read (`nvidia-smi`, `config.json`) rather than assumed, and marked `(assumed)` otherwise - Add a report template, apply-time editing rules, and a prohibition list covering fabricated flags and non-existent metrics ## references/teacher-knobs.md - Add the R-T01..R-T10 rule catalog derived from one causal chain: the teacher is a pure `max_new_tokens=0` prefill - Give three sizing formulas (logits peak, per-token KV, per-image embedding) instead of hardware-specific constants - Add four self-verification recipes: locate the sglang the runtime actually imports, build the real `ServerArgs` parser to check a flag (grep gives false negatives on the generated dataclass flags), check the mirrored `--teacher-sglang-*` form against the skip list, and look up env defaults ## references/balance.md - Document how to measure the student/teacher split three ways, and warn that Ray log deduplication makes line-summed metrics unusable - Give the GPU-split formula plus the divisibility constraints that make the optimum usually unreachable, and the overshoot failure mode - Record that in-place student-offload / teacher-swap is not implemented, which primitives are missing, and its four blockers ## references/baselines.md - Snapshot the OPD example scripts' teacher configuration, with a refresh command so the snapshot self-corrects - Add "obviously untuned" detection patterns that outlive the values --- # 📝 Documentation ## Keep the skill environment- and revision-independent - Cite code by file plus symbol name, never by line number - State runtime facts as verification recipes rather than pinned sglang versions or image-specific source paths (cherry picked from commit 866d5403791b6e92da14b217d9467d2d97e7a798)
(cherry picked from commit 097c0b86a5064ddb3c4603066b54da27fcaeabdb)
# 🐛 Bug Fix ## Stabilize replayed tool-call state - Parse assistant `tool_calls[*].function.arguments` from a JSON object string or mapping during message validation - Serialize arguments as compact, key-sorted JSON so Python `json.dumps()` responses and TypeScript `JSON.stringify()` replays use the same SessionForest state identity - Convert finite integral floats within the JavaScript safe-integer range to integers, which also converts negative zero to zero - Reject malformed JSON, duplicate keys, non-object roots, non-finite numbers, non-string object keys, unsupported Python values, and strings that cannot be encoded as UTF-8 ## Reject empty terminal responses - Fail a terminal generation when it produces no tokens, decodes to empty text, or leaves no text, reasoning content, or tool call after response parsing --- # ♻️ Refactor ## Reuse validated session inputs - Remove repeated message and tool normalization inside the shard after HTTP request validation - Reuse normalized explicit export messages and tools when resolving their committed SessionForest state - Validate decoded assistant messages once, then use that same representation for the Forest response node and HTTP response payload --- # ✅ Tests ## Cover Python and TypeScript replay - Add one regression test that commits Python `json.dumps()` tool arguments, replays the same object in TypeScript `JSON.stringify()` form, and verifies that parent matching returns the committed response with only the tool result left as the unmatched suffix (cherry picked from commit 6bedff4eb4687c29884dbb05b409b19922971eed)
(cherry picked from commit 27dff4120e5b4e03c0bd48274c15173fb7fd7778)
(cherry picked from commit ae5ec78e06379c03ed660ce57cde8605199e03f7)
NINGBENZHE
force-pushed
the
sync/validate-github-main-3dbe2529
branch
from
August 28, 2026 14:06
3dbe252 to
3e2ed70
Compare
# ✅ Tests ## Keep CPU-only CI collection safe - Skip Megatron and SGLang backend tests before importing unavailable dependencies - Isolate agentic rollout and argument tests from optional backend modules - Preserve non-DCS control-plane coverage when the Megatron stack is unavailable
# 🔧 CI/CD ## Preserve controller test collection - Define sampler stubs as classes so identity-aware samplers can inherit from them - Initialize sampler state required by the Controller import path
# ✅ Tests ## Load controller without optional backends - Extract the Controller dependency stubs into a reusable core test helper - Reuse the isolated loader for control-plane data-source affinity coverage - Keep real DCS behavior tests separate from the import isolation path
Yangruipis
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Why
How
Testing
pre-commit run --all-filespassespytest tests/)Type of Change
Screenshots / Logs