feat(worker): supervise data-parallel group routing by thxCode · Pull Request #783 · gpustack/gpustack-operator

thxCode · 2026-10-23T14:13:40Z

What type of PR is this?

/kind enhancement
/kind api-change
/area worker

What this PR does / why we need it:

A healthy leader can keep a partial data-parallel group in Router discovery. Supervise each managed group as one serving unit. Withdraw every endpoint on a known member fault, and restore only after complete readiness, a verified group probe and a fresh Router observation for the current member generation.

Validate paired Prefill/Decode probes and keep each role's qualification independent.
Bind group identity and recovery to live Pod UIDs. Preserve previously qualified groups through observation loss only when fresh complete membership is unchanged and safe.
Project group eligibility into Services and mixed Router discovery. Require a Router for externally exposed DP roles.
Publish group membership, state, reason, qualification time and serving confirmation in status.roles[].groups[].
Cover withdrawal, partial recovery, replacement, observation loss and unaffected groups. Local HTTP fixtures check arithmetic, SSE, exact Chinese and JSON responses across recovery phases.
Update routing, External DP, P/D and status documentation; place Scaling after External DP in navigation.

Which issue(s) this PR links to:

NONE

Special notes for your reviewer:

Local validation passed: make test, go build ./..., whole-tree golangci-lint, make lint docs, generated-artifact checks and the full race suite (-shuffle=on, coverage enabled).
make generate completed twice with no generated changes. Commit-message and signoff checks passed; history tidying preserved the exact verified tree.
The make lint wrapper exits 2 after all 19 agents-shell self-test cases pass. Its cleanup attempts to delete the shell's current-directory ancestor, which this host's mavis-trash guard refuses. The Go and shell gates themselves passed.
Validation uses local tests and fake clients. Kubernetes, image, GPU and real-network acceptance remain separate and were not run.
ViaRoutingProxy qualification verifies a bound proxy response with the exact copied content while naming the prefiller. It does not prove individual DP rank coverage or KV transfer; Router serving counts come from a separate observation.
Independent zcode crosscheck completed. Its observation-loss finding was reproduced and fixed; a narrow follow-up verified partial-read safety and fresh-member withdrawal. Follow-up tests also caught a fresh member fault whose state and reason were masked by Router uncertainty; both now preserve the known readiness fault.

Does this PR introduce a user-facing change?

ModelDeployment routing now withdraws an entire managed data-parallel group when any member is unsafe. Recovery requires complete readiness, a verified group probe and current Router evidence. Status reports group qualification and serving state. Action required: External DP roles that expose a separate HTTP endpoint per rank must configure a Router.

- report the physical listener port and pod hint from the worker base URL - add an observer test for DP-aware workers on one physical listener - align the observation test fixture with per-rank worker URLs - document the single-host SGLang DP attention and P/D PoC boundary Signed-off-by: thxCode <thxcode0824@gmail.com>

Signed-off-by: thxCode <thxcode0824@gmail.com>

Add observeModelDeploymentGroupForwardPair to evaluate paired Prefill and Decode replica groups through either SGLang head-to-head Mooncake transfer or the llm-d routing proxy front door while failing closed on single-view P/D probes and unsupported shapes. Task 1 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

…eadiness Evaluate Prefill and Decode replicas through paired group-forward probes with independent role barriers, add group state and blocking reason classification, and withdraw carried endpoint eligibility when any group member is not Ready=True. Task 2 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

- Enable managed prefill and decode eligibility without widening role routing. - Cover group withdrawal, leader-only selection, and managed-command takeover. Task 3 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

- bind observations and retained eligibility to the current group generation - block restoration until a fresh coherent Router view is available - preserve mixed route discovery with atomic candidate labels - bound each observation fetch independently of the collection budget Task 4 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

- reject External DP roles that require a missing Router - publish generation-bound group state, members and serving confirmation - preserve qualification history and report elastic collective uncertainty - synchronize generated API schemas and apply configurations Task 5 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

- document the fail-closed group and serving status contracts - cover rank withdrawal, recovery, replacement and observation barriers - validate arithmetic, streaming, Chinese and structured response content Task 6 of dp-group-router-supervisor. Signed-off-by: thxCode <thxcode0824@gmail.com>

gpustack-code-review · 2026-10-23T14:19:50Z

✅ OpenCodeReview: Review complete: 0 finding(s) across 3 selected item(s).

Reviewed b6a63ce..1f0a92f only; earlier commits in this PR were reviewed in a previous run.

Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode · 2026-10-23T14:49:04Z

Review round 1: all four emitted findings were checked against the source (one inline, three routed to the summary).

Proxy evidence: kept the accepted ViaRoutingProxy qualification contract. Its bound response names the prefiller and returns the expected content, but does not prove individual DP rank coverage or KV transfer. Corrected the eligible group reason to expose that limit using the existing typed attempt shape. Qualification, states and serving counts are unchanged; counts come from the separate Router observation. The inline thread stays open because the requested stronger qualification policy was intentionally not adopted.
Retention documentation: removed the duplicate predicate from Routing and linked to the owning Group status contract.
P/D documentation: replaced repeated container measurements with a link to the owning SGLang DP routing section, preserving the Kubernetes and multi-host acceptance limits.
Generation reminder: make generate completed twice without artifact drift; the generated API output was already committed.

Validation includes a RED/GREEN status-level regression, the controller race package, Go lint/vet/gofmt, documentation lint and generated-artifact stability. The existing host-only make lint wrapper cleanup limit remains documented in the PR description.

Also applied the requested navigation correction: Scaling follows External DP in the sidebar and page index, with each Benchmark child retained under its parent. The rendered navigation and documentation gate pass.

An SGLang Server role that declares an attention data parallel width of two or more across two or more nodes under SGLANG_RUST_SERVER derives the External load balance, so every member of the group is published and the router balances across the members' own listeners. The vote reads the environment value the way the engine's own reader does, a value the engine would refuse votes nothing, and any lesser shape stays leader-served Internal. The attention width is a parse degree with no status slot and never enters the width check's multiplication. The role kind joins the derivation so every reader agrees on a role's shape. Signed-off-by: thxCode <thxcode0824@gmail.com>

Document the SGLang External DP serving shape: one distributed launch over the size member Pods, one attention data parallel group per member, SGLANG_RUST_SERVER giving every group its own listener. State the classification contract, the honest difference from vLLM (the FFN stays tensor parallel across members, withdrawal is group atomic and scale down runs the sixty second handoff grace), the limits, and the probe contract. Correct the stale sentence that claimed an SGLang role can only resolve Internal, name the RouterRequired admission rule, and mark every SGLang runtime statement as uncaptured until the acceptance run records it. Signed-off-by: thxCode <thxcode0824@gmail.com>

Add six self recovering benchmark cases for the runtime acceptance matrix: SGLang External DP with per member content checks, group scaling and the fault row; SGLang multi server behind the gateway; vLLM multi server content classes through the router; a vLLM prefill decode pair driven by an Active scaler with transfer evidence and a prefiller contribution precheck; a SGLang prefill decode arm reading its rendered transfer leg; and the vLLM External DP manual group scaling leg with the scale down grace verdict. None is executed yet: each header states it awaits the acceptance run. Signed-off-by: thxCode <thxcode0824@gmail.com>

Server side dry runs only: the external SGLang shape without a router is refused naming RouterRequired, the same shape is admitted with either router, and three control rows that each remove one fact of the classification trio stay admitted without a router, which is what proves the refusal is the external shape rule. Signed-off-by: thxCode <thxcode0824@gmail.com>

An engine's runtime caches default under the container's home and /tmp, which the pod's derived ephemeral-storage limit measures, so a replica that compiles kernels or falls back to a hub download is evicted while it is still starting. Render the four environment names every such writer honors onto subdirectories of the compile-cache host volume the operator already mounts: XDG_CACHE_HOME for the hub, torch hub and pip trees, FLASHINFER_WORKSPACE_BASE for the JIT workspace, TORCHINDUCTOR_CACHE_DIR and TRITON_CACHE_DIR for the compiled kernels. The names are defaulted, not owned: a role's own entry replaces one, an unknown engine gets none, and a role that took over its command line keeps its own environment. The render version moves to 2; version 1 pods stay compatible, and the golden fixtures re-baseline with unchanged spec hashes. Signed-off-by: thxCode <thxcode0824@gmail.com>

…ases Without an artifactRef the operator renders the model as the repository and the engine downloads the weights onto its writable layer, which the derived 15Gi ephemeral limit evicts mid-download while the warmed NodeModelStore cache sits unread. Bind the artifact the store caches and serve under the /models name, in the multi-server content cases and the sedp case, and align every request body to the served name the operator pins. Signed-off-by: thxCode <thxcode0824@gmail.com>

A prefill role keeps the requests it has accepted and not yet scheduled in the inflight queue, and its ordinary waiting queue is empty while that queue is not. The sampler read only the ordinary queue, so a busy prefill role read as one running request and nothing waiting, and the backlog the role was holding never reached the scaler. A replica that was added and never received a request exports no latency histogram. The sampler read the histogram's absence as a sign of trouble and held the whole role above its floor for ever, although the role's gauges and counters said that nothing was running and nothing was waiting. The prefill role's inflight queue is now read and added to the waiting count of a prefill role, the way the decode role's transfer queue is. The idle gate no longer asks for a latency histogram: a role that served nothing is idle on its gauges and counters, and an idle role is fully covered, so the floor return is no longer clamped away. The latency signal stays missing, which is what holds the algorithms that measure latency, and only the gauge path reads the role as idle. Signed-off-by: thxCode <thxcode0824@gmail.com>

The agents shell gate's self-test stands in the tree it made, and its exit trap removed that tree from inside it. A delete that refuses to remove a directory a process is standing in leaves the tree behind and fails the run that created it, so the gate reported its own self-test as untrustworthy on a host whose delete is recoverable. Signed-off-by: thxCode <thxcode0824@gmail.com>

…e series The directed pair probe discarded the answer it rejected, so its failure reason could not tell a model that drifted from a context that was never whole, and a rerun existed to answer a question the reason could have carried. The reason now names the answer, bounded to two hundred runes on rune boundaries, because the engines answer in more than one script and a bound that splits a character corrupts the evidence it carries. The pair observation recorded no mooncake series, so the push the prefill half counted and the pull the decode half performed were invisible to anyone reading the observation. Both halves' mooncake exposition is now recorded with the attempt, prefixed by the half, and the observation says plainly that the connector exports no read-side success counter - its failed-receive count covers failed pulls only - so the copied answer stays the read-side evidence and a reader cannot take the counters for it. The test exposition's queue gauge carries the decode engine type like its siblings; the model name there was a typo. Signed-off-by: thxCode <thxcode0824@gmail.com>

…s per rank The chat template wraps its answer in a <think>...</think> block on the vLLM chat route whatever the thinking switch says - the recorded rejection caught a byte-perfect copy sitting behind an empty wrapper. The pair probe's compare now reads the answer's visible text: a leading think block is stripped before the existing trim and exact compare, and the rejected answer stays raw in the failure reason, where the wrapper is still visible for the next classification. The wrapper is the template's noise, not the model's answer. The SGLang external-dp shape now names its width to the gateway. The gateway's --dp-aware fans its worker registration out per dp member, which is the one-worker-per-member contract the shape is held to; the flag is carried upstream as of sgl-project/sglang a9c97c9f69 and into our image by pack/llm-router/patches/sglang-gateway/0003. A width under two renders nothing, and so does every other engine. Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode · 2026-10-24T08:51:01Z

Runtime acceptance update (DP supervisor / SGLANG_RUST_SERVER External DP)

Seven-cell real-GPU matrix executed on a live Nebius mk8s cluster (8×H100), cluster destroyed with double proof afterwards. Full record: the run's FINAL-ACCEPTANCE report and issue ledger.

Verdicts: admission 6/6 · V-S full GREEN (content 0-fail + manual 1→2→1 leg 21/21, 16 requests split 8/8 across both workers) · S-S GREEN (content 4/4 + real worker-set 1→2→1; the −1 gauge semantics source-proven at gateway-v0.3.1) · S-PD GREEN 29/29 on the fixed image · V-EDP GREEN (60s drain grace measured at 70s) · V-EEP GREEN (resize 2→4 converged, served through the resize 0 errors, effective=stable width 4) · V-PD serving path GREEN (25 PD decisions, 75 transfers / 30.7 GB / 0 failed; the scaler rows follow from the case's RunningRequests config not counting its 23-deep waiting queue — case moved to QueueDepth, decision pinned by unit tests) · S-EDP content+grace GREEN, per-member gateway registration blocked by an upstream sgl-model-gateway defect (discover_metadata 10s step, no knob; upstream draft report prepared with our backport of a9c97c9f69 in-tree).

Fixes landed along the way (each with RED/GREEN tests and, where applicable, live re-verification): engine runtime caches off the writable layer (eviction fix) · scaler prefill-backlog read + idle-without-histograms floor return (pre-existing in main, byte-identical — not a branch regression) · group-forward probe names the rejected answer and records both halves' mooncake series · visible-copy compare (strips the template's think wrapper) · --dp-aware render for the SGLang external-dp shape + the gateway cherry-pick patch (compile-fixed) · the case-suite defect set (readers, clocks, freeze/cluster-shape discipline) · docs: captured results backfilled, version-applicability stated, routing by configuration combination, neutral dev-image guidance.

Open follow-ups tracked in the run ledger: the gateway discover-step timeout (upstream), the --dp-aware render gate now fixed onto the attention width, the surviving-group AwaitingObservation observation, and the unreproduced elastic wedge.

thxCode · 2026-10-24T16:54:09Z

Review round: the three open findings are answered.

InstanceType double dash (module page and walkthrough): verified false positive. pkg/nodefeature/helper.go generates names as gpustack----, and the acceptance-run manifests frozen with this PR scheduled successfully against the double-dash spelling. The single-dash examples on pages outside this diff predate the current unified-pool naming; re-checking them belongs with re-verifying those pages, not with this change. Both threads carry the reply and stay open for your judgement.
XDG_CACHE_HOME credential relocation (runtime cache): addressed by scoping the defaulted set to cache-only names - HF_HUB_CACHE, PIP_CACHE_DIR, TORCH_HOME beside the three kernel-cache names - so the Hub token, datasets and Xet caches stay under HF_HOME on the container filesystem. Render version advanced to 3; runtime-cache tests, the serialized-render digest pin and the legacy-render goldens re-baselined. Thread replied and resolved.
The round-1 ViaRoutingProxy thread keeps the answer it already carries (the published group reason now names the proxy-shape limit; the stronger qualification policy was intentionally not adopted) and stays resolved.

Verification on the tree being pushed: full pkg/worker suite, make lint, and make lint docs (83 site pages) all pass.

…le layer Signed-off-by: thxCode <thxcode0824@gmail.com>

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

…r the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode · 2026-10-24T18:12:10Z

History note: the branch was force-pushed once to add a missing Signed-off-by to one mid-stack commit (the gateway patch compile fix); no content changed, and every fixup and reply above still describes the same trees.

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

…r the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode · 2026-10-24T18:50:27Z

Review round 3: the thirteen findings from this run are answered.

Inline threads:

make generate reminder (API types): the regenerated output is present and the verify-generated CI job passes on the head. Replied, nothing to change.
pair-probe cost per pass: acknowledged as a real tuning question and left open - the full matrix is the qualification contract (every prefill replica needs pair evidence), so a cap decides which replicas wait, and that trade-off is the spec owner decision, not a review fixup.
deprecated --dp-size/--enable-dp-attention spelling: deliberate and documented - the external-dp page states the classification is keyed on the modern spelling only; covering the deprecated pair would be an unverified behavior change. Follow-up issue if wanted.
modelDeploymentEngineBool accepting yes/y: confirmed against SGLang v0.5.22 get_bool_env_var (truthy is exactly true/1) and fixed, with the shape test now pinning yes as Internal. Thread replied and resolved.
regression-test reminder for the admission gate: covering tests ship in the PR (model_deployment_test.go tables). Replied.

Summary-routed findings:

protobuf tag varint on Ready: false positive - the field at api/worker/v1alpha1/model_deployment.go:1390 already reads protobuf:"bytes,4,opt,name=ready", and verify-generated passes.
r.pairable computed per call: noted as a per-pass caching refactor; the support check runs once per answers call today. Deferred to the maintainer.
mooncake scrape on SGLang pairs: fixed - the pair engine table carries a Mooncake flag, the vLLM entry sets it, and an SGLang pair scrapes nothing; the recording test moved to the vLLM proxy path with an SGLang negative case beside it.
unparseable reply reported as unbound: fixed - the reply carries an Unreadable flag and its own failure reason, keeping an unreadable answer apart from an unbound one.
finish_reason demanding stop exactly: fixed - the copied leg now uses modelDeploymentQualifyFinishRan like the ordinary probe, so a copy that finishes at the length cap verifies while an abort still holds; tests pin both sides.
RouterRequired naming only the first offending role: fixed - the message now names every offending role.
deployment.md name-count sentence: fixed to seven.

Verification: full pkg/worker suite, make lint, and make lint docs all pass on the pushed tree.

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode · 2026-10-25T02:30:40Z

Review round: the seven findings from this run are answered.

Inline threads:

scaling-benchmark dropped facts in the SGLang PD move: restored - the run paragraph again records that each role published its group independently and that the thinking switch held on the arithmetic, Chinese and JSON requests. Thread replied and resolved.
routing.md matrix without version scope: a scope sentence under the matrix names the recorded versions (vLLM 0.29.0, SGLang 0.5.18, the routers of gpustack/llm-router:v0.3.1) and the re-verify rule. Thread replied and resolved.
walkthrough losing the SGLang manifest: the manifest lives in the module page SGLang shape section alongside the contract and classification table; the procedure removal was by maintainer review, and the intro pointer is now scoped to the vLLM captured run. Thread replied and resolved.
prefill-decode two-paragraph removal: intentional, by maintainer review. The invalid-destination safety fact survives in the argv.go render comment, which is why the split path never renders the flag; whether the docs restate it once more is left to the maintainer. Thread replied and stays open.

Summary-routed findings:

external-dp lead said "refuses the deployment" while the admission bullet is role-scoped: the lead now says admission refuses a role that declares external data parallelism.
external-dp Results restated the registration boundary: condensed to the recorded outcome with a link to routing.md, the owning section.
scaling-benchmark "A first later runtime-acceptance run" wording and a dangling line wrap: reworded and rewrapped.
routing.md matrix intro sentence fragment: now a full sentence.

Verification on the pushed tree: make lint docs reports 83 site pages, 0 broken links, 0 errors, and the docs shape check reports no warnings.

thxCode added 8 commits October 23, 2026 22:11

style(worker): format instance webhook test

e4d0531

Signed-off-by: thxCode <thxcode0824@gmail.com>

gpustack-code-review Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread pkg/worker/controllers/worker/model_deployment_group_forward_probe.go

thxCode added 3 commits October 23, 2026 22:24

docs: place scaling after external DP in navigation

6342799

Signed-off-by: thxCode <thxcode0824@gmail.com>

fix(worker): expose proxy probe limits in group status

ab50bc9

Signed-off-by: thxCode <thxcode0824@gmail.com>

docs(model-deployment): link to owning routing and status rules

300fd8c

Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode added 4 commits October 24, 2026 03:38

gpustack-code-review Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread docs/modules/model-deployment/external-dp.md

Comment thread docs/walkthroughs/model-deployment/external-dp.md Outdated

thxCode added 2 commits October 24, 2026 05:59

gpustack-code-review Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread pkg/worker/controllers/worker/model_deployment_runtime_cache.go Outdated

thxCode added 4 commits October 24, 2026 09:13

thxCode mentioned this pull request Oct 24, 2026

feat(llm-router): register one worker per attention-dp rank #784

Merged

gpustack-code-review Bot reviewed Oct 24, 2026

View reviewed changes

thxCode added 4 commits October 25, 2026 02:10

fixup! fix(worker): keep engine runtime caches off the replica writab…

2808119

…le layer Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fix(worker): keep engine runtime caches off the replica writab…

702bae0

…le layer Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fix(worker): compare the visible copy and register external dp…

30a43e0

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! docs(model-deployment): record the SGLang runtime results unde…

232ca16

…r the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode force-pushed the spec/dp-group-router-supervisor branch from c95ca21 to 232ca16 Compare October 24, 2026 18:12

gpustack-code-review Bot reviewed Oct 24, 2026

View reviewed changes

thxCode added 2 commits October 25, 2026 02:49

fixup! fix(worker): compare the visible copy and register external dp…

df32519

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! docs(model-deployment): record the SGLang runtime results unde…

ab6df7b

…r the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode added 2 commits October 25, 2026 04:04

fixup! fix(worker): compare the visible copy and register external dp…

17565c4

… members per rank Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

426864c

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

gpustack-code-review Bot reviewed Oct 25, 2026

View reviewed changes

Comment thread docs/modules/model-deployment/scaling-benchmark.md Outdated

thxCode added 10 commits October 25, 2026 08:45

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

ff3029b

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

e6a591d

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

3bc0255

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

79c4f94

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

95f0722

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

d9083bb

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

cd0b3c3

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

48bdd70

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

9a44e13

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

b6a63ce

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

gpustack-code-review Bot reviewed Oct 25, 2026

View reviewed changes

Comment thread docs/modules/model-deployment/prefill-decode.md

Comment thread docs/modules/model-deployment/routing.md

Comment thread docs/walkthroughs/model-deployment/external-dp.md

fixup! fixup! docs(model-deployment): record the SGLang runtime resul…

1f0a92f

…ts under the tested-facts rules Signed-off-by: thxCode <thxcode0824@gmail.com>

thxCode merged commit 0822021 into main Oct 25, 2026
10 checks passed

thxCode deleted the spec/dp-group-router-supervisor branch October 25, 2026 02:38

This was referenced Oct 25, 2026

fix(worker): carry the dp supervisor review follow-ups #785

Merged

refactor(agents): consolidate the e2e case corpus without row loss #786

Merged

Conversation

thxCode commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR links to:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

gpustack-code-review Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

thxCode commented Oct 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thxCode commented Oct 24, 2026

Runtime acceptance update (DP supervisor / SGLANG_RUST_SERVER External DP)

Uh oh!

thxCode commented Oct 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thxCode commented Oct 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thxCode commented Oct 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thxCode commented Oct 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thxCode commented Oct 23, 2026 •

edited

Loading

gpustack-code-review Bot commented Oct 23, 2026 •

edited

Loading