Skip to content

sglang 0.9.18, vllm 0.8.17: pd router boot script logs identity + handoff - #73

Merged
remotego merged 2 commits into
mainfrom
pd-ds-router-boot-log-lines
Oct 20, 2026
Merged

remotego merged 2 commits into
mainfrom
pd-ds-router-boot-log-lines

Conversation

@remotego

Copy link
Copy Markdown
Collaborator

Summary

The router boot script from #72 runs a silent gap between the argv line (router: prefill ... / router: decode ...) and the router process actually starting — during which probe_identity may be querying sidecars, writing the snapshot file, or (worst case) stuck on the 90-minute budget. kl logs on a router pod showed nothing between those two moments, which made it hard to tell whether boot was progressing or wedged.

Three breadcrumbs added, all on stderr so probe_identity's stdout return value stays clean:

router: <ip> identity <restarts>|<watcher_started_at>   # per-IP success
router: <ip> identity sentinel (...reason...)           # per-IP sentinel (old sidecar / hw disabled)
router: boot script done, exec into router              # handoff to exec

Sentinel reasons disambiguate the three no-identity paths:

  • hangWatcher disabled at render — HW_PORT=0
  • sidecar has no /engine — 404 from an older sidecar
  • The existing 90min budget exhausted line (unchanged) covers the third case.

Rendered-output change

Only the boot-script body changes — adds the echo >&2 lines, no behavior change. Affects both charts' pd-ds DisaggregatedSet router Deployment. 52 non-pd cases unchanged.

Gold rebaked per R3 at this branch's HEAD; 54/54 verify.

Test plan

  • helm template renders clean for all pd cases, both charts
  • bash -n on the extracted boot scripts
  • E2E against mock hang-watchers: 200 → stdout carries tuple, stderr carries log line; 404 → sentinel on stdout, reason on stderr; hw-disabled → same shape
  • Live deploy: kl logs <router-pod> on a pd render shows the three breadcrumbs in order (argv line → per-IP identity/sentinel → handoff → router's own first log)

🤖 Generated with Claude Code

…doff

Otherwise the container log shows a quiet gap between the "router:
prefill / router: decode" argv line and vllm-router's own first line,
with no indication that probe_identity ran (or hung) between them.
Three breadcrumbs, stdout-safe:

  - per-IP success:       "router: <ip> identity <restarts>|<ts>"
  - per-IP sentinel:      "router: <ip> identity sentinel (...reason...)"
  - handoff to exec:      "router: boot script done, exec into router"

All three go to stderr so they land in the container log; stdout
stays reserved for probe_identity's return value. Sentinel reasons
disambiguate the three no-identity paths (hangWatcher disabled at
render, sidecar has no /engine, 90min budget exhausted).
The prior commit's subject still references 0.9.18 / 0.8.17, but those
versions belong to #72 (the engine-identity drift probe). This bump
re-attaches the log-lines change to its own version pair so the followup
release can be tagged cleanly.

No template / values change.
@remotego
remotego merged commit 69ba177 into main Oct 20, 2026
4 checks passed
@remotego
remotego deleted the pd-ds-router-boot-log-lines branch October 20, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant