Dingo router upstream sync by guojn1 · Pull Request #76 · githubgxll/dynamo

guojn1 · 2026-10-10T09:27:22Z

Overview:

Details:

Where should the reviewer start?

Related Issues

⚠️ This section is required. Every pull request references the issue it implements.

Closes #XXXX

…i-dynamo#15063) Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

…ynamo#15028) Signed-off-by: Tzu-Ling <tzulingk@nvidia.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Signed-off-by: Lavanya <lvijayakrish@nvidia.com>

Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com> Signed-off-by: Anant Sharma <anants@nvidia.com> Co-authored-by: Anant Sharma <anants@nvidia.com>

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

…ns (ai-dynamo#15202) Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

…amo#15325) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

) Signed-off-by: Harry Lee <harrileeeee@gmail.com>

Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com> Co-authored-by: Abhi Jain <abhi.jain@baseten.co>

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

…dynamo#15101) Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com> Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

Signed-off-by: Dr. Stefan Schimanski <sschimanski@nvidia.com>

Automated refresh from the public Dynamo Google Calendar. Generated by .github/workflows/community-events-refresh.yml. Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>

…ultinode (ai-dynamo#15383) Signed-off-by: jthomson04 <jwillthomson19@gmail.com> Signed-off-by: jthomson04 <jothomson@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com> Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>

ai-dynamo#15428) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

…ynamo#15384) Signed-off-by: Tushar Sharma <tusharma@nvidia.com>

…#15252) Signed-off-by: C2 <cwang@coreweave.com> Signed-off-by: Ching-Chia Wang <cwang@coreweave.com>

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

…mo#15434) Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…mo#15328) Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

…mo#15014) Signed-off-by: krishung5 <krish@nvidia.com> Signed-off-by: furionw <qiwa@nvidia.com> Co-authored-by: furionw <qiwa@nvidia.com> Co-authored-by: Krishnan Prashanth <140860868+KrishnanPrash@users.noreply.github.com> Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

…i-dynamo#14838) Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>

…amo#14613) Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>

) Signed-off-by: tanmayv25 <tanmay2592@gmail.com> Signed-off-by: tanmayv25 <tanmayv@nvidia.com>

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

…r, sglang image build

Integrate base tip 00565c1, preserving DingoRouter-base custom behavior. Resolve eight conflicted files: - Use the base SGLang v0.5.21 image tag; retain the available CUDA 13.0.3 SBOM corpus instead of referencing the removed historical corpus. - Preserve GLM sampling defaults, parser inference and reasoning-aware stops. Combine answer-only stops with UTF-8 decoding, aligned logprobs and local cancellation; close generation before exposing a local terminal response. - Carry cumulative usage and cache details through decoder and frontend, including buffered/empty-token updates. Preserve typed metrics, media counts, terminal-frame deduplication, cancellation and KV cache reporting. - Preserve both no_stop_trim and continuous_usage_stats protocol fields. - Keep base documentation and regression cases; adapt metrics assertions to the current typed metrics carrier and retain the relocated frontend doc link. Validation: Python AST/duplicate-definition checks, Ruff F821/F811, format checks and git diff --check. Existing unrelated F841 assignment in sglang decode_handler remains unchanged. No tests executed for this merge.

Preserve cache salt across HTTP, routing and Python processors. Load the SGLang image NIXL C API dynamically with its runtime plugins. Keep shared libraries outside runtime wheels and retain video support. Restore binding unit coverage and align SGLang cancellation/cache tests. Validate AISimulate staging through the version-pinned helper. Validation: original CPU full run plus targeted regressions; 110 SGLang cases and 39 packaging/media checks passed. Prior GPU validation passed before the latest wheel packaging change. Skipped and unrun scenarios remain documented outside the repository; no post-fix full rerun.

Pass configured package indexes via BuildKit secret envs so optional secret mounts cannot shadow the former build arguments with empty values. Preinstall setuptools and wheel in the reusable builder and build the GPU memory service wheel offline with its explicit Python interpreter. Remove empty continuation lines from runtime wheel repair rendering. Validated the original index propagation failure and corrected secret wiring on the hd04 runner. GMS built successfully with network disabled in its actual Python 3.12 builder image and on the Python 3.11 dev host.

tedzhouhk and others added 30 commits September 29, 2026 21:28

feat(ais): migrate perf-model consumers to canonical configuration (a…

8be4df8

…i-dynamo#15063) Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

test(sidecar): validate disaggregated native serving (ai-dynamo#15082)

7f7b533

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

fix(operator): read every vLLM flag spelling through one parser (ai-d…

b4909a6

…ynamo#15028) Signed-off-by: Tzu-Ling <tzulingk@nvidia.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

ci: publish build images to east and west ECR (ai-dynamo#15331)

2d5b116

Signed-off-by: Lavanya <lvijayakrish@nvidia.com>

chore: bump trtllm to v1.3.0rc28 (ai-dynamo#15215)

887a6c1

Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com> Signed-off-by: Anant Sharma <anants@nvidia.com> Co-authored-by: Anant Sharma <anants@nvidia.com>

test(sidecar): cover DEP rank-aware KV routing (ai-dynamo#15086)

9262eab

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

refactor(router): centralize policy configuration and flag deprecatio…

2de120f

…ns (ai-dynamo#15202) Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

ci(recipes): run DeepSeek V4 Pro aggregated nightly benchmark (ai-dyn…

00416b5

…amo#15325) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

refactor(replay): delegate Weka ingestion to AISimulate (ai-dynamo#14355

b59582d

) Signed-off-by: Harry Lee <harrileeeee@gmail.com>

fix(runtime): recover hung etcd watches with keepalive (ai-dynamo#15266)

3f46e3d

Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com> Co-authored-by: Abhi Jain <abhi.jain@baseten.co>

feat(llm): preserve opaque coding-agent headers (ai-dynamo#15367)

8c226c9

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

fix(multimodal): preserve URL path case in ImageLoader cache key (ai-…

777977d

…dynamo#15101) Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com> Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

fix(operator): support B300 in DGDR GPU discovery (ai-dynamo#15216)

4d28ee2

Signed-off-by: Dr. Stefan Schimanski <sschimanski@nvidia.com>

docs: refresh community events

0c0ce6d

Automated refresh from the public Dynamo Google Calendar. Generated by .github/workflows/community-events-refresh.yml. Signed-off-by: dynamo-ops <170655669+dynamo-ops@users.noreply.github.com>

ci: gate deploy tests on operator and copy success (ai-dynamo#15394)

3079ce5

fix: IPv6-only fixes for listeners, ZMQ, SGLang KV events, and vLLM m…

9949fea

…ultinode (ai-dynamo#15383) Signed-off-by: jthomson04 <jwillthomson19@gmail.com> Signed-off-by: jthomson04 <jothomson@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

docs(tokenspeed): add LongCat disaggregation example (ai-dynamo#15013)

2365cf5

Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com> Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>

fix(recipes): use default TCP transport for Kimi aggregated KV routing (

8ad295a

ai-dynamo#15428) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

fix(recipes): update DeepSeek V4 Pro aggregated runtime (ai-dynamo#15427

dac068f

) Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>

ci(deploy): count only live Pods in the operator readiness gate (ai-d…

1512ffd

…ynamo#15384) Signed-off-by: Tushar Sharma <tusharma@nvidia.com>

fix(vllm): declare the ModelExpress startup weight version (ai-dynamo…

df8ad66

…#15252) Signed-off-by: C2 <cwang@coreweave.com> Signed-off-by: Ching-Chia Wang <cwang@coreweave.com>

chore(deps): bump AISimulate to September 30 nightly (ai-dynamo#15422)

835be3a

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

fix(vllm): OPS-8696 let the EPD encode worker report healthy (ai-dyna…

b85db41

…mo#15434) Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

ci(sidecar): run aggregated e2e tests post-merge and nightly (ai-dyna…

16c6d3d

…mo#15328) Signed-off-by: Julien Darve <jdarve@NVIDIA.com>

perf(router): reduce scheduler and dispatch overhead (ai-dynamo#14666)

99ae6e5

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

test(vllm): stop a shared NIXL port stalling decode worker startup (a…

e6bdf6f

…i-dynamo#14838) Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>

fix(router): account for all output blocks in response chunks (ai-dyn…

ae83593

…amo#14613) Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>

feat(sidecar): speak TensorRT-LLM's OpenEngine gRPC API (ai-dynamo#14349

bf13a46

) Signed-off-by: tanmayv25 <tanmay2592@gmail.com> Signed-off-by: tanmayv25 <tanmayv@nvidia.com>

perf(llm): reduce decoder prompt copies (ai-dynamo#15234)

aacb1ab

Signed-off-by: jthomson04 <jwillthomson19@gmail.com>

guojn1 deployed to external_collaborator October 11, 2026 08:10 — with GitHub Actions Active

guojn1 had a problem deploying to external_collaborator October 11, 2026 08:16 — with GitHub Actions Failure

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from efdf816 to 96c5e1e Compare October 11, 2026 08:31

guojn1 deployed to external_collaborator October 11, 2026 08:31 — with GitHub Actions Active

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from 96c5e1e to 33c8e01 Compare October 12, 2026 10:34

guojn1 had a problem deploying to external_collaborator October 12, 2026 10:34 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 12, 2026 12:26 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 12, 2026 12:35 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 12, 2026 12:59 — with GitHub Actions Failure

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from 999253f to 07cdb9f Compare October 12, 2026 13:36

guojn1 had a problem deploying to external_collaborator October 12, 2026 13:37 — with GitHub Actions Failure

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from 07cdb9f to 5fe4c26 Compare October 12, 2026 14:14

guojn1 had a problem deploying to external_collaborator October 12, 2026 14:14 — with GitHub Actions Failure

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from 5fe4c26 to 0bf304a Compare October 12, 2026 14:33

guojn1 had a problem deploying to external_collaborator October 12, 2026 14:33 — with GitHub Actions Failure

fix: upstream-sync regressions, CI apt mirror, smoke test push trigge…

c974eb5

…r, sglang image build

guojn1 force-pushed the DingoRouter-t5-upstream-sync branch from 0bf304a to c974eb5 Compare October 12, 2026 14:57

guojn1 had a problem deploying to external_collaborator October 12, 2026 14:58 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 13, 2026 02:31 — with GitHub Actions Failure

fix(build): build pinned AISimulate wheel when unpublished

e374281

guojn1 had a problem deploying to external_collaborator October 13, 2026 03:44 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 13, 2026 10:51 — with GitHub Actions Failure

guojn1 had a problem deploying to external_collaborator October 14, 2026 08:32 — with GitHub Actions Failure

fix(ci): resolve SGLang packages and cache runtime dependencies

565b6dc

guojn1 had a problem deploying to external_collaborator October 14, 2026 09:28 — with GitHub Actions Failure

githubgxll approved these changes Oct 14, 2026

View reviewed changes

githubgxll merged commit adca349 into DingoRouter-base Oct 14, 2026
4 of 7 checks passed

Navigation Menu

Repository navigation

Dingo router upstream sync - #76

Dingo router upstream sync#76
githubgxll merged 1946 commits into
DingoRouter-basefrom
DingoRouter-t5-upstream-sync

guojn1 commented Oct 10, 2026

Uh oh!

Uh oh!

This branch had an error being deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

guojn1 commented Oct 10, 2026

Overview:

Details:

Where should the reviewer start?

Related Issues

Uh oh!

Uh oh!

This branch had an error being deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants