Repository navigation
Size the route shared dicts to start on 64 KiB-page arm64 hosts - #13
Merged
Merged
Conversation
nginx carves each shared zone into pages of the kernel page size, and arm64 kernels often use 64 KiB pages. There the per-route cluster_avg dict (16k) is too small for the slab pool, and the openresty that loads the generated route exits with "ngx_slab_alloc() failed: no memory". Measured on Kunpeng 920 with 64 KiB pages: 128k fails, 192k starts. Raise it to 1m, like the other route dicts, and add a test that renders a route and refuses any dict under 256k. Checked against the old template: the test fails on cluster_avg 16k.
Moves the chart (version, appVersion), the kustomize image tag, the sample and the e2e scripts' default image tags to 0.4.3, the release with the route shared dicts sized for 64 KiB-page hosts.
This was referenced Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On arm64 hosts with 64 KiB pages (Kunpeng 920, Kylin V10), openresty exits at startup with
ngx_slab_alloc() failed: no memorywhen a shared dict is too small for the slab pool at that page size. Measured there withopenresty -t: 16k / 64k / 128k fail, 192k and up start.The route template autoconfig renders declares
cluster_avg_<route> 16k, so the first ModelRoute on such a host would take openresty down. (The image's own dicts are fixed in modelsphere/llm-openresty#25.)Change
internal/sink/route.tmpl:cluster_avg_<route>→ 1m, like the other route dicts.route_video.tmpldeclares none.TestRenderRoute_SharedDictsFitLargePages: renders a route and fails on anylua_shared_dictunder 256k (and on finding none, so it cannot pass vacuously). Against the old template it fails withlua_shared_dict cluster_avg_glm is 16k; with this change,go test ./internal/...passes.Release. Also moves the chart, kustomize image tag, sample and e2e defaults to 0.4.3, so the
0.4.3tag can be cut once this merges.