Repository navigation
feat: serve JobSLORequirement routes; follow the pdengine and scale-to-zero CRD bounds - #8
Merged
Merged
Conversation
…o-zero CRD bounds slo-api only watched LLMSLORequirement, so job services that decision-gen already sizes from JobSLORequirement had no front door, and the bound rules had drifted from the CRD: - The index watches both kinds; a route resolves across them (409 when a serviceId is on more than one CR). A job route's body has `queue` (maxDepth, promql) in place of ttft/otps. A cluster without the JobSLO CRD indexes that kind as empty, so readiness is not held at 503. - `pdengine` bounds on LLMSLORequirement (helm-charts#95), and bound values of 0 (scale-to-zero, helm-charts#20). - Cross-field CRD rules stay with the API server, checked on the merged spec. Its 422 is now 400 invalid_request with the Status causes (not the message, which names the CR), instead of 502. - config/rbac grants jobslorequirements, as the chart already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 2, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
What this PR does and why
The SLO CRDs have moved on and slo-api had not followed:
JobSLORequirement. decision-gen already sizes job services from it, and the chart's slo-api ClusterRole already grantsjobslorequirements, but slo-api only watchedLLMSLORequirement, so job services had no front door.pdenginebounds (feat(llm-slo-decision-gen): add pdengine bound type to LLMSLORequirement CRD helm-charts#95, consumed by feat: PD engine scaling for pdengine LLMSLORequirements slo-scaler-decision-gen#11): new bound type, plus CEL rules that both bounds share a type and that apdenginemax is not 1.400.Changes:
CRIndexruns one watch per kind, keyed by(plural, ns, name). A route resolves across both kinds, since decision-gen sizes them side by side; a serviceId on more than one CR is409, as before. A cluster without the JobSLO CRD (404on list) indexes that kind as empty and looks again every 60s, so/readyzis not held at503.translate.SECTIONS,ALLOWED_DEPLOY_TYPES_*): a job route takesqueue: {maxDepth, promql}instead ofttft/otps, with boundsreplica(orconcurrencyfor the minimum).promqlmay be left out of a PUT to keep the stored one (it is a merge patch).DELETEon a job route keepsqueue, which its CRD requires.422now maps to400 invalid_request. The message is built from the Statusdetails.causes, not the top-level message, which names the CR — callers never see a CR name. A single cause'sspec.x.yfield becomes the API'sx.y.config/rbac/rbac.yamlgrantsjobslorequirements; README, OpenAPI, CHANGELOG and the demo form's bound types follow. The README also no longer says reads of an ambiguous route return the first CR — they have always been409.What existing LLM clients see. Requests and responses that worked before are unchanged. The differences:
400 invalid_requestwith the API server's reason, where it was502 upstream_error(the error path only;400is also the correct "don't retry").GET /configalso lists job routes, in the same summary shape (route, highPriority, bounds).409. That setup is new, and decision-gen would size both under one key anyway.409message says "SLO requirement CRs"; itscodeis unchanged.Without this: job SLOs can only be changed with kubectl, a
pdengineor zero bound cannot be written through the API, and a bound-type mismatch shows up as an opaque502.Which issue(s) this PR fixes
Refs modelsphere/helm-charts#95, modelsphere/slo-scaler-decision-gen#11. Until helm-charts#95 lands, a
pdenginebound is refused by the cluster's CRD and comes back as400with the API server's reason.How it was tested
20 new tests: job routes (GET/PUT keeps
promql, LLM fields refused, DELETE keepsqueue, list), a serviceId shared across kinds is409, a missing JobSLO CRD still syncs, a relist of one kind keeps the other,pdengine/zero bounds,queuevalidation, and422 → 400without the CR name. The fake API is now plural-aware (it returned every CR for every plural). The new guards were checked against broken code: with the404handling, the422mapping, or the causes-based message reverted, their tests fail.Not tested on a cluster.
Checks
Release note
🤖 Generated with Claude Code