Skip to content

feat: serve JobSLORequirement routes; follow the pdengine and scale-to-zero CRD bounds - #8

Merged
remotego merged 1 commit into
mainfrom
feat/jobslo-pdengine
Nov 1, 2026
Merged

remotego merged 1 commit into
mainfrom
feat/jobslo-pdengine

Conversation

@remotego

@remotego remotego commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

What type of PR is this?

  • feature

What this PR does and why

The SLO CRDs have moved on and slo-api had not followed:

Changes:

  • CRIndex runs one watch per kind, keyed by (plural, ns, name). A route resolves across both kinds, since decision-gen sizes them side by side; a serviceId on more than one CR is 409, as before. A cluster without the JobSLO CRD (404 on list) indexes that kind as empty and looks again every 60s, so /readyz is not held at 503.
  • Validation is per kind (translate.SECTIONS, ALLOWED_DEPLOY_TYPES_*): a job route takes queue: {maxDepth, promql} instead of ttft/otps, with bounds replica (or concurrency for the minimum). promql may be left out of a PUT to keep the stored one (it is a merge patch). DELETE on a job route keeps queue, which its CRD requires.
  • Cross-field rules are not duplicated in Python. They depend on the merged spec, which the API server checks anyway; its 422 now maps to 400 invalid_request. The message is built from the Status details.causes, not the top-level message, which names the CR — callers never see a CR name. A single cause's spec.x.y field becomes the API's x.y.
  • config/rbac/rbac.yaml grants jobslorequirements; README, OpenAPI, CHANGELOG and the demo form's bound types follow. The README also no longer says reads of an ambiguous route return the first CR — they have always been 409.

What existing LLM clients see. Requests and responses that worked before are unchanged. The differences:

  • A write the CRD rejects is 400 invalid_request with the API server's reason, where it was 502 upstream_error (the error path only; 400 is also the correct "don't retry").
  • GET /config also lists job routes, in the same summary shape (route, highPriority, bounds).
  • A serviceId on both an LLM and a job CR is 409. That setup is new, and decision-gen would size both under one key anyway.
  • The 409 message says "SLO requirement CRs"; its code is unchanged.

Without this: job SLOs can only be changed with kubectl, a pdengine or zero bound cannot be written through the API, and a bound-type mismatch shows up as an opaque 502.

Which issue(s) this PR fixes

Refs modelsphere/helm-charts#95, modelsphere/slo-scaler-decision-gen#11. Until helm-charts#95 lands, a pdengine bound is refused by the cluster's CRD and comes back as 400 with the API server's reason.

How it was tested

$ python -m pytest -q
76 passed in 12.93s
$ python tests/smoke_run.py   # all steps ✓

20 new tests: job routes (GET/PUT keeps promql, LLM fields refused, DELETE keeps queue, list), a serviceId shared across kinds is 409, a missing JobSLO CRD still syncs, a relist of one kind keeps the other, pdengine/zero bounds, queue validation, and 422 → 400 without the CR name. The fake API is now plural-aware (it returned every CR for every plural). The new guards were checked against broken code: with the 404 handling, the 422 mapping, or the causes-based message reverted, their tests fail.

Not tested on a cluster.

Checks

  • Tests and linters for what changed pass locally (see the repository's README or CONTRIBUTING)
  • Docs updated where behaviour, a config key or a chart value moved
  • No internal hostnames, IPs, registries or credentials

Release note

slo-api serves JobSLORequirement routes (`queue` in place of ttft/otps), accepts `pdengine` and zero bounds, and answers a write the CRD's rules reject with 400 instead of 502. action required: if slo-api's RBAC is not the chart's, grant get/list/watch/patch/update on jobslorequirements.

🤖 Generated with Claude Code

…o-zero CRD bounds

slo-api only watched LLMSLORequirement, so job services that decision-gen
already sizes from JobSLORequirement had no front door, and the bound rules
had drifted from the CRD:

- The index watches both kinds; a route resolves across them (409 when a
  serviceId is on more than one CR). A job route's body has `queue`
  (maxDepth, promql) in place of ttft/otps. A cluster without the JobSLO
  CRD indexes that kind as empty, so readiness is not held at 503.
- `pdengine` bounds on LLMSLORequirement (helm-charts#95), and bound values
  of 0 (scale-to-zero, helm-charts#20).
- Cross-field CRD rules stay with the API server, checked on the merged
  spec. Its 422 is now 400 invalid_request with the Status causes (not the
  message, which names the CR), instead of 502.
- config/rbac grants jobslorequirements, as the chart already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@remotego
remotego merged commit 38059d8 into main Nov 1, 2026
4 checks passed
@remotego
remotego deleted the feat/jobslo-pdengine branch November 1, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant