Skip to content

fix(inferences): stop showing a healthy scaler as failing - #101

Merged
aceforeverd merged 4 commits into
mainfrom
fix/scaler-deferred-condition
Oct 26, 2026
Merged

aceforeverd merged 4 commits into
mainfrom
fix/scaler-deferred-condition

Conversation

@zhanghaohit

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • bug fix

What this PR does and why

The install-progress "Scaling" step marked the scaler as failing whenever any LLMScaler condition had status: "False". ScalingDeferred reads the other way round: llm-operator sets it True while a rollout or scale-down settle guard holds a step back, and False otherwise (NotDeferred, or DeferralTimedOut after a forced write). So every healthy scaler showed NotDeferred in red.

Condition Before After
ScalingDeferred=False bad, NotDeferred falls through to the replica check, e.g. ok "1 replicas, steady"
ScalingDeferred=True falls through (ignored) wait, "Deferred: "
DecisionAvailable=False bad bad (unchanged)

Adds one string, steps.scaler.deferred, in both locales.

Which issue(s) this PR fixes

Fixes #100

How it was tested

  • Three cases added to installSteps in lib.test.ts. Against the old scalerStep the first two fail and the third passes, which is the intended split.
  • npm run typecheck, npm run build pass. npm test passes on Node 24 (the CI version): 47 files, 421 tests. On Node 25 src/shell/preferences.test.ts fails on main as well (Node's own localStorage global shadows jsdom's); unrelated to this change.
  • The status in inferences: scaling step shows a healthy scaler as failing (NotDeferred) #100 was taken from a live LLMScaler on llm-operator 0.5.1-rc1.

Checks

  • go test ./... and gofmt -l . is empty (no Go change)
  • npm run typecheck && npm test && npm run build in web/ (if web/ changed)
  • helm lint --strict helm/console (if the chart changed)
  • Docs updated where behaviour, a config key or a chart value moved
  • No internal hostnames, IPs, registries or credentials

Notes for reviewers

Deliberately left out: the swiss copy has the same check in components/ReleaseObjects.tsx (the scaler card badge and the condition list dots) and routes/DeploymentDetail.tsx. Per coding-conventions those are fixed upstream in modelsphere/swiss and resynced, not edited here. Until then the cluster-resources scaler card still shows NotDeferred as a warning badge.

Release note

The deployment page no longer shows a healthy autoscaler as failing with "NotDeferred"; a deferred scale step now shows as waiting with its reason.

The scaling step treated every LLMScaler condition with status False as a
failure. ScalingDeferred is inverted: llm-operator sets it True while a
rollout or scale-down settle guard holds a step back, and False otherwise
(NotDeferred, or DeferralTimedOut after a forced write). Every healthy
scaler therefore showed "NotDeferred" in red.

- ScalingDeferred=False no longer counts as failing
- ScalingDeferred=True shows as waiting, with its reason
- any other condition at False (DecisionAvailable) still fails the step

Fixes #100
@zhanghaohit zhanghaohit added kind/bug Behaviour differs from the docs or the design area/swiss The mounted model deployment pages labels Oct 25, 2026
@aceforeverd
aceforeverd requested a review from huskcass October 26, 2026 03:02

@huskcass huskcass left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers. The fix matches llm-operator's ScalingDeferred semantics (True while a rollout or settle guard holds a step back, False otherwise), and the three new tests pin the split. vitest for src/modules/inferences and typecheck pass locally.

Low, non-blocking:

  • The check is still "any False except ScalingDeferred is bad". If llm-operator adds another condition where False is the normal state, it will show red again. Checking only DecisionAvailable=False as a failure would be sturdier.
  • A forced write after the defer timeout (ScalingDeferred=False, DeferralTimedOut) now shows as plain ok/steady, so the step gives no hint that a guard was overridden. Fine for this fix; worth a follow-up if that case matters on the page.

Verdict: good to merge.

— Review Golden

@aceforeverd

Copy link
Copy Markdown
Contributor

LGMT.

BTW, I'm adding new status field to LLMScaler e.g .status.phase and .status.message for summary, so web does not need to aware scaler logic. This patch is kept as backwards compatibility

@aceforeverd

Copy link
Copy Markdown
Contributor

hold on util modelsphere/llm-operator#23 . It revised LLMScalers .status.conditions to match HPA v2. The healthy check here should match condition type ScalingActive and AbleToScale, with a fallback to old conditions if not found

…tive

llm-operator now reports LLMScaler status the way HPA does. The scaling
step still failed on any condition at False, so a healthy scaler with
ScalingLimited=False (DesiredWithinRange) showed as failing.

- only AbleToScale=False or ScalingActive=False fails the step, with the
  condition's reason and message
- AbleToScale=True with RolloutInProgress, ScaleDownSettling or
  ScaleDownStabilized shows as holding; Synced and DeferralTimedOut are
  a normal scale
- ScalingLimited and any other condition never fail the step
- older operators: DecisionAvailable stands in when ScalingActive is
  absent, ScalingDeferred=True holds when AbleToScale is absent; a stale
  old condition next to the new one is ignored

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

@huskcass huskcass left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at b01111a.

No blockers.

Verdict: scalerStep matches the condition contract in llm-operator docs/llmscaler.md for new and old objects; typecheck, tests and build pass locally and the gate/go/web checks are green. Mergeable.

— Review Golden

- AbleToScale=False with a message and no ScalingActive
- an old object with only ScalingDeferred=True
- AbleToScale=False next to a stale ScalingDeferred=True

Also say in the comment that DecisionAvailable is still written as a
deprecated mirror of ScalingActive, not left behind like ScalingDeferred.

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

@huskcass huskcass left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers.

Comment and test-only follow-up: the scalerStep comment now matches the operator behavior, and the three added cases cover the remaining edge cases; logic unchanged. Good to merge.

— Review Golden

Comment thread web/src/modules/inferences/lib.ts Resolved
ScalingDeferred only shipped in a release candidate. The released
operator to stay compatible with is 0.5.0, which writes DecisionAvailable
under the Custom provider and no condition under Prometheus.

- no ScalingDeferred handling; a missing AbleToScale means no hold or
  failure information
- tests use the 0.5.0 status shape in place of the ScalingDeferred cases

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

@huskcass huskcass left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at dea3127.

No blockers.

Verdict: the step now reads operator v0.5.0 objects correctly (Custom DecisionAvailable True/False, Prometheus with no conditions) and the AbleToScale/ScalingActive handling is unchanged from b01111a; typecheck, tests and build pass locally and the gate/go/web checks are green. Mergeable.

— Review Golden

@aceforeverd
aceforeverd merged commit 6766d76 into main Oct 26, 2026
4 checks passed
@aceforeverd
aceforeverd deleted the fix/scaler-deferred-condition branch October 26, 2026 08:48
aceforeverd added a commit that referenced this pull request Oct 26, 2026
* fix(inferences): stop showing a healthy scaler as failing

The scaling step treated every LLMScaler condition with status False as a
failure. ScalingDeferred is inverted: llm-operator sets it True while a
rollout or scale-down settle guard holds a step back, and False otherwise
(NotDeferred, or DeferralTimedOut after a forced write). Every healthy
scaler therefore showed "NotDeferred" in red.

- ScalingDeferred=False no longer counts as failing
- ScalingDeferred=True shows as waiting, with its reason
- any other condition at False (DecisionAvailable) still fails the step

Fixes #100

* fix(inferences): read LLMScaler health from AbleToScale and ScalingActive

llm-operator now reports LLMScaler status the way HPA does. The scaling
step still failed on any condition at False, so a healthy scaler with
ScalingLimited=False (DesiredWithinRange) showed as failing.

- only AbleToScale=False or ScalingActive=False fails the step, with the
  condition's reason and message
- AbleToScale=True with RolloutInProgress, ScaleDownSettling or
  ScaleDownStabilized shows as holding; Synced and DeferralTimedOut are
  a normal scale
- ScalingLimited and any other condition never fail the step
- older operators: DecisionAvailable stands in when ScalingActive is
  absent, ScalingDeferred=True holds when AbleToScale is absent; a stale
  old condition next to the new one is ignored

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

* test(inferences): cover more LLMScaler condition fallbacks

- AbleToScale=False with a message and no ScalingActive
- an old object with only ScalingDeferred=True
- AbleToScale=False next to a stale ScalingDeferred=True

Also say in the comment that DecisionAvailable is still written as a
deprecated mirror of ScalingActive, not left behind like ScalingDeferred.

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

* fix(inferences): drop ScalingDeferred from the LLMScaler step

ScalingDeferred only shipped in a release candidate. The released
operator to stay compatible with is 0.5.0, which writes DecisionAvailable
under the Custom provider and no condition under Prometheus.

- no ScalingDeferred handling; a missing AbleToScale means no hold or
  failure information
- tests use the 0.5.0 status shape in place of the ScalingDeferred cases

Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>

---------

Co-authored-by: huskcass <333684917+huskcass@users.noreply.github.com>
Co-authored-by: aceforeverd <11715200+aceforeverd@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/swiss The mounted model deployment pages kind/bug Behaviour differs from the docs or the design

Projects

None yet

Development

Successfully merging this pull request may close these issues.

inferences: scaling step shows a healthy scaler as failing (NotDeferred)

3 participants