Skip to content

fix(models): honor runtime operation deadlines in the dashboard - #5299

Merged
gabsprogrammer merged 1 commit into
Osmantic:public-betafrom
0xacee:fix/models-honor-operation-deadlines
Sep 15, 2026
Merged

gabsprogrammer merged 1 commit into
Osmantic:public-betafrom
0xacee:fix/models-honor-operation-deadlines

Conversation

@0xacee

@0xacee 0xacee commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Problem and change

The model UI mixes stale or missing operation deadlines. Activation stops after 610 seconds although the current API permits a 2700-second host call plus 120 seconds of download-busy retry grace. Delete and benchmark requests have no deadline. Download start drops its timer before reading an error body, which can leave its action pending indefinitely.

Keep activation polling through the current host/grace budget plus five seconds (2825 seconds), while preserving the final authoritative inventory check. Add one bounded request helper for download, delete and benchmark, including error-body reads and cleanup of unused response bodies. Budgets are 15 seconds for download acceptance, 35 seconds for the 30-second delete call and 400 seconds for the 128-token benchmark's 384-second provider timeout. Timeout messages explicitly ask for refresh because the server may still be working.

Validation

  • On fresh upstream/public-beta, the deadline cases had 6 failures / 1 passing control.
  • All 88 model hook, activation snapshot and Models page tests passed.
  • Tests cover stalled headers/error bodies, signal cancellation, late completions, action cleanup, exact terminal activation budget and successful activation after the previous ten-minute limit.
  • Updated the existing exact DELETE request assertion to include its new AbortSignal.
  • ESLint, production dashboard build and git diff --check passed.

Related work and gates

Inspected live PRs and patches: #4962 stops activation polling on unmount; #3376 improves inventory timeout messages; #3027 validates inventory shape. This PR addresses operation budgets and request completion. #4962 overlaps the activation loop and should retain the new duration when composed.

The backend's bounds are in routers/models.py: load_model passes timeout=2700 with 120-second busy retry grace, delete uses the 30-second default and the fixed 128-token benchmark uses max(60, max_tokens*3). These are UI request budgets, not proof that an aborted server-side operation was cancelled. Existing inventory refresh/final verification behavior remains.

Tests use fake time and local response fixtures. Live model activation/benchmark qualification and independent human review remain pending. AI-assisted implementation and validation by Codex; ready for review is not approval to merge.

@gabsprogrammer
gabsprogrammer merged commit 574ec52 into Osmantic:public-beta Sep 15, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants