fix(model-cache): propagate caller cancellation into catalog fetches

[DaubnerF]

What

Caller cancellation now reaches the model-catalog fetcher layer. getModels and refreshModels accept an optional AbortSignal, which is threaded through the model-cache single-flight to whichever fetcher is dispatched. Before, every caller joined one non-cancellable shared promise: a caller that gave up left the HTTP request and the in-flight entry running until the request settled.

How

• Each in-flight entry owns a refcounted AbortController. The network request aborts only when the last waiter leaves; a partial abort detaches that waiter and leaves the fetch running.
• At last-waiter abort the shared entry is released synchronously, so a joiner arriving after the release starts a fresh fetch instead of joining the doomed one.
• Every fetch routed through the single-flight carries a 15 s bound; the timeout surfaces as an abort of the shared fetch.
• Fetchers whose HTTP client natively supports cancellation (axios signal, fetch signal) cancel the request at the network level. SDK-bound fetchers (poe, the LM Studio client calls) expose no cancellation surface, so they honor cancellation at their await boundaries: the shared entry is released and waiters stop waiting.

Heads-up

The uniform 15 s single-flight timeout replaces per-fetcher bounds on single-flight-served fetchers: litellm from 5 s to 15 s, kenari/nanogpt/opencode-go/deepseek/moonshot from 10 s to 15 s. The auth-scoped fetchers (zoo-gateway, kimi-code) keep their own bounds; the per-model OpenRouter endpoints path is unchanged.

Tests

44 new tests: 15 single-flight behavior tests (refcounting, release-on-abort, fresh joiner, timeout) plus per-fetcher abort tests across the touched providers. Provider surface: 1880 passed, 0 failed.

Closes #1615

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: Zoo-Code-Org/Zoo-Code/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 22db6cd5-8180-458e-8d59-42db7c628fc5

📥 Commits

Reviewing files that changed from the base of the PR and between e653dd8 and 649a11d.

📒 Files selected for processing (1)

• src/api/providers/fetchers/__tests__/poe.spec.ts

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.

📜 Recent review details

🧰 Additional context used

📓 Path-based instructions (5)

Treat model, provider, MCP, path, command, and tool data as untrusted.

⚙️ CodeRabbit configuration file

Files:

• src/api/providers/fetchers/__tests__/poe.spec.ts

Require regression coverage at the lowest valid harness with behavior-focused assertions, including relevant negative, error, false/unset, and boundary cases.

⚙️ CodeRabbit configuration file

Files:

• src/api/providers/fetchers/__tests__/poe.spec.ts

Check strict typing and exhaustive behavior across normal, boundary, error, cancellation, retry, and compatibility paths.

⚙️ CodeRabbit configuration file

Files:

• src/api/providers/fetchers/__tests__/poe.spec.ts

Verify extension/webview contracts, cancellation and error propagation, VS Code lifecycle correctness, and behavior under retries and partial failure.

⚙️ CodeRabbit configuration file

Files:

• src/api/providers/fetchers/__tests__/poe.spec.ts

Act as an adversarial second-opinion reviewer.

⚙️ CodeRabbit configuration file

Files:

• src/api/providers/fetchers/__tests__/poe.spec.ts

🔇 Additional comments (1)

src/api/providers/fetchers/__tests__/poe.spec.ts (1)

154-163: LGTM!

Also applies to: 165-183, 185-203

---

📝 Summary

Summary by CodeRabbit

• New Features

  • Added cancellation support for model catalog requests across providers using AbortSignal.
  • Canceled requests now surface an AbortError instead of returning incomplete or fallback results.
  • Added coordinated cancellation, timeout handling, and cache-safe behavior for concurrent model requests.

• Bug Fixes

  • Prevented aborted requests from being silently treated as successful catalog fetches.
  • Ensured cancellation state remains consistent when multiple callers share a request.

Walkthrough

The change adds optional abort-signal support across model-catalog fetchers. The model cache now coordinates cancellable single-flight requests with bounded timeouts, waiter tracking, and immediate release. Tests cover provider propagation, abort errors, fallback ordering, fan-out requests, and shared-flight lifecycle behavior.

Changes

Model catalog cancellation

Layer / File(s)	Summary
Shared cancellation and single-flight coordination src/shared/api.ts, src/api/providers/fetchers/modelCache.ts, src/api/providers/fetchers/__tests__/modelCache.spec.ts	GetModelsOptions now accepts signal. The model cache propagates signals, applies a 15-second flight timeout, tracks waiters, detaches abort listeners, and releases a flight when its last waiter aborts. Tests cover pre-aborted calls, joiners, timeout, cleanup, and auth-scoped bypasses.
Provider fetcher signal propagation src/api/providers/fetchers/{deepseek,kenari,kimi-code,litellm,lmstudio,moonshot,nanogpt,ollama,opencode-go,openrouter,poe,requesty,unbound,vercel-ai-gateway,zoo-gateway}.ts	Provider fetchers now accept optional signals. Supported fetch and Axios requests receive those signals. Abort errors are rethrown before generic logging or fallback handling. Fixed per-request timeouts were removed where replaced by signals; Zoo Gateway retains its 15-second timeout.
Provider cancellation coverage src/api/providers/fetchers/__tests__/*	Tests verify signal forwarding, AbortError rejection, default signal configuration, DeepSeek fallback ordering, Ollama fan-out cancellation, LMStudio probe cancellation, and Poe behavior when its SDK has no cancellation surface.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant ModelCache
  participant ProviderFetcher
  participant HTTPClient
  Caller->>ModelCache: Request models with AbortSignal
  ModelCache->>ProviderFetcher: Dispatch shared fetch with signal
  ProviderFetcher->>HTTPClient: Start cancellable catalog request
  Caller->>ModelCache: Abort signal
  ModelCache->>ProviderFetcher: Abort when the last waiter leaves
  ProviderFetcher->>HTTPClient: Cancel request where supported
  ModelCache-->>Caller: Reject with AbortError

Loading

---

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

• Ignore (reviewers only)

❌ Failed checks (1 error, 2 warnings)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	❌ Error	The issue explicitly excludes auth-scoped fetches. This PR adds caller-signal handling and tests for the direct zooGateway and kimiCode paths. modelCache.ts passes the caller signal to those pat…	Remove the zooGateway and kimiCode auth-scoped cancellation changes and their tests from this PR, or move them to a separately scoped issue and pull request.
Regression Evidence	⚠️ Warning	The changed auth-scoped refreshModels() path lacks focused cancellation coverage. modelCache.ts now forwards options.signal at lines 561–564, but the added auth-scoped test covers only `getModel…	Add a model-cache test for auth-scoped refreshModels() that passes an AbortSignal, asserts the signal reaches the provider fetcher, and verifies the refresh degradation result when the signal aborts. Include a pre-aborted case if the in…
Lifecycle Resource Cleanup	⚠️ Warning	The changed single-flight path retains resources after early cancellation and can duplicate provider work. dedupedFetch() creates an AbortSignal.timeout(15_000) and registers onTimeout at `model…	Dispose the per-flight timeout on early release. Use an explicitly clearable timer, or otherwise provide a cancellation/disposal path that removes the timeout listener and releases the timer when the last waiter leaves. For Poe and LM Studi…

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Issue #1615 coding requirements are met. GetModelsOptions.signal reaches getModels, refreshModels, and the single-flight coordinator. Per-waiter cancellation rejects promptly. The refcounted fli…
Security Boundaries	✅ Passed	No changed path meets the security failure conditions. The production diff adds AbortSignal propagation and cancellation guards in modelCache.ts and provider fetchers. It does not add secret or PI…
Persistence Integrity	✅ Passed	No changed persistence path meets the failure condition. The PR changes in-memory single-flight state and cancellation flow in modelCache.ts; writeModels() and readModels() are unchanged. Both `…
Title check	✅ Passed	The title clearly identifies the main change: propagating caller cancellation into model-cache catalog fetches.
Description check	✅ Passed	The description explains the implementation, cancellation behavior, timeout changes, affected fetchers, tests, and linked issue. It omits the template's pre-submission checklist and some optional sect…

Full details: Out of Scope Changes check

Explanation

The issue explicitly excludes auth-scoped fetches. This PR adds caller-signal handling and tests for the direct zooGateway and kimiCode paths. modelCache.ts passes the caller signal to those paths, and those fetchers implement new cancellation behavior outside the single-flight requirement.

Full details: Regression Evidence

Explanation

The changed auth-scoped refreshModels() path lacks focused cancellation coverage. modelCache.ts now forwards options.signal at lines 561–564, but the added auth-scoped test covers only getModels() at lines 1648–1691. No test invokes refreshModels() with zooGateway or kimiCode and verifies signal forwarding or cancellation behavior.

Resolution

Add a model-cache test for auth-scoped refreshModels() that passes an AbortSignal, asserts the signal reaches the provider fetcher, and verifies the refresh degradation result when the signal aborts. Include a pre-aborted case if the intended contract requires the fetcher not to start.

Full details: Lifecycle Resource Cleanup

Explanation

The changed single-flight path retains resources after early cancellation and can duplicate provider work. dedupedFetch() creates an AbortSignal.timeout(15_000) and registers onTimeout at modelCache.ts:416-419. Last-waiter cancellation only aborts the flight controller and deletes the map entry at modelCache.ts:496-502; it does not clear the timeout or remove onTimeout. The timer and listener can therefore remain for up to 15 seconds, and the listener remains longer when the provider promise does not settle. The changed Poe path confirms a concrete non-cancellable case: getPoeModels() calls fetchPoeModels({ apiKey, baseURL }) without the signal at poe.ts:14-18. After the last waiter aborts, a new getModels() call starts a second Poe SDK fetch while the first fetch is still pending. LM Studio has the same issue for its SDK calls at lmstudio.ts:83-95.

Resolution

Dispose the per-flight timeout on early release. Use an explicitly clearable timer, or otherwise provide a cancellation/disposal path that removes the timeout listener and releases the timer when the last waiter leaves. For Poe and LM Studio, either add cancellation support to the underlying SDK operation or keep the non-cancellable operation tracked until it settles and prevent a replacement flight from starting duplicate provider work.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Review status

Thanks for contributing. This comment tracks the review sequence and the next action.

Current step: Address automated review findings and push fixes.

After fixes are pushed and required CI passes, automated review restarts.

Review-state labels are managed by this workflow; do not edit them manually.

[codecov]

Codecov Report

❌ Patch coverage is 85.41667% with 21 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/api/providers/fetchers/modelCache.ts	82.43%	10 Missing and 3 partials ⚠️
src/api/providers/fetchers/deepseek.ts	70.37%	4 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

[coderabbitai]

Pre-merge checks failed. Please resolve the failing checks before merging.