Skip to content

perf(resources): share concurrent snapshot collection - #4966

Merged
gabsprogrammer merged 1 commit into
Osmantic:public-betafrom
0xacee:perf/resources-share-inflight-snapshot-collection
Sep 14, 2026
Merged

gabsprogrammer merged 1 commit into
Osmantic:public-betafrom
0xacee:perf/resources-share-inflight-snapshot-collection

Conversation

@0xacee

@0xacee 0xacee commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Why this matters

Multiple Dashboard clients arriving at an expired resource cache each start their own host-agent stats request and disk scan. Eight concurrent authenticated /api/services/resources reads reproduce eight calls to each collector; disconnecting all eight and returning starts a ninth. The disk collector performs the real directory traversal in these fixtures.

Share one in-flight collection per cache key. Container and disk work still run concurrently, retain their separate 20/60-second TTLs, and publish complete snapshots after collection finishes. A disconnected HTTP waiter cannot release ownership of a still-running thread. Collection failures remain visible and leave the next explicit request free to collect again.

Restart acknowledgement invalidates ownership of an older container collection as well as its cached value. Existing waiters may finish with their earlier observation, but that read cannot overwrite the post-restart cache.

Validation

  • Six authenticated HTTP regressions cover concurrent reads, one/all clients disconnecting, warm cache reuse, independent TTL expiry, collection failure/recovery and restart invalidation. Four fail on the base; all six pass with the change. Normal eight-client waves now make one host request and one disk scan, with unchanged totals.
  • pytest -q tests/test_resource_collection.py tests/test_resources.py: 26 passed. Uses a real sparse-file directory scan and controlled host-agent receipts; it does not execute Docker stats or restart a live container. Completed tests leave no retained collection tasks.
  • Temporarily applied feat(resources): query scoped snapshots without disk scans #3852's exact scoped-resource implementation and tests: resource, scoped-resource and main API suites pass together, 108 tests. Restored this standalone branch afterward. Existing Starlette deprecation and one existing main-test AsyncMock warning remain explicit.
  • Ruff and git diff --check pass. Existing API CI will collect the new test file; independent review remains pending.

Overlap check

Searched live open/closed PR titles and routers/resources.py changes for concurrency/coalescing. #3852 adds filtering and disk omission; #2579/#2779/#2007 validate stats; #2078/#2857 concern directory errors. None shares resource collection work or prevents its old result from repopulating cache after restart. Merged #4878 applies coalescing to the separate GPU detail endpoint.

No functional dependency. Prefer this PR before #3852, preserving the shared full-cache collection while applying filtering only to responses. Their combined validation is above; optional CPU/RAM-only reads retain #3852's no-disk behavior. The cache keys and response format remain unchanged.

The sharing scope is one API process, matching the existing cache. This does not add disk-scan deadlines or coordinate workers across processes. No persisted state or migration is involved; reverting returns to per-request collection. Live Docker, deployed multi-client latency and hardware validation remain unperformed.

Current validation

At the 2026-09-14 audit, head 4267f9a72cb77178b63569c76207b8deb16c1a9e has 32 successful and 4 skipped hosted checks, with no failed or pending checks. Independent human review and the declared deployed/platform checks remain merge gates.

Local synthetic integration head e8f982f09ba4c714ee2a65cb6867ea3e4328bfde combines the 30 September 14 candidates. The affected model/import, resource, Usage and Talk API suites pass 324 tests, including the actual pinned GGUF reader.

@gabsprogrammer
gabsprogrammer merged commit 44d37a9 into Osmantic:public-beta Sep 14, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants