You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Comet owns per-user waiting queues and hierarchical output-token WFQ, but only SMG knows whether a model partition can accept another request. The merged registry in #54 provides replay-safe tokens but is intentionally unwired.
Wire the registry to authoritative priority-scheduler slots so Comet can reserve capacity without putting request bodies into SMG's shared queue.
The feature is disabled by default and inert when disabled.
A service-key-authenticated internal issue endpoint selects an exact configured admission partition and atomically acquires one scheduler slot without queueing.
The inference path redeems a matching credit at most once and reuses its held SchedulerPermit instead of acquiring a second slot or entering the shared scheduler queue.
Cancellation and TTL expiry release the held scheduler permit, and permit payloads are dropped outside registry locks.
Credit issue is immediate: no available slot returns retryable 429 rather than queueing inside SMG.
When explicitly enabled with partitioned engine-feedback admission, the scheduler's total ceiling follows telemetry revisions; SlotPool inflight remains the authoritative local usage count.
Handler-side adaptive admission remains a safety backstop for telemetry races. A resulting pre-execution 429 releases the redeemed permit normally and is not charged by Comet.
Credit issue, redemption, cancellation, expiry, binding mismatch, replay, capacity shrink, and concurrent single-winner behavior have focused tests.
Metrics use bounded partition/outcome labels and never expose bearer tokens.
Scope boundaries
No Comet user-facing admission API or client wrapper in this issue.
No SMG per-user queue or fair-share ordering.
No engine, pool, worker-selection, or cache-routing changes.
The first production consumer will use explicit admission partitions only.
Problem
Comet owns per-user waiting queues and hierarchical output-token WFQ, but only SMG knows whether a model partition can accept another request. The merged registry in #54 provides replay-safe tokens but is intentionally unwired.
Wire the registry to authoritative priority-scheduler slots so Comet can reserve capacity without putting request bodies into SMG's shared queue.
Design context: LLM360/comet#576 and #53.
Acceptance criteria
SchedulerPermitinstead of acquiring a second slot or entering the shared scheduler queue.SlotPoolinflight remains the authoritative local usage count.Scope boundaries