Skip to content

feat(scheduler): add single-use capacity credit contract #53

Description

@DavidBellamy

Problem

Comet needs to order batch and interactive work before large request bodies enter shared queues, but SMG must remain the sole authority for scheduler capacity. A Comet admission decision therefore needs a short-lived, single-use SMG capacity credit that cannot be forged, replayed, or rebound to another request.

Design context: LLM360/comet#576 and LLM360/comet#577.

First slice

Add an unwired scheduler-local credit registry that:

  • binds a credit to allocator generation, policy epoch, partition, canonical model, tenant, request attempt, and output-token estimate
  • uses an unguessable opaque token with redacted debug output
  • returns the same active credit for an idempotent duplicate issue
  • rejects a conflicting duplicate request attempt
  • atomically redeems a matching credit at most once
  • rejects binding mismatches without consuming the rightful credit
  • expires and cancels credits while dropping owned capacity payloads outside the registry lock
  • retains bounded terminal tombstones long enough to reject immediate replay

The first slice has no endpoint, config, CLI, Python binding, adaptive admission change, SlotPool change, or request-path wiring.

Follow-up

  • store an acquired SchedulerPermit as the credit payload
  • add an authenticated local Comet to SMG issue/cancel API
  • add proxy-only redemption and strip all client copies of the internal credit header
  • validate generation handoff and expiry during blue/green rollout

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions