Skip to content

bug: InferenceQueue is not priority-aware despite being documented as such #49

@jpleva91

Description

@jpleva91

Bug: InferenceQueue ignores Priority

File: internal/scheduler/queue.go

The package doc and struct comment both describe the queue as "priority-aware", and four Priority constants are defined (PriorityWorker, PriorityPlanner, PriorityCorrector, PriorityEvaluator). However, Submit() never uses the priority — it simply blocks on a semaphore channel:

func (q *InferenceQueue) Submit(ctx context.Context, req InferenceRequest) error {
    // ...
    select {
    case q.slots <- struct{}{}:   // FIFO channel — priority ignored
    case <-ctx.Done():
        return ctx.Err()
    }
    // ...
}

The InferenceRequest.Priority field is never read. All requests are served FIFO regardless of their declared priority. This means PriorityEvaluator = 3 gets no advantage over PriorityWorker = 1.

Impact

  • Evaluator and Corrector agents (higher priority) are not given inference slots ahead of Worker agents. Under load this can cause governance-critical evaluation to queue behind routine worker calls.
  • Misleading documentation increases the chance of future callers setting priorities and expecting them to be honoured.

Suggested fix

Replace the semaphore channel with a proper heap-based priority queue (e.g. container/heap) that dequeues by descending priority. Or, if strict priority isn't needed yet, remove the Priority field from InferenceRequest and update the docs to say the queue is FIFO with a concurrency cap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priorityagent:claimedAgent dispatched — do not re-dispatchbugSomething isn't workingsprintCurrent sprint priority

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions