Skip to content

Supervisor insight: expose MTP draft acceptance and fallback efficiency #16

Description

@igorls

Problem

The Supervisor records speculative decoding totals, but it does not turn them into an operator-facing decision about the configured draft window.

Required insight

Add one insight object to the existing /api/insights computation and render that same object in the dashboard. It should report:

  • accepted_tokens / drafted_tokens over the stated measurement window;
  • accepted_per_position so decay across draft positions is visible;
  • fallback_steps, with a warning when fallback frequency rises;
  • representative request IDs and the exact number of measured requests;
  • unavailable when the active log window contains no speculative telemetry.

Recommendations such as reducing the draft window must be marked inferred. The measured counters and position curve remain the evidence.

Acceptance

A fixture with strong early acceptance and weak late-position acceptance produces the expected measured curve and an inferred tuning recommendation. Missing telemetry produces unavailable rather than zero.

Split from #12.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions