Skip to content

Decide headroom routing in the front door per session, not at launch #395

Closed
5 of 5 issues completed
Closed

Description

@Mearman

Whether a session goes through headroom is fixed when it launches: the launcher bakes x-agent-shim-headroom: 1 into the session's custom headers, and a launch with no provider, headroom or usage tracking never reaches the front door at all. A session cannot start using headroom later, stop using it, or be moved off it while the daemon is unwell.

The door already follows headroom's socket per request, so the missing piece is the per-session decision. It has to be made without clobbering Anthropic's prompt cache, which keys on the exact bytes of the prefix: headroom compresses the resent history deterministically each turn, so switching a session off headroom mid-session presents the original messages where the cached prefix holds compressed ones and rewrites the whole context. Switching on is cheap while headroom keeps the prefix frozen. A bypass on a transient reset is the worst case, so a session already using headroom keeps replaying through it, which the hop already does.

The design: every launch goes through the door, which decides once per session at its first request and changes the decision only at a cache boundary (the cache has expired, the context is small relative to what compression would save, or the session just compacted). The work is split into the sub-issues below.

Activity

  1. added theissue type on Oct 12, 2026
  2. Mearman commented on Oct 12, 2026

    @Mearman
    MemberAuthor

    Shipped: usage records carry viaHeadroom, every launch goes through the door by default, a session's routing is decided once and kept while its cache is warm, a session goes direct while the daemon is unavailable and rejoins at its next cache boundary, agent-shim headroom enable|disable changes routing live, and a resumed conversation's pool pick weighs its context size.

    Two things differ from the first plan. The small-context boundary is not implemented, since no recorded quantity says what compression would save, so any threshold would be invented (noted in the sub-issue). The resume pick reads the conversation's transcript rather than depending on the session-to-identity binding, which stays tracked in the issue that names it.

    The one open check, whether headroom's compressed prefix really keeps the cache as warm as a direct session, needs records from a release that has the field and is tracked separately in #412.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions