Repository navigation
feat: keep output that is usually read whole in view - #12
Merged
Merged
Conversation
On 1,491 recorded sessions a block is reopened 15% of the time under 500 tokens and up to 87% at a few thousand, and every reopen costs an extra model turn. A reopen policy, per command class and size, now decides whether a block may be offloaded; when it may not, only inline codecs apply. MCP optimize takes optional tool and command arguments and exec uses its own command. The built-in rates come from that corpus; eval --learn fits rates on half the user's sessions, replays the other half with the gate, and writes ~/.secondwind/policy.json. On held-out sessions the learned gate cut reopens per session from 23.6 to 4.8 with no unfaithful block. The optimize description no longer claims query helps text.
…12 bytes With the binary's reopen gate deciding what stays in view, the 6 KB floor only skipped savings: on held-out sessions it kept 13% of what the gate saves, and 512 bytes keeps 99.9%. A call to the binary takes about 30 ms.
Each state is drawn on the terminal and desktop surfaces: off offers to turn on, an empty session says where output will land, a net gain shows in the accent color and a net loss in red.
hardy30894
force-pushed
the
feat-reopen-gate
branch
from
October 23, 2026 02:36
3a10dae to
67e17c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #11 (the eval harness); review that first.
Why
On 1,491 recorded Claude Code sessions, how often the model needs an offloaded block back depends mostly on what produced it and how big it is:
Bash:grep |Bash:sedBash:catReadEvery reopen costs an extra model turn, so offloading a large
catbuys a turn and saves little.What
crates/optimize/src/policy.rs): Laplace-smoothed rates per command class and size band, falling back to the class, then to a prior. A block may be offloaded only when its predicted rate is under the threshold, 0.4. Otherwise the optimizer runs with offloading off, so only inline codecs, which never need a reopen, apply.optimizetakes optionaltoolandcommand;execuses its own command. Both classify with the same functionevaluses.secondwind eval --learnfits the rates on the even-numbered sessions, replays the odd ones with and without the gate, prints both, and writes~/.secondwind/policy.json.eval --gatereplays with the active policy. The built-in table holds the classes with at least 300 offloads in the corpus above.querydescription no longer claims it helps text output.Results on held-out sessions (learned on the other half)
For the mod, Bash only: the old 6 KB rule saved 6.6% of tool output with 5.9 reopens per session; the gate at 512 bytes saves 9.7% with 4.8, and cuts billed cost by about 21% versus 5%.
Tests
Policy learning, backoff, round trip and refusal of a bad file; the gate in
evaluate; end to end through the binary: a largecatstays in view over MCP andexecwhile the same block unnamed offloads, a learned policy replaces the built-in one, andeval --learnwrites a policy and reports held-out sessions. Mod tests: the command reaches the binary, output under 512 bytes never does.cargo test --workspace: 363 passed. Clippy clean.