Repository navigation
fix: keep a block in view when its preview would tell the model too little - #17
Merged
Merged
Conversation
The stub's label names what the model saw in place of the block: diff summary, match map, code outline, content table, json index, prose digest, or only the first line. --blocks now writes it per result.
…ittle On 1,491 recorded sessions a prose digest was reopened 34 to 80% of the time for no net saving, and in this session's real stubs four of six blocks under 2 KB shown only by their first line had to be fetched back. Such blocks now stay in view, booked as thin_preview; repeats, relevance splits and prose mode are unchanged. Held out, on the mod's path, reopens per session fall from 4.5 to 2.3 and the context saving from 8.6% to 5.2%. Tests that used prose as their offloadable fixture now use code, which still offloads with an outline.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Running the mod in a real session, small results kept coming back as an 80-character head preview,
[559 bytes offloaded, preview: valid bindings/...], hiding exactly the lines needed: a list of checks, a version, a count. Of the twelve real offloads in that session that had only a head preview, four of the six under 2 KB had to be fetched back, against one of the six above.secondwind evalnow records each offload's preview kind. On 1,491 recorded sessions:The prose digest never pays. For small head-only blocks the corpus measure looks good, but it counts a reopen only when the model later quotes a hidden token, and the result of a short check is read for a decision rather than quoted; the session above shows reopens about three times higher than the measure for those blocks.
What
offload::worth_offloadingdecides from the preview a block would get: a prose digest is never offloaded, and a block whose only preview is its first line is offloaded only from 2 KB. Such a block stays in view, and inline codecs still apply. The refusal is booked asthin_preview. Blocks under the store's 512-byte minimum keep their old label. Byte-identical repeats, JSON relevance splits and opt-in prose mode are unchanged.Trade-off, held-out half, Claude Code mod's path (Bash, gated)
Reopens halve; token savings drop by about 40% on the corpus measure, which undercounts these reopens.
MIN_HEAD_OFFLOAD_BYTESis the dial: 1 KB keeps more savings and removes fewer turns. Every replay: 0 unfaithful blocks.Tests
A new test pins that a short head-only status listing stays in view through the optimizer, that the same listing at four times the size still offloads, that prose stays in view, that code still offloads with its outline, and that tiny blocks keep their old label. Four existing tests used prose as their offloadable fixture; they now use code lines, which keeps what each tests (offload mechanics, the cache guard, replay). Optimizer: 288 passed; CLI: 29 passed. Clippy clean.