fix(ledger): read the line budget in command position, and read sed ranges - #742
Conversation
…anges `rations_its_output` shipped in #737 with two errors at opposite ends of the same function, and both were found within a day of it landing. It matched the name anywhere in the string, so `grep -rn head src/` and `ls | grep tail` turned the project scope off for the whole reply. The word was somebody else's argument, not a reader rationing itself, and both shapes run constantly in this repository. Greptile raised this on the PR and it merged before the fix (#738). It covered `head` and `tail` only, so it shipped one command family too narrow. `sed -n 60,200p` of a source file names its lines exactly as `tail -5` does, and the maintainer hit it on a fresh context: 28 lines of a JSX render branch folded on the file's first appearance, with the next step being an edit to that render path. An edit written against a marker is a worse outcome than a wasted retrieval (#741). `registry::passes_through_verbatim` already groups `sed` with `head` and `tail` for the collapse fallback under this same reasoning, which is the precedent that should have been read the first time. Command position is decided from the end of the previous token, because a shell separator is normally glued there (`notes.md;`). A separator glued to the following token is not recognised and that shape folds as before; a tokeniser is a larger thing than this predicate deserves, and agents write the spaced form. `awk` stays uncovered on purpose. Its line selection is an expression rather than a flag, no report has produced one, and the obvious guess is wrong: `NR` appears in `{print NR}`, which prints line numbers rather than selecting them. The negative case is in the test so a future change that starts matching it is a decision and not a side effect. Closes #738, closes #741
| tokens.iter().enumerate().any(|(i, tok)| { | ||
| let base = tok.rsplit('/').next().unwrap_or(tok); | ||
| (base == "head" || base == "tail") | ||
| && tokens.get(i + 1).is_none_or(|next| { | ||
| if !opens_a_command(&tokens, i) { | ||
| return false; |
There was a problem hiding this comment.
Wrappers bypass line-budget detection
When head, tail, or sed is invoked through a supported wrapper such as docker exec app tail -5 app.log, opens_a_command rejects the selector because its preceding token is not a shell separator. The project-fold guard then fails to preserve the explicitly selected lines, causing them to be replaced by a retrieval marker in compound output.
Knowledge Base Used: Restore command-aware ledger folding
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/ledger/mod.rs
Line: 947-949
Comment:
**Wrappers bypass line-budget detection**
When `head`, `tail`, or `sed` is invoked through a supported wrapper such as `docker exec app tail -5 app.log`, `opens_a_command` rejects the selector because its preceding token is not a shell separator. The project-fold guard then fails to preserve the explicitly selected lines, causing them to be replaced by a retrieval marker in compound output.
**Knowledge Base Used:** [Restore command-aware ledger folding](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/reverts/incident-mitigation_737-20260830-command-budgeted-output-cdc73f6.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| }), | ||
| "sed" => tokens[i + 1..].iter().any(|a| names_line_numbers(a)), |
There was a problem hiding this comment.
Sed scan crosses command boundaries
For a command such as sed -n '/failed/p' build.log; echo 5p, the sed branch scans every remaining token and mistakes the later command's 5p argument for a numeric sed address. This unnecessarily disables project folding for the entire payload and consumes context with repeated output.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/ledger/mod.rs
Line: 954-955
Comment:
**Sed scan crosses command boundaries**
For a command such as `sed -n '/failed/p' build.log; echo 5p`, the `sed` branch scans every remaining token and mistakes the later command's `5p` argument for a numeric sed address. This unnecessarily disables project folding for the entire payload and consumes context with repeated output.
**Knowledge Base Used:**
- [Graphs, ledger, and context analytics](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/docs/graphs-ledger-and-context-analytics.md)
- [Restore command-aware ledger folding](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/reverts/incident-mitigation_737-20260830-command-budgeted-output-cdc73f6.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.rations_its_output was narrowed to command position by #742, and read that position off the previous token. split_whitespace has already eaten the newline by then, so a two-line command is one command and nothing on the second line can open one. That is the #741 shape back for the way an agent actually writes a command: cd /path/to/repo sed -n '58,95p' src/app.tsx Replaying both versions of the predicate over the 4,193 commands recorded in ~/.omni/omni.db, the narrowing gave up the guard on 168 of them, and 136 of those to a newline. 3.2% of the corpus, folding lines the caller had just named. Split the reply at its separators and ask the question per segment. A newline is one of them, the sed scan can no longer reach a later command's 5p, and a separator glued to the following token works too. sudo, env, time, nohup and VAR=value introduce a command without being one, and a wrapper's argument list is somebody else's command line, so the name counts anywhere inside it. Closes #750 Closes #751
Two errors at opposite ends of one function, both found within a day of #737 shipping it.
rations_its_outputmatchedheadortailanywhere in the string, sogrep -rn head src/andls | grep tailturned the project scope off for the whole reply (#738). And it coveredheadandtailonly, sosed -n 60,200pof a source file fell straight through: 28 lines of a JSX render branch folded on the file's first appearance in a cleared context, with the next step being an edit to that render path (#741). An edit written against a marker is worse than a wasted retrieval.The name now counts only in command position, and
sedaddresses that name line numbers count. A pattern address and a substitution do not, because those filter, which is a different claim from naming the lines wanted.Driving the built binary through
--post-hook, same store, same primed block, same payload, only the command differing. Three arms must not fold and three must:make cigreen. Four break tests, each with its diff proven before the run: command-position guard removed,sedarm removed,sedaddress check neutered, and the-c/-fexclusion removed. All four went red.awkis deliberately uncovered and its negative case is in the test, so a future change that starts matching it is a decision rather than a side effect. Its line selection is an expression rather than a flag, and the obvious guess matches{print NR}, which prints line numbers rather than selecting them.One known limit, stated rather than hidden: a separator glued to the following token (
cat a.md;tail -5 b) is not recognised and folds as before. Agents write the spaced form, and a tokeniser is a larger thing than this predicate deserves.Closes #738, closes #741
Greptile Summary
The PR narrows line-budget detection to command position and adds numeric
sedaddress recognition so ledger project folding does not hide explicitly requested lines.head,tail, andsed.sedprint addresses.grep -rn head src/turns off the project scope (#737 follow-up) #738 and Ledger folds 28 lines of a source file on its first appearance in a cleared context: project-scope 'already shown' outlives the context that saw them (0.7.8) #741.Confidence Score: 4/5
The wrapped-command regression should be fixed before merging because it can again hide explicitly selected lines behind a project-history marker.
The command-position predicate excludes realistic wrapper-prefixed
head,tail, andsedinvocations, bypassing the guard added to preserve line-budgeted output; the sed scan also crosses command boundaries and can unnecessarily disable folding.Files Needing Attention: src/ledger/mod.rs
Important Files Changed
Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "docs(changelog): record the command-posi..." | Re-trigger Greptile