Skip to content

fix(ledger): read the line budget in command position, and read sed ranges - #742

Merged
fajarhide merged 2 commits into
mainfrom
fix/741-738-line-budget-command-position
Sep 1, 2026
Merged

fix(ledger): read the line budget in command position, and read sed ranges#742
fajarhide merged 2 commits into
mainfrom
fix/741-738-line-budget-command-position

Conversation

@fajarhide

@fajarhide fajarhide commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Two errors at opposite ends of one function, both found within a day of #737 shipping it.

rations_its_output matched head or tail anywhere in the string, so grep -rn head src/ and ls | grep tail turned the project scope off for the whole reply (#738). And it covered head and tail only, so sed -n 60,200p of a source file fell straight through: 28 lines of a JSX render branch folded on the file's first appearance in a cleared context, with the next step being an edit to that render path (#741). An edit written against a marker is worse than a wasted retrieval.

The name now counts only in command position, and sed addresses that name line numbers count. A pattern address and a substitution do not, because those filter, which is a different claim from naming the lines wanted.

Driving the built binary through --post-hook, same store, same primed block, same payload, only the command differing. Three arms must not fold and three must:

sed -n 60,200p app/run-panel.tsx      no rewrite     #741
sed -n '60,200p' app/run-panel.tsx    no rewrite     #741, quoted
cat notes.md; echo ---; tail -5 x     no rewrite     #735, still works
grep -rn head src/                    folds          #738
sed -n '/failed/p' build.log          folds          a pattern is not a line budget
cat handlers.log                      folds          the feature is intact

make ci green. Four break tests, each with its diff proven before the run: command-position guard removed, sed arm removed, sed address check neutered, and the -c/-f exclusion removed. All four went red.

awk is deliberately uncovered and its negative case is in the test, so a future change that starts matching it is a decision rather than a side effect. Its line selection is an expression rather than a flag, and the obvious guess matches {print NR}, which prints line numbers rather than selecting them.

One known limit, stated rather than hidden: a separator glued to the following token (cat a.md;tail -5 b) is not recognised and folds as before. Agents write the spaced form, and a tokeniser is a larger thing than this predicate deserves.

Closes #738, closes #741

Greptile Summary

The PR narrows line-budget detection to command position and adds numeric sed address recognition so ledger project folding does not hide explicitly requested lines.

Confidence Score: 4/5

The wrapped-command regression should be fixed before merging because it can again hide explicitly selected lines behind a project-history marker.

The command-position predicate excludes realistic wrapper-prefixed head, tail, and sed invocations, bypassing the guard added to preserve line-budgeted output; the sed scan also crosses command boundaries and can unnecessarily disable folding.

Files Needing Attention: src/ledger/mod.rs

Important Files Changed

Filename Overview
src/ledger/mod.rs Adds command-position and numeric-sed parsing, but wrapped selectors bypass the preservation guard and the sed argument scan crosses command boundaries.
changelog.d/738.fixed.md Documents the command-position false-positive correction and its intentional glued-separator limitation.
changelog.d/741.fixed.md Documents preservation of output selected by numeric sed addresses.

Fix all with Greploop Fix All in Claude Code Fix All in Codex Fix All in Cursor Fix All in Conductor

Prompt To Fix All With AI
### Issue 1
src/ledger/mod.rs:947-949
**Wrappers bypass line-budget detection**

When `head`, `tail`, or `sed` is invoked through a supported wrapper such as `docker exec app tail -5 app.log`, `opens_a_command` rejects the selector because its preceding token is not a shell separator. The project-fold guard then fails to preserve the explicitly selected lines, causing them to be replaced by a retrieval marker in compound output.

### Issue 2
src/ledger/mod.rs:954-955
**Sed scan crosses command boundaries**

For a command such as `sed -n '/failed/p' build.log; echo 5p`, the `sed` branch scans every remaining token and mistakes the later command's `5p` argument for a numeric sed address. This unnecessarily disables project folding for the entire payload and consumes context with repeated output.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "docs(changelog): record the command-posi..." | Re-trigger Greptile

Greptile also left 2 inline comments on this PR.

…anges

`rations_its_output` shipped in #737 with two errors at opposite ends of the
same function, and both were found within a day of it landing.

It matched the name anywhere in the string, so `grep -rn head src/` and
`ls | grep tail` turned the project scope off for the whole reply. The word was
somebody else's argument, not a reader rationing itself, and both shapes run
constantly in this repository. Greptile raised this on the PR and it merged
before the fix (#738).

It covered `head` and `tail` only, so it shipped one command family too narrow.
`sed -n 60,200p` of a source file names its lines exactly as `tail -5` does, and
the maintainer hit it on a fresh context: 28 lines of a JSX render branch folded
on the file's first appearance, with the next step being an edit to that render
path. An edit written against a marker is a worse outcome than a wasted
retrieval (#741). `registry::passes_through_verbatim` already groups `sed` with
`head` and `tail` for the collapse fallback under this same reasoning, which is
the precedent that should have been read the first time.

Command position is decided from the end of the previous token, because a shell
separator is normally glued there (`notes.md;`). A separator glued to the
following token is not recognised and that shape folds as before; a tokeniser is
a larger thing than this predicate deserves, and agents write the spaced form.

`awk` stays uncovered on purpose. Its line selection is an expression rather than
a flag, no report has produced one, and the obvious guess is wrong: `NR` appears
in `{print NR}`, which prints line numbers rather than selecting them. The
negative case is in the test so a future change that starts matching it is a
decision and not a side effect.

Closes #738, closes #741
Comment thread src/ledger/mod.rs
Comment on lines 947 to +949
tokens.iter().enumerate().any(|(i, tok)| {
let base = tok.rsplit('/').next().unwrap_or(tok);
(base == "head" || base == "tail")
&& tokens.get(i + 1).is_none_or(|next| {
if !opens_a_command(&tokens, i) {
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Wrappers bypass line-budget detection

When head, tail, or sed is invoked through a supported wrapper such as docker exec app tail -5 app.log, opens_a_command rejects the selector because its preceding token is not a shell separator. The project-fold guard then fails to preserve the explicitly selected lines, causing them to be replaced by a retrieval marker in compound output.

Knowledge Base Used: Restore command-aware ledger folding

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/ledger/mod.rs
Line: 947-949

Comment:
**Wrappers bypass line-budget detection**

When `head`, `tail`, or `sed` is invoked through a supported wrapper such as `docker exec app tail -5 app.log`, `opens_a_command` rejects the selector because its preceding token is not a shell separator. The project-fold guard then fails to preserve the explicitly selected lines, causing them to be replaced by a retrieval marker in compound output.

**Knowledge Base Used:** [Restore command-aware ledger folding](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/reverts/incident-mitigation_737-20260830-command-budgeted-output-cdc73f6.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor Fix in Conductor

Comment thread src/ledger/mod.rs
Comment on lines +954 to +955
}),
"sed" => tokens[i + 1..].iter().any(|a| names_line_numbers(a)),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Sed scan crosses command boundaries

For a command such as sed -n '/failed/p' build.log; echo 5p, the sed branch scans every remaining token and mistakes the later command's 5p argument for a numeric sed address. This unnecessarily disables project folding for the entire payload and consumes context with repeated output.

Knowledge Base Used:

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/ledger/mod.rs
Line: 954-955

Comment:
**Sed scan crosses command boundaries**

For a command such as `sed -n '/failed/p' build.log; echo 5p`, the `sed` branch scans every remaining token and mistakes the later command's `5p` argument for a numeric sed address. This unnecessarily disables project folding for the entire payload and consumes context with repeated output.

**Knowledge Base Used:**
- [Graphs, ledger, and context analytics](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/docs/graphs-ledger-and-context-analytics.md)
- [Restore command-aware ledger folding](https://app.greptile.com/weekndlabs/-/custom-context/knowledge-base/fajarhide/omni/-/reverts/incident-mitigation_737-20260830-command-budgeted-output-cdc73f6.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor Fix in Conductor

@fajarhide
fajarhide merged commit f1fc2a4 into main Sep 1, 2026
14 checks passed
@fajarhide
fajarhide deleted the fix/741-738-line-budget-command-position branch September 1, 2026 14:44
fajarhide added a commit that referenced this pull request Sep 2, 2026
rations_its_output was narrowed to command position by #742, and read that
position off the previous token. split_whitespace has already eaten the
newline by then, so a two-line command is one command and nothing on the
second line can open one. That is the #741 shape back for the way an agent
actually writes a command:

    cd /path/to/repo
    sed -n '58,95p' src/app.tsx

Replaying both versions of the predicate over the 4,193 commands recorded in
~/.omni/omni.db, the narrowing gave up the guard on 168 of them, and 136 of
those to a newline. 3.2% of the corpus, folding lines the caller had just
named.

Split the reply at its separators and ask the question per segment. A newline
is one of them, the sed scan can no longer reach a later command's 5p, and a
separator glued to the following token works too. sudo, env, time, nohup and
VAR=value introduce a command without being one, and a wrapper's argument
list is somebody else's command line, so the name counts anywhere inside it.

Closes #750
Closes #751
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant