Make search output self-describing for LLM consumers - #7
Merged
Conversation
Two LLM-facing output additions for -llm/-elide modes: - A pattern that ends the scan with zero kept matches is now named in a trailing '--- no matches: ... ---' footer instead of vanishing silently; qualified with '(scan stopped early)' when -limit or -max-output-bytes cut the scan. Counted over the whole scan, not the displayed subset, so -sample/-hotspots stay truthful. - New -desc postfix modifier (and 'description' field in -patterns-from rules): when any pattern carries one, output opens with an echo of the interpreted query plus a one-line legend per pattern, making each result block self-describing. Input tokens are shared with -explain-plan so the two renderings cannot drift. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Classify every match's lexical role so readers stop re-deriving it: - New RoleIndex (src/roles.cpp): a single-pass, escape-aware lexer builds comment/string spans per file (line + block comments with optional nesting, single-line/multiline/raw strings, Python triple quotes) and marks import-prefixed lines. Covers the scope-pack languages plus py/sh/rb/yaml/toml; unknown extensions get no tags. - ScopeIndex::anchor_on_line identifies matches sitting on a signature line; -llm renders those as '[def func X]' instead of the misleading '[in func X]'. Position-accurate roles win: a trailing comment on a signature line is a comment, not a def. - Surfaces: JSONL gains a "role" field (omitted for plain code), -llm appends [comment]/[string]/[import] tags, -format gains $ROLE. On by default for per-match output modes, built lazily only for files with kept matches; -no-roles disables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two more LLM-facing output features: - New -rollup output mode: one line per innermost enclosing scope with line range, kind/name, match count, a count-descending per-pattern breakdown, and the scope's first matched line as a representative. Scopeless matches group under a single "(top level)" row, so files without a scope pack collapse to a per-file summary. Implies -scope auto like -elide; shares the query header and result footers. - Co-occurrence footer in -llm/-elide/-rollup: when >=2 patterns matched, a trailing '--- files: ... ---' line gives per-pattern distinct-file counts and the overlap — an exact 'both:' crosstab for two patterns, 'multi-pattern:' files with their pattern sets for more. 'both: 0' is stated explicitly; counts are whole-scan and post-filter, qualified on early stops. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The drill-down half of the search → expand loop: - New 'hprscript expand <file:line[@hash]>' subcommand: prints the full enclosing scope of a search hit (scope pack resolved per file, -scope override honored) without the caller reconstructing a pattern and re-scanning. Multiple refs batch in one call; scopeless lines fall back to a numbered context window with the ref line marked; -max-block-bytes caps renders at a line boundary. - New -refs search flag (-llm/-rollup): appends a 6-hex content hash of the line's whitespace-trimmed text to each line number. expand verifies it before rendering — a moved line is recovered by content and noted as '(ref line moved: A -> B)'; a vanished one reports in-band as stale with exit 3, never silently expanding wrong code. ref_hash6 in src/expand.hpp is the single definition of ref identity, shared by search output and expand verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six output-informativeness features for LLM-facing modes, from the idea list discussed in-session. All are additive; the only default-output changes are deliberate (new footers, role tags), and every feature has an opt-out or is opt-in.
Query legend (
-desc) and zero-match footer-desc <text>names a pattern's meaning (alsodescriptionin-patterns-fromrules). When any pattern has one,-llm/-elide/-rollupoutput opens with an echo of the interpreted query plus a one-line legend per pattern — result blocks survive transcript summarization on their own.--- no matches: ghost (1 of 2 patterns) ---footer instead of vanishing silently. Whole-scan, post-filter counts; qualified with(scan stopped early)when a limit cut the scan.Per-match role tags
src/roles.cpp): single-pass, escape-aware comment/string span index per matched file (7 scope-pack languages + py/sh/rb/yaml/toml), plus import-line marking."role"field in JSONL,[comment]/[string]/[import]tags in-llm,$ROLEin-format. With-scopeactive, a hit on a signature line renders[def func X]instead of the misleading[in func X].-no-rolesdisables.Scope rollup (
-rollup)(top level), so files without a scope pack collapse to per-file summaries. Implies-scope auto.Co-occurrence footer
-llm/-elide/-rollupend with per-pattern distinct-file counts and the overlap: exactboth:crosstab for two patterns,multi-pattern:file list with pattern sets for more.both: 0is stated explicitly.Stable refs and the expand verb
hprscript expand <file:line[@hash]>prints a hit's full enclosing scope — the drill-down half of the search → expand loop; batches refs, falls back to a marked context window for scopeless lines, honors-max-block-bytes.-refsappends a 6-hex trimmed-line content hash to-llm/-rollupline numbers. expand verifies it: moved lines are recovered by content (ref line moved: 6 → 7), vanished ones reportstalein-band with exit 3 — never silently the wrong code.Testing
tests/run.sh(620 total, all passing), including literal FNV-1a hash values, moved-ref recovery, footer qualifiers, precedence cases, and mode-conflict validation.--help, and both agent skills.🤖 Generated with Claude Code