Skip to content

Make search output self-describing for LLM consumers - #7

Merged
pinkhasn merged 4 commits into
mainfrom
llm-query-header-and-absence
Aug 13, 2026
Merged

Make search output self-describing for LLM consumers#7
pinkhasn merged 4 commits into
mainfrom
llm-query-header-and-absence

Conversation

@pinkhasn

Copy link
Copy Markdown
Owner

Six output-informativeness features for LLM-facing modes, from the idea list discussed in-session. All are additive; the only default-output changes are deliberate (new footers, role tags), and every feature has an opt-out or is opt-in.

Query legend (-desc) and zero-match footer

  • -desc <text> names a pattern's meaning (also description in -patterns-from rules). When any pattern has one, -llm/-elide/-rollup output opens with an echo of the interpreted query plus a one-line legend per pattern — result blocks survive transcript summarization on their own.
  • A batched pattern with zero kept matches is named in a trailing --- no matches: ghost (1 of 2 patterns) --- footer instead of vanishing silently. Whole-scan, post-filter counts; qualified with (scan stopped early) when a limit cut the scan.

Per-match role tags

  • New lexer module (src/roles.cpp): single-pass, escape-aware comment/string span index per matched file (7 scope-pack languages + py/sh/rb/yaml/toml), plus import-line marking.
  • Surfaces: "role" field in JSONL, [comment]/[string]/[import] tags in -llm, $ROLE in -format. With -scope active, a hit on a signature line renders [def func X] instead of the misleading [in func X].
  • Position-accurate roles win (a trailing comment on a signature line is a comment). Built lazily only for files that produce output; -no-roles disables.

Scope rollup (-rollup)

  • New output mode: one line per innermost enclosing scope — line range, kind/name, count-descending per-pattern breakdown, first matched line as representative. Scopeless matches group as (top level), so files without a scope pack collapse to per-file summaries. Implies -scope auto.

Co-occurrence footer

  • With ≥2 matching patterns, -llm/-elide/-rollup end with per-pattern distinct-file counts and the overlap: exact both: crosstab for two patterns, multi-pattern: file list with pattern sets for more. both: 0 is stated explicitly.

Stable refs and the expand verb

  • hprscript expand <file:line[@hash]> prints a hit's full enclosing scope — the drill-down half of the search → expand loop; batches refs, falls back to a marked context window for scopeless lines, honors -max-block-bytes.
  • -refs appends a 6-hex trimmed-line content hash to -llm/-rollup line numbers. expand verifies it: moved lines are recovered by content (ref line moved: 6 → 7), vanished ones report stale in-band with exit 3 — never silently the wrong code.

Testing

  • ~110 new assertions in tests/run.sh (620 total, all passing), including literal FNV-1a hash values, moved-ref recovery, footer qualifiers, precedence cases, and mode-conflict validation.
  • Docs updated throughout: HPRSCRIPT.md sections for each feature, flag tables, --help, and both agent skills.

🤖 Generated with Claude Code

pinkhasn and others added 4 commits August 13, 2026 20:27
Two LLM-facing output additions for -llm/-elide modes:

- A pattern that ends the scan with zero kept matches is now named in a
  trailing '--- no matches: ... ---' footer instead of vanishing
  silently; qualified with '(scan stopped early)' when -limit or
  -max-output-bytes cut the scan. Counted over the whole scan, not the
  displayed subset, so -sample/-hotspots stay truthful.

- New -desc postfix modifier (and 'description' field in -patterns-from
  rules): when any pattern carries one, output opens with an echo of the
  interpreted query plus a one-line legend per pattern, making each
  result block self-describing. Input tokens are shared with
  -explain-plan so the two renderings cannot drift.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Classify every match's lexical role so readers stop re-deriving it:

- New RoleIndex (src/roles.cpp): a single-pass, escape-aware lexer
  builds comment/string spans per file (line + block comments with
  optional nesting, single-line/multiline/raw strings, Python triple
  quotes) and marks import-prefixed lines. Covers the scope-pack
  languages plus py/sh/rb/yaml/toml; unknown extensions get no tags.

- ScopeIndex::anchor_on_line identifies matches sitting on a signature
  line; -llm renders those as '[def func X]' instead of the misleading
  '[in func X]'. Position-accurate roles win: a trailing comment on a
  signature line is a comment, not a def.

- Surfaces: JSONL gains a "role" field (omitted for plain code), -llm
  appends [comment]/[string]/[import] tags, -format gains $ROLE.
  On by default for per-match output modes, built lazily only for
  files with kept matches; -no-roles disables.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two more LLM-facing output features:

- New -rollup output mode: one line per innermost enclosing scope with
  line range, kind/name, match count, a count-descending per-pattern
  breakdown, and the scope's first matched line as a representative.
  Scopeless matches group under a single "(top level)" row, so files
  without a scope pack collapse to a per-file summary. Implies -scope
  auto like -elide; shares the query header and result footers.

- Co-occurrence footer in -llm/-elide/-rollup: when >=2 patterns
  matched, a trailing '--- files: ... ---' line gives per-pattern
  distinct-file counts and the overlap — an exact 'both:' crosstab for
  two patterns, 'multi-pattern:' files with their pattern sets for
  more. 'both: 0' is stated explicitly; counts are whole-scan and
  post-filter, qualified on early stops.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The drill-down half of the search → expand loop:

- New 'hprscript expand <file:line[@hash]>' subcommand: prints the full
  enclosing scope of a search hit (scope pack resolved per file, -scope
  override honored) without the caller reconstructing a pattern and
  re-scanning. Multiple refs batch in one call; scopeless lines fall
  back to a numbered context window with the ref line marked;
  -max-block-bytes caps renders at a line boundary.

- New -refs search flag (-llm/-rollup): appends a 6-hex content hash of
  the line's whitespace-trimmed text to each line number. expand
  verifies it before rendering — a moved line is recovered by content
  and noted as '(ref line moved: A -> B)'; a vanished one reports
  in-band as stale with exit 3, never silently expanding wrong code.
  ref_hash6 in src/expand.hpp is the single definition of ref identity,
  shared by search output and expand verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pinkhasn
pinkhasn merged commit 448a284 into main Aug 13, 2026
3 checks passed
@pinkhasn
pinkhasn deleted the llm-query-header-and-absence branch August 13, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant