Skip to content

refactor(llm): unify parse/repair/retry policy behind a Transport seam - #582

Merged
ajianaz merged 1 commit into
developfrom
refactor/llm-structured-findings
Oct 8, 2026
Merged

ajianaz merged 1 commit into
developfrom
refactor/llm-structured-findings

Conversation

@ajianaz

@ajianaz ajianaz commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

What

Replace the 2.4k-line engine/llm.rs with engine/llm/{mod,transport,prompts,findings,repair}.rs, built around ONE "structured findings from a model response" step (findings::request_findings) that owns empty-response recovery, parse, repair, partial salvage and the retry policy. Review, streaming review and scan all go through it.

Why

Closes #570. parse_review_response / parse_scan_response had diverged, review_diff retried with a stricter prompt but review_diff_stream did not, prompt assembly was duplicated, and llm.rs printed to stdout.

How

  • Seam: trait Transport { async fn complete(&self, &Turn) -> Result<Completion> } (static dispatch). HttpTransport does non-stream and SSE; tests use scripted fakes (and an SSE replay that runs the real SseAccumulator) with no network.
  • Policy (findings.rs): complete_with_recovery (review/scan: reasoning models can return empty content — misleading 'EOF while parsing' error #536: empty + finish_reason=length doubles the budget up to the ceiling, else salvage from reasoning_content, else explicit error), parse_findings (strict -> truncation repair -> salvage of complete objects, one function for both flows), then one stricter-prompt retry. Used by review, stream and scan, and complete_with_recovery also by the raw commit-message calls.
  • No printing in the LLM layer: LlmEvents (status/delta/retry, all no-op by default). Spinner is an LlmEvents impl in llm/mod.rs; progress::StdoutStream prints deltas for review --stream / commit --stream.
  • Prompts: build_scan_prompt + shared push_focus_and_rules replace the inline assembly in scan_files; fix(llm): harden response parsing, prompt injection and stream limits #573 hardening (untrusted-data clause, diff fence, SSE caps) kept and now tested on the retry path too.
  • All 88 old tests were moved with their modules; the tautological chat_request_* tests now exercise the real build_request_body.

Behavior changes (user-visible)

  • Streaming review now retries once with the stricter prompt on an unparseable response (previously failed immediately). StdoutStream::retry prints a one-line notice to stderr before the second stream; the discarded first output has already been written to stdout (can't be unprinted).
  • Streaming (review and commit) now gets review/scan: reasoning models can return empty content — misleading 'EOF while parsing' error #536 empty-response recovery (doubled budget / reasoning salvage); streams capture finish_reason and reasoning_content deltas.
  • Scan now retries once on an unparseable response (previously never).
  • Review now also salvages complete objects from non-truncation damage (before: only EOF errors were repaired; others errored and retried). Salvage logs a warning.
  • Token usage on a retried response now sums both attempts (was: retry only). Retry starts from the escalated token budget.
  • Error text: an empty review response now reads provider returned an EMPTY response ... LLM response is not valid JSON (length=0) ... so both old assertions hold; scan's non-JSON message now also applies to review.
  • Trailing newline after streamed review output is printed by the caller (review.rs) on success, as before.
  • No CHANGELOG edit (see docs(changelog): document changes since v0.15.0 under [Unreleased] #581).

Testing

  • cargo test --features tree-sitter passes (1036 unit + integration)
  • cargo fmt --all -- --check passes
  • cargo clippy --all-targets --features tree-sitter -- -D warnings passes
  • New fake-transport tests: stream and non-stream give the same result and same retry on a malformed first response; stream retry regression; truncated JSON repaired without retry; ] and ||| inside a finding body (fix(llm): harden response parsing, prompt injection and stream limits #573); empty + length doubles the budget (plain and SSE); ceiling -> explicit error; reasoning salvage; retry starts from escalated budget; review/scan share the retry; hardening present on retry; SSE decoding across split chunks, line cap, total cap; request body max-token param naming.
  • Manual smoke against a live provider not done (no network/credentials in this environment).

Related Issues

Closes #570

Checklist

  • Branch name follows convention
  • Branch is from develop
  • Conventional Commits, signed off
  • No secrets or credentials committed
  • One logical change per PR

🤖 Generated with Claude Code

Review, streaming review and scan now share one findings step
(llm::findings) that owns empty-response recovery, parse, repair, partial
salvage and the single stricter-prompt retry. llm.rs is split into
transport / prompts / findings / repair modules; the LLM layer no longer
prints (LlmEvents sink), so tests drive the policy with a fake transport.

Closes #570

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: ajianaz <ajianaz@users.noreply.github.com>
@ajianaz
ajianaz merged commit 359840f into develop Oct 8, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(llm): unify parse/repair/retry policy; separate transport, prompts, and response repair

1 participant