Skip to content

Integrate Auto Patcher into OpenAnt - #197

Open
elaav wants to merge 30 commits into
masterfrom
auto-patcher
Open

Integrate Auto Patcher into OpenAnt#197
elaav wants to merge 30 commits into
masterfrom
auto-patcher

Conversation

@elaav

@elaav elaav commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR integrates the Auto Patcher research project into OpenAnt as a new openant patch command.

The new workflow generates evidence around AI-generated patches rather than automatically applying them. It produces a candidate patch, evaluates it using deterministic validation and adversarial review, and generates a Trust Report to help humans decide whether the patch should be trusted.

The target repository is never modified.


New workflow

pipeline_output.json
        ↓
openant patch
        ↓
Candidate Patch
        ↓
Challenger
        ↓
Deterministic Trust Signals
        ↓
Trust Report

What's included

  • New openant patch CLI command
  • Auto Patcher engine integrated into OpenAnt
  • Go → Python bridge
  • Repository grounding
  • Patch applicability validation
  • Challenger and repair loop
  • Deterministic Trust Signals
  • Trust Report generation
  • Recommendation policy
  • README documentation

Validation

Validated with:

  • Go test suite
  • Python test suite
  • CLI wrapper tests
  • Eligibility tests
  • End-to-end LIVE execution against a real repository (urllib3)
  • Successful Trust Report generation

Design principles

This feature intentionally does not:

  • automatically apply patches
  • modify the target repository
  • commit changes
  • push changes

Its purpose is to generate evidence that helps humans decide whether an AI-generated patch should be trusted.


Follow-up work

This PR focuses on the integration itself.

Potential follow-ups include:

  • Trust Report presentation improvements
  • Cost and token accounting improvements
  • Additional documentation
  • Further Trust Signal refinements

@gadievron

Copy link
Copy Markdown
Collaborator

Review: Integrate Auto Patcher (#197) — REQUEST CHANGES

⚠ Snapshot caveat — please read before the findings. Since #197 was opened, two feature PRs
have landed on main after it: #199repository-supplied threat models, application-type
context, and multi-language scanning
— and #198Swift language parser. This review is
pinned to #197's head 6d93b1b, which predates both
, so nothing below reflects the current
main. Their multi-language-scanning / repo-supplied-threat-model / new-parser changes plausibly
overlap the repository-scanning / language_support / grounding surface the auto-patcher grounds
against (the same surface several findings below touch), and #199 lands the threat-model + CLI work
that #197's cli.py merge was assumed additive against. Independent of everything in this review:
rebase #197 onto current main (past #198 + #199) and re-run the applicability + grounding path
there before merge.
That re-validation should happen regardless of these findings — a clean
re-check does not depend on them, and they do not substitute for it.

Thanks @elaav. Reviewed by cloning master, merging #197 locally, and running the tool
plus a focused bug-hunt pass that reproduced each finding in-process. Merge mechanics are clean:
0 conflicts on master; suite 1965 passed / 16 failed, the 16 identical on clean
master
(differential-checked — pre-existing environmental/parser failures, not this PR).

This review reports the defects (what breaks, where, and the reproduction); it intentionally
does not prescribe fixes — the direction is yours.

Scope: we opened the verdict-eligibility + trust-signal + grounding + diff/hygiene paths.
The rest of the 68-file / ~18k-line engine was not opened — treat its absence below as
not examined, not approved.

Changelog

  • 2026-07-30 (review 6 — Round-7 deploy-verdict sweep): a 1200-state truth-table sweep of the
    verdict path surfaced MED F-37 — the recommendation builder (pipeline.py:915) gates on a
    blacklist (safety != "High Risk"), so a deployment_safety="Not Verified" (impact analysis
    couldn't run for the language) still ships "Deploy After Validation", undoing the deliberate
    display fix that shows "Not Verified". Plus MED F-38 (patch_applicability._strip_fences
    deletes a trailing ```-content diff context line → a cleanly-applying patch falsely reported
    "Does Not Apply") and LOW F-39 (a failed re-run into a reused --output dir leaves a stale
    trust-report beside a fresh vulnerability file). F-37/F-38 reproduced; F-39 diff-verified. The
    deploy-verdict surface is now truth-table-characterized (4 wrong-verdict causes: F-23, F-24, F-37).
  • 2026-07-30 (review 5 — Round-6 "backward" pass): F-23 hardened — the applicability
    error/timeout state also falls through the Security-Improvement ladder (pipeline.py:775
    → "High"), not just the integrity arm; each arm is independently sufficient to yield
    "Deploy After Validation", so a partial one-ladder fix does not close the P0. Plus MED F-35
    (non-string description/vulnerable_codeTypeError aborts openant patch for an eligible
    finding) and MED F-36 (multi-file diff: a new file's diff --git/index lines are absorbed
    into the prior hunk → a correct patch is corrupted). All reproduced. (Note: an 8th abs-path
    ignore-token site exists in core/parser_adapter.py:53 but is pre-existing base code, not
    modified by Integrate Auto Patcher into OpenAnt #197
    — reported separately, not counted here.)
  • 2026-07-30 (review 4 — Round-5 delta): new MED F-33 — the "TRUNCATED OUTPUT DETECTED"
    banner contradicts its own Stage-Stop table (_STAGE_LABELS omits the finding_calibration
    stage PR Integrate Auto Patcher into OpenAnt #197 added, so a calibration-only truncation shows the banner while the table lists
    every stage clean — the reviewer can't see which stage was cut). Plus LOW F-34
    Behavior-Summary FUNC_RE matches the removed -def line on a rename hunk, naming the
    deleted function. A third observation (a +++ <content> diff reorder at
    diff_hunk_repair.py:137) folded into F-14, which already covered that branch. All reproduced.
  • 2026-07-30 (review 3 — Round-4 delta): new HIGH F-29 — the patch-generator fence
    regex silently truncates the diff at the first inner ```, so the real fix (e.g. an
    hmac.compare_digest change) vanishes while the patch reports applicable=True (the
    fail-silent pole of the F-23 spine). Plus MED F-30 (grep ranking evicts the real
    vulnerable file from LLM context) and MED F-31 (string impact rendered char-by-char →
    garbled vuln text into the whole pipeline) — plus LOW F-32 (classify_vuln_class check-order
    mislabels command-injection as path-traversal). All reproduced; 0 dedup against the prior
    findings (F-01..F-28).
  • 2026-07-30 (review 2 — bug-hunt pass folded in): new P0 headline F-23 (trust
    signals fail open on an applicability error → greenlights an unverified patch), plus
    HIGH F-25 (a valid patch hard-blocked) and HIGH F-18 (symlink → out-of-repo read).
    F-01 (repo-root fail-open) re-verified, still open, now co-HIGH. The earlier "F1 casing"
    item stays a latent MEDIUM. 24 reproduced observations → 17 new findings (7 abs-path sites
    are one finding, F-12) + 1 folded into F-07 (raised to MED). Reproduction raised our
    confidence, not the count.

What this PR gets right (verified, no hedge)

  • The target repo is never modified — applicability is git apply --check over stdin, no
    apply step anywhere. The single most important safety property, and it holds.
  • Fail-closed allow-listPATCH_ELIGIBLE is an explicit allow-list, not a denylist.
  • No silent mock (on the CLI path)openant patch hard-fails on unset LLM_PROVIDER
    before the LLM layer loads. (Heads-up: the lower llm_client layer has an is_mockcall_llm
    desync — F-19 below — reachable only by non-CLI callers; the CLI guard is genuinely sound.)
  • Opaque Trust Report + unbiased input — never re-parses the report's verdict; withholds
    OpenAnt's own suggested_fix to keep the candidate independent. Per-run isolation + honest
    hedged notes throughout.

Merge blockers — the 5 HIGH

🔴 F-23 (P0, headline) — Trust signals fail open when the applicability check errors/times out.
_compute_trust_signals (pipeline.py:744-785) has no arm for applicable is None (the shape
check_applicability returns on a git apply --check timeout/exception, patch_applicability.py:54),
so it falls through to ✓ Clean / "Applies cleanly", Security ✓ High, Deployment ✓ Low Risk
→ recommendation "Deploy After Validation" for a patch that was never verified to apply.
Reproduced deterministically. The report even contradicts itself — the Applicability section prints
⚠ Error while the trust panel says Clean. The existing not_applicable → "Not Verified" (:838)
and skipped → "Not Verified" (:752) arms cover their cases; the error path slips past both.
The same error state also falls through the Security-Improvement ladder — :766 guards only
skipped, so the error reaches :775 (elif not still_vulnerable) → security_improvement "High",
"Adversarial review found no remaining exploit path". The two ladders are each independently
sufficient
to produce "Deploy After Validation": forcing the security signal to "Unknown" flips the
decision to "Manual Review Required", so guarding only the integrity arm (:758) leaves the P0 open.
The error state originates at patch_applicability.py:112 (timeout → applicable=None) and is
re-fabricated by pipeline.run's own except (:2235-2240).

🔴 F-25 — duplicate_assignment hygiene check hard-blocks a valid patch.
_check_duplicate_assignments (patch_hygiene.py:103-121) flags every ALL-CAPS name in
added − removed but never reads context (unchanged) lines, so it can't tell a genuinely-new
constant from a re-added one. A patch whose only change adds MAX_REQUEST_BYTES = 1048576 to an
existing file → HIGH duplicate_assignmentpatch_integrity "Critical Issues""Do Not
Apply"
(pipeline.py:893). Reproduced. (F-23 and F-25 are the two poles of the same trust
failure: greenlight-when-you-shouldn't vs block-when-you-should.)

🔴 F-29 (fail-silent) — The fence regex silently truncates the diff, so the real fix vanishes.
_extract_diff_block's _FENCE_RE (patch_generator.py:22-25) uses a non-greedy (.*?)```' that closes on the **first** ``` anywhere in the body — not a fence on its own line. When the LLM's diff legitimately contains a fenced example (patching a README/docstring with a code block), extraction stops at that inner fence and **drops the rest of the diff**.repair_hunk_headers` then
recomputes the `@@` counts to match the truncated body, so `git apply --check` returns 0 (this is
the reproduced hop) → `applicable=True` via `pipeline.py:1810/2209`, shown as the "Proposed patch" —
while the actual code fix (e.g. the `hmac.compare_digest` change in `auth.py`) is silently
discarded
. Reproduced with real modules + real `git apply` (`'hmac.compare_digest' in extracted →
False`; `git apply --check` rc=0). This is the fail-silent pole of the same spine as F-23/F-24:
the patch presents as complete, its security content gone.
(Distinct from F-20, which is prompt-injection of `vulnerability_text` at `:73` — this is the
extraction regex at `:22-25/38`.)

🔴 F-18 (security) — Symlink-following path traversal reads out-of-repo files into the LLM prompt.
A symlink inside the target repo pointing outside it (pkg/link.py → ../../secret.txt) is resolved
by the suffix branch and read by _grep_repo/_find_class_definitions with no containment check;
_safe_under (repo_locator.py:1111) calls relative_to on the unresolved path so it never
detects the escape. Since the auto-patcher runs untrusted repos, a planted symlink exfiltrates
host files into the generator prompt. Reproduced (.read_text() returned an out-of-repo secret).

🔴 F-01 — Repo grounding fails open to the current directory.
When --repo-root is omitted, Test-Support and Impact-Surface silently ground against Path.cwd()
while Patch-Applicability fails closed. Reproduced with the README invocation (no --repo-root):
the Trust Report listed hundreds of OpenAnt's own test files (273 in that run) as "existing
coverage" for a target-repo finding; with a real --repo-root, 0. The Go bridge inherits the
operator's shell CWD (invoke.go:62). (Re-verified still-present this round.)

🟠 MEDIUM (20) — full list + exact anchors in the attached registry; the notable ones:

  • patch.py's effective_verdict dropped the .lower() its siblings (analyzer.py,
    reporter.py:282) all apply. (A static read first suggested a silent HIGH; we ran the tool
    and found it milder — latent and loud, not silent: mainline prefers a lowercase
    stage2_verdict, so it only fires on the empty-stage2/uppercase edge, raising a visible
    "…not eligible for remediation".)
  • PATCH_ELIGIBLE has no PATCH_DROPPED complement and no producer-partition test — unlike
    the DISCLOSURE_ELIGIBLE invariant, which has both.
  • No repo_root validation--repo-root /tmp/does-not-existexit 0 with an empty report.
  • Non-dict nested location — a string location → opaque AttributeError in render.
  • F-24 impact-analysis failure → "Low Risk" (pipeline.py:1719, bare except) — same
    fail-open family as F-23. F-12 the abs-path ignore-token bug (one root cause, 7 sites): a
    repo under a build//venv//node_modules/ ancestor silently blanks grounding/tests/sinks
    because the ignore check iterates the absolute p.parts. F-21 challenger \b1\b regex flips
    the verdict on "No, fixed in hunk #1". F-27 calibration maps LLM blocks by position,
    relabeling a hypothesis observed. F-14/F-15 diff-repair misparses --- <content>/---prefixed
    lines → corrupts a valid patch (F-14 also covers the +++ <content> reorder at :137 — an added
    body line beginning ++ lands above the @@ header). F-19 the mock/LIVE desync (non-CLI callers). F-20 untrusted
    repo source concatenated raw into the generator prompt (input-side sibling of the fence issue).
    F-30 the Pass-2 grep ranks candidate files by summed occurrence with no per-token cap and
    no requirement that the named symbol be present — a generic backticked token (`path`) lets
    an unrelated file evict the file that defines the real vulnerable symbol from context (the
    model then patches the wrong file). F-31 render_vulnerability_markdown iterates impact/
    steps_to_reproduce assuming a list, but the producer (core/reporter.build_pipeline_output)
    emits them as strings → one bullet per character; the garbled markdown is both written to
    {finding_id}-vulnerability.md and fed as vulnerability_text into every downstream stage.
    F-33 the ⚠️ TRUNCATED OUTPUT DETECTED banner scans all stage stop-reasons, but the
    rendered "Stage Stop Reasons" table iterates only the 4-entry _STAGE_LABELS — which omits the
    finding_calibration stage Integrate Auto Patcher into OpenAnt #197 added (run_metadata.py:79/135). When only calibration truncates,
    the banner fires while the table lists every stage clean end_turn → the evidence table appears to
    refute the integrity warning, and the reviewer never sees which stage was cut. Reproduced;
    independently surfaced by two Round-5 lenses.
    F-35 render_vulnerability_markdown appends description (:82) and vulnerable_code (:86)
    raw — no coercion, unlike impact/steps — so a non-string value makes "\n".join(lines)
    (:98) raise TypeError: sequence item N: expected str instance, list found. Both fields come
    straight from reporter.py:408/409 (uncoerced model output); run_patch (:159) calls this after
    check_eligible, before the LLM, so an eligible finding with a list/dict description can never
    be patched (exit 2). Distinct from F-31 (silent char-iter of impact/steps). F-36 in a
    multi-file diff, a new file's diff --git/index header lines arrive while the prior hunk is still
    open, hit elif in_hunk (diff_hunk_repair.py:153) before the else that handles them (:156),
    and are counted as context in the prior hunk → its @@ header is rewritten wrong and git apply rejects a correct patch. Runs on every generated patch.
    F-37 the recommendation builder's rule 4 (pipeline.py:915) gates on a blacklist
    if improvement in ("High","Medium") and safety != "High Risk" → "Deploy After Validation". So
    deployment_safety="Not Verified" (impact/regression analysis could-not-run for the language,
    :838) is treated exactly like "Low/Medium Risk" and still ships deploy — even though the display
    ladder was deliberately changed (:828-831) to show "Not Verified" instead of "Low Risk".
    Reproduced with a clean apply, so it is independent of the applicability-error verdict bugs.
    F-38 patch_applicability._strip_fences (:37) deletes the last line whenever its .strip()
    equals ```/~~~ — but a unified-diff context line for a source line that is a ``` fence
    has a one-space prefix, so .strip() matches and the real diff line is deleted → the hunk line-count
    breaks → check_applicability returns "Does Not Apply" (exit 128, "corrupt patch") for a patch
    that git apply --check-es cleanly; the opening check (:35) only recognizes ```, so ~~~ is
    handled asymmetrically. (Fail-safe direction — rejects a good patch, never accepts a bad one.)

Nice-to-have (LOW/NIT follow-ups, not merge gates)

Fence-injection in the rendered markdown · diff parse_diff/str.splitlines() Unicode over-split
(F-16/F-17) · root-level tests/ mis-exclusion inflates impact (F-28)
· "None identified." treated as a real challenger finding · __future__ unused-import
false-positive · name-based grounding · concurrent-run overwrite · unbounded rglob · gate order ·
module-sink dedup collapse · classify_vuln_class check-order forces PATH_TRAVERSAL when a
CWE-22/23 signal co-occurs with command-injection → wrong sink guidance (F-32, conditional) ·
Behavior-Summary FUNC_RE matches the removed -def line on a rename hunk → names the deleted
function as "modified", misdirecting the "Validate behavior" guidance (F-34, advisory section) ·
a failed re-run into a reused --output dir leaves a stale trust-report.md beside a fresh
vulnerability.md — the paired artifacts describe different runs (F-39; diff-verified — the
write-ordering is confirmed in source, the stale-artifact outcome is inferred).
(Exact anchors in the registry; add inline only if you want them line-tagged.)

One framing note (not a code defect)

The Trust-Report top-line glyphs over-state the hedged per-signal notes beneath them —
F-23 is the concrete case: an errored, never-verified patch currently shows ✓ Clean. The ✓
speaks louder than the honest note beneath it.

Severity & scope

The 5 HIGH (F-23, F-29, F-25, F-18, F-01) are the merge-blockers; the MEDIUM cluster next; the
LOW/NIT are follow-ups. cli.py is a trivial additive merge (unique patch subcommand) — no
hunk collision. Scope: we opened the eligibility / trust-signal / grounding / diff / hygiene
paths; the rest of the 68-file PR was not examined — treat its absence here as not examined,
not approved.


Line-anchored findings (file:line for every finding)

Each comment states the defect and its evidence only — no remediation direction (that's in the internal registry).

Finding file:line comment
[F-23] utilities/autopatcher/pipeline.py:744 _compute_trust_signals has no applicable is None/error arm — an errored/timed-out applicability (patch_applicability.py:54) falls through to ✓ Clean/✓ High → "Deploy After Validation" for an unverified patch. The :838/:752 arms cover not_applicable/skipped but not the error path.
[F-25] utilities/autopatcher/patch_hygiene.py:103 _check_duplicate_assignments flags added − removed ALL-CAPS names without reading context lines → a new-constant patch is HIGH duplicate_assignment → "Do Not Apply" (pipeline.py:893).
[F-18] utilities/autopatcher/repo_locator.py:1111 _safe_under calls relative_to on the unresolved path, and _iter_files/_grep_repo/_find_class_definitions apply no containment check → a symlink escaping the repo is read into the prompt.
[F-29] utilities/autopatcher/patch_generator.py:22 _FENCE_RE's non-greedy (.*?)```' closes on the **first inner** ``` (not a full-line fence), so a diff containing a fenced example is truncated;repair_hunk_headers` re-fits `@@` and `git apply --check` rc=0 → the real fix is silently dropped while `applicable=True`. Distinct from F-20 (injection at `:73`).
[F-30] utilities/autopatcher/repo_locator.py:875 _grep_repo ranks files by sum(len(pat.findall(content)) for pat in patterns) (merge :385-387, sort :900) with no per-token cap and no requirement the named symbol be present → a generic backticked token (`path`) lets an unrelated file evict the file defining the real vulnerable symbol from context.
[F-31] core/patch.py:91 render_vulnerability_markdown does impact = finding.get("impact") or [] then iterates (also steps_to_reproduce at :96), but the producer core/reporter.build_pipeline_output emits both as strings → one bullet per character; the garbled text is written to {finding_id}-vulnerability.md and fed as vulnerability_text downstream. (Sibling of F-05 — same function, same missing-guard root.)
[F-32] utilities/autopatcher/vulnerability_patterns.py:130 classify_vuln_class checks path-traversal CWEs (:130) before command-injection (:132) over the whole advisory body → any advisory mentioning CWE-22/23 is classified PATH_TRAVERSAL even when the real class is command injection; extract_repo_sinks then scans os.path.normpath instead of os.system/shell=True, omitting the real sink. (Conditional: needs a co-occurring path-traversal signal.)
[F-33] utilities/autopatcher/run_metadata.py:79 render_metadata_section: any_truncated (:102-104) scans all stage stop-reasons and fires the ⚠️ TRUNCATED OUTPUT DETECTED banner (:107), but the "Stage Stop Reasons" table (:135) iterates only the 4-entry _STAGE_LABELS, which omits the finding_calibration stage #197 added (finding_calibration.py:109). Calibration-only truncation → banner present, table all-clean end_turn, no calibration row → the table appears to refute the warning and the cut stage is invisible.
[F-34] utilities/autopatcher/behavior_summary.py:43 FUNC_RE = ^[\+\-\s]*def\s+… includes -, and removed (-) lines precede added (+) lines in a hunk, so on a rename hunk (-def run_raw_query+def login_user) the first match is the removed function; analyze (:56) returns function="run_raw_query" → flows into ### Behavior Summary (pipeline.py:1987) + the "Validate behavior" action, pointing testing at the deleted function. (Advisory section; LIVE-only.)
[F-35] core/patch.py:82 render_vulnerability_markdown appends description (:82) and vulnerable_code (:86) raw into lines (no coercion, unlike impact/steps), so a non-string value makes "\n".join(lines) (:98) raise TypeError: sequence item N: expected str instance, list found. Fields come uncoerced from reporter.py:408/409; run_patch (:159) calls this after check_eligible/before the LLM → an eligible finding with a list/dict description can never be patched (exit 2). Distinct from F-31 (silent char-iter of impact/steps).
[F-36] utilities/autopatcher/diff_hunk_repair.py:153 In a multi-file diff, a new file's diff --git a/… b/… / index … lines arrive while the prior hunk is still open (in_hunk True), so they hit elif in_hunk (:153) before the else at :156 (# preamble, diff --git lines, etc.) that is meant to handle them → they are appended to the prior hunk body as context and its @@ header is recomputed wrong (@@ -1,2 +1,2 @@@@ -1,4 +1,4 @@), so git apply rejects a correct patch. Runs on every generated patch (pipeline.py:2214/2281/2347). Distinct from F-14 (---/+++ single-line prefixes).
[F-37] utilities/autopatcher/pipeline.py:915 _build_recommendation_v1 rule 4 is if improvement in ("High","Medium") and safety != "High Risk" → "Deploy After Validation" — a blacklist on safety. deployment_safety="Not Verified" (impact/regression analysis not run for the language, :838) therefore ships deploy, undoing the deliberate display fix at :828-831 that shows "Not Verified" instead of "Low Risk". Reproduced with a clean apply (independent of the applicability-error verdict bugs F-23/F-24).
[F-38] utilities/autopatcher/patch_applicability.py:37 _strip_fences does if lines[-1].strip() in ("```","~~~"): lines = lines[:-1] — a unified-diff context line for a source line that is itself a ``` fence has a one-space prefix, so .strip() matches and the real diff line is deleted → hunk line-count breaks → check_applicability returns "Does Not Apply" (exit 128, "corrupt patch") for a patch that git apply --check-es cleanly (consumed at pipeline.py:2234/2291/2357). The opening check (:35) only matches ```, so ~~~ is asymmetric.
[F-39] core/patch.py:221 run_patch writes {id}-vulnerability.md at :161 (before _run_pipeline at :178) and {id}-trust-report.md at :221 (after), non-atomically. On a re-run into a reused --output dir where the pipeline raises mid-way, the vulnerability file is refreshed but the previous run's trust-report persists (ends with a full Run Metadata table, reads current) → the two paired artifacts describe different runs. (Diff-verified: write-ordering confirmed; stale-artifact outcome inferred. Distinct from F-09 concurrency.)
[F-12] utilities/autopatcher/repo_locator.py:262 Abs-path ignore-token bug (also language_support.py:63, testing_support.py:26, vulnerability_patterns.py:249, repo_locator.py:863,938, impact_surface.py:366) — the ignore check iterates the absolute p.parts, so a repo under a build//venv/ ancestor silently blanks. One root cause, 7 loci.
[F-01] utilities/autopatcher/pipeline.py:1546 Fails open: ts_root = … else Path.cwd() grounds Test-Support against the shell CWD when repo_root is None.
[F-01] utilities/autopatcher/pipeline.py:2483 Same else Path.cwd() fail-open on the Impact-Surface path.
[F-01] utilities/autopatcher/impact_surface.py:359 Same else Path.cwd() — grounds on CWD when repo_root is None.
[F-01] apps/openant-cli/internal/python/invoke.go:62 cmd.Dir set only when non-empty → inherits the operator's shell CWD.
[F-02] core/patch.py:42 effective_verdict returns the verdict verbatim; siblings (analyzer.py:148,150,329, reporter.py:282) all .lower(). Fires on the empty-stage2/uppercase edge → a visible "…not eligible for remediation".
[F-03] core/verdict_taxonomy.py:111 PATCH_ELIGIBLE has no PATCH_DROPPED complement and no producer-completeness assertion (cf. :20-21 for disclosure).
[F-04] core/patch.py (run_patch, repo_root entry) No existence/dir check on repo_rootexit 0 on a nonexistent path with an empty report; no-repo / not-git / nonexistent all render "Skipped" today.
[F-05] core/patch.py:68 location = finding.get("location") or {} guards None but not a non-dict; a string locationAttributeError at .get("file") (:76).
[F-06] core/patch.py:80-88 vulnerable_code/description/impact inlined verbatim in a bare fence — a in the value breaks out into live markdown feeding downstream LLMs.
[F-07] utilities/autopatcher/impact_surface.py:404 any(h in f.lower() for h in ENTRYPOINT_HINTS) is unanchored — 'app'⊂'mapper', 'main'⊂'domain' force entrypoint_hit=True → impact high → deployment_safety "High Risk" on ordinary filenames. MEDIUM (raised from LOW on the reproduced entry-point evidence).
[F-11] core/patch.py:137 LLM_PROVIDER gate precedes check_eligible (:154) → eligibility/misspelling errors are masked when no provider is set.

(F-08/F-09/F-10 are follow-up nits mentioned in the summary; add inline only if you want them line-anchored.)

@gadievron

Copy link
Copy Markdown
Collaborator

Review follow-up (round 9) — 4 additional findings on #197

A delta on the main consolidated review. Same method: each finding reproduced first-hand at #197 head 6d93b1b. Bug information only — no fix prescriptions. IDs continue the stable registry (F-40..F-43).

Same snapshot caveat as the main review: this batch is still pinned to 6d93b1b and does not reflect the now-merged #198/#199 — the independent rebase-and-recheck against current main recommended there still applies, regardless of these findings.

MEDIUM

  • [F-40] patch_hygiene.py:146_imported_names ignores the import X as Y alias. The plain-import branch returns m.group(1).split(".")[0] and never reads the as alias, so import numpy as np computes the local name numpy while the code uses np.… → the usage check never matches → false unused_import (MEDIUM). The from X import Y as W branch handles aliases correctly (a control diff produces no finding), which isolates the plain-import branch as the defect; the import a, b comma form is mishandled too. Repro: import numpy as np used by np.array([1])[('unused_import','MEDIUM')].

  • [F-41] patch_hygiene.py:61_parse_file_patches misparses a +++ -content line as a file header → false HIGH empty_hunk → "Do Not Apply". The elif line.startswith("+++ ") header branch is tested before the hunk-add branch, so a hunk that adds a line whose content begins ++ (emitted as +++ …) is swallowed as a from-file header; _flush() then runs with empty added/removed lines → a false HIGH empty_hunk on the real file (plus a phantom _FilePatch). Traced end-to-end: _compute_trust_signalspatch_integrity="Critical Issues"_build_recommendation_v1"Do Not Apply", blocking a correct patch. Repro: diff line +++ new entry[('empty_hunk','HIGH'),('empty_hunk','HIGH')]"Do Not Apply". (This is patch_hygiene's own diff parser — a distinct locus from the earlier diff_hunk_repair +++ finding, and this one is patch-blocking.)

  • [F-42] impact_surface.py:276_resolve_symbol_at_line attributes a decorator-only change to the enclosing class, not the method. Python's AST records FunctionDef.lineno at the def line, so a method's span starts there and the decorator line above it falls inside the class span but outside the method span. A hunk that changes only a decorator (-@require_login / +@require_admin) resolves to the class (changed_symbols=['UserController']), or to nothing at module level → the wrong/empty symbol flows into _search_usages + _classify, inflating or zeroing the impact of a security-relevant authorization change. Repro: for class UserController: / @require_admin / def delete(...), _resolve_symbol_at_line(idx, 2) (the decorator line) → 'UserController' (should be 'delete').

LOW

  • [F-43] testing_support.py:42_module_name_for_path fails same-module test detection on a src/-layout repo. The dotted module name is derived relative to repo_root, so src/mypkg/core.py yields src.mypkg.core, but a real test imports from mypkg.core import … (the import root is src/, not the repo root). The anchored regex (:73) never matches → proximity stays "repo" → the report states "Test Support: None" with a −0.15 penalty even though a same-module test exists. Flat layout with the identical test scores "Some" / 0.0. Repro: src/mypkg/core.py + tests/test_flows.py (from mypkg.core import handle) → "repo"/"None"/−0.15; flat mypkg/core.py → "same-module"/"Some"/0.0. (Advisory — rating + confidence delta.)

Severity by the same rule as the main review — current = min(claimed, evidence-tier-cap); all four are exec-proven (reproduced first-hand at 6d93b1b), so each sits at its reproduced level. Round-9 registry entries: F-40F-43.

elaav added 4 commits July 30, 2026 12:36
- add patch CLI command
- merge Auto Patcher engine
- generate Trust Reports for findings
- add integration tests and live smoke validation
elaav added 3 commits July 30, 2026 14:15
- Fail closed when applicability is unavailable (F-23)
- Treat unavailable impact analysis as Not Verified (F-24)
- Enforce explicit allowlists for Deploy After Validation (F-37)
- Document recommendation policy invariants
- Add boundary and regression tests for trust policy
@gadievron

Copy link
Copy Markdown
Collaborator

Review follow-up (round 8) — 4 additional findings on #197

A further delta on the main consolidated review (companion to the round-9 follow-up). Same method: each finding reproduced first-hand at #197 head 6d93b1b. Bug information only — no fix prescriptions. IDs F-44F-47.

Same snapshot caveat as the main review: pinned to 6d93b1b; does not reflect the now-merged #198/#199 — the independent rebase-and-recheck against current main still applies, regardless of these findings.

These cluster in the patch-hygiene parser — three false positives that block or downgrade a correct patch — plus one LLM-response parser bug.

HIGH

  • [F-44] patch_hygiene.py:52_parse_file_patches mislabels is_new_file by one file in a multi-file diff → a valid patch is blocked. The flush that finalizes file N reads the next file's +++/from-path, so the new-file flag is shifted by one file. In a 2-file diff where file 1 is a genuine new file adding constants and file 2 edits an existing file, file 1 is parsed is_new_file=False, so its brand-new constants lose the new-file exemption and are flagged HIGH duplicate_assignment → the patch cascades to "Do Not Apply", blocking a correct multi-file patch. (Symmetric false-negative: an existing file preceding a new file is mislabeled is_new=True, so a genuine duplicate in it is silently missed.) Repro: a 2-file diff, new file first (--- /dev/null + +++ b/consts.py adding MAX_SIZE/TIMEOUT, then an app.py edit) → [('duplicate_assignment','HIGH'),('duplicate_assignment','HIGH')]; the same patch with the new file last → [] (order-dependent).

MEDIUM

  • [F-45] patch_hygiene.py:58 — a removed -- comment line (emitted --- …) is eaten as a from-file header → false HIGH empty_hunk → "Do Not Apply". A patch that removes only a -- deprecated comment line (SQL/Lua/Ada/Haskell) emits diff body --- deprecated, which hygiene's own parser swallows as a header instead of a removal; the file then has zero added/removed lines → HIGH empty_hunk "no-op, should be removed" → "Do Not Apply". Repro: removing a SQL -- deprecated line → [('empty_hunk','HIGH')]. (The removed-side analog of the earlier +++ finding, in patch_hygiene's own parser; the Do-Not-Apply cascade is real, not advisory.)

  • [F-46] patch_hygiene.py:162_check_unused_imports flags an import used only by context code. Usage is searched only within the patch's added non-import lines; unchanged context lines aren't retained. So adding a missing import html where the existing body already calls html.escape(x) (a context line) is flagged MEDIUM unused_import → integrity Clean → Minor Issues. "Add a missing import" and __init__ re-export (name already in __all__) are common correct patches. Repro: adding import html used by an existing html.escape(x) context line → [('unused_import','MEDIUM')].

  • [F-47] pipeline.py:154_split_review splits on an inline section keyword. The section regex is compiled without re.MULTILINE and its **…** branch has no line-start anchor, so a reviewer reply whose Validation notes body naturally cross-references **Affected areas** is split as if a new header appeared mid-line. The last-write-wins loop then overwrites affected_areas with the tail of Validation notes and truncates Validation notes — both render verbatim into the report (:2003 / :2012), so a reviewer sees a wrong Affected-areas and incomplete validation steps. Repro: a Validation-notes body saying "As noted … under Affected areas, verify callers." → affected_areas = ', verify callers.\n- Run the SSRF regression suite' (real client.py content dropped), validation_notes = 'As noted in the section above under'.


Severity by the same rule as the main review — current = min(claimed, evidence-tier-cap); all four are exec-proven (reproduced first-hand at 6d93b1b). Registry entries: F-44F-47.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants