Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 26 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ context locally, then runs the ordered stages below.
| 8 | Hardware & portability | Drivers/HW: DMA, IRQ, barriers, endianness |
| 9 | Comment / code consistency | Audit comments touched by the patch against the actual code |
| 10 | Consolidation pass | Turns regular concerns into candidate additions |
| 11 | Strong adjudication and addition validation | Independently confirms specialist challenges and filters regular additions |
| 11 | Strong adjudication and addition validation | Independently adjudicates every baseline finding and filters regular additions |
| 12 | Additive merge and LKML report | Union survivors into the baseline and render the report |

`BORO_MODEL` is used for regular discovery and consolidation. In normal
Expand All @@ -265,21 +265,24 @@ message, and diff. In a normal review it uses `BORO_VALIDATION_MODEL`;
under `--fast` it uses `BORO_MODEL`. Regular discovery is independent
and can only contribute novel findings; it cannot rewrite or replace baseline
findings or their lore provenance. A specialist may separately challenge a
fast finding, but this is only a proposal. The baseline stays read-only while
all specialists run. The strong validation model then independently inspects
the repository and may confirm a proposal only by returning its complete,
exact-target, unhedged proof verbatim. The host applies only those confirmations;
validation failure, ambiguity, missing tool inspection, malformed output, or
`--validation-mode=off` preserves the baseline. Deterministic upstream-fix
findings are not challengeable. The validator also drops regular candidates
that clearly report the same underlying problem. Local merging removes exact
identities only, so distinct findings at the same location are retained.
fast finding, but this is only optional evidence and the baseline stays
read-only while all specialists run. The strong validation model then
independently adjudicates every baseline finding against the repository,
including findings no specialist challenged. It may DROP one only by returning
the complete exact finding with an unhedged structured contradiction proof
after repository-tool inspection. The host applies only exact, tool-verified
DROP decisions; validation failure, ambiguity, missing tool inspection,
malformed output, or `--validation-mode=off` preserves the finding.
Deterministic upstream-fix findings must receive `KEEP` and cannot be dropped.
The validator also drops regular candidates that clearly report the same underlying problem.
Local merging removes exact identities only, so distinct findings at the same
location are retained.

`--validation-mode` changes only the post-discovery stages:

- `filter` (default): adjudicate specialist challenges, validate regular
additions, union survivors into the fast baseline, then render LKML prose.
- `findings`: perform the same adjudication and additive merge while skipping
- `filter` (default): adjudicate every baseline finding, validate regular
additions, union survivors, then render LKML prose.
- `findings`: perform the same baseline adjudication and additive merge while skipping
LKML rendering.
- `off`: preserve every fast finding and add raw regular candidates without
validation.
Expand Down Expand Up @@ -378,11 +381,15 @@ The option `--validation-mode` selects whether (and how) regular-stage
candidate additions are validated:

- `filter` (default): validates regular additions, independently adjudicates
specialist baseline challenges, unions survivors with the protected fast
baseline, then renders per-commit LKML prose. The viewer / human
every protected baseline finding with one structured, repository-tool-backed
`KEEP` or `DROP` proof per finding, unions survivors with validated regular
additions, then renders per-commit LKML prose. Incomplete, reordered, or
inexact baseline adjudication is rejected and retried; a `DROP` is applied
only when its exact proof passes host validation. The viewer / human
report's Findings section shows `validated_findings`; the LKML section
shows prose built from those survivors.
- `findings`: performs the same challenge adjudication, additive validation,
shows prose built from those survivors. Commits with no survivors have no
LKML report body; the renderer does not independently add or remove findings.
- `findings`: performs the same baseline adjudication, additive validation,
and merge, **skips** the per-commit LKML
pass entirely (saves one LLM call per commit; the human report's LKML
section is empty in this mode). `scripts/boro-json-view` auto-detects
Expand All @@ -397,8 +404,8 @@ stdout instead of the human report - the same shape consumed internally,
with per-commit `findings[]`, `lkml_report` (filter/off modes only),
`validated_findings[]` (filter/findings modes when validation
succeeded), and a `usage_summary`. In filter/findings modes,
`findings[]` is the protected fast baseline after any specialist challenges
independently confirmed by the strong validator, and
`findings[]` is the protected fast baseline after exact, tool-verified DROP
decisions from the strong validator, and
`validated_findings[]` is the baseline plus accepted regular additions.
Each finding may carry optional `location` and `references` fields:

Expand Down
4 changes: 2 additions & 2 deletions resources/prompts/libvirt/inline-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,5 +61,5 @@ text code snippet or call chain when it helps.
- Order findings from most to least serious.
- Keep the tone the kind of reply a regular libvir-list reviewer would send.

If no issues remain after filtering, the report should simply state that nothing
of concern was found.
The caller skips this formatter when the validated findings set is empty. Format
every supplied finding; do not independently add or remove findings.
4 changes: 2 additions & 2 deletions resources/prompts/qemu/inline-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,5 +59,5 @@ text code snippet or call chain when it helps.
- Order findings from most to least serious.
- Keep the tone the kind of reply a regular qemu-devel reviewer would send.

If no issues remain after filtering, the report should simply state that nothing
of concern was found.
The caller skips this formatter when the validated findings set is empty. Format
every supplied finding; do not independently add or remove findings.
80 changes: 46 additions & 34 deletions resources/review-validation-findings.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The user message gives you a JSON object of this exact shape:
"diff": "<unified diff for the commit, may be truncated>",
"baseline_findings": [
{
"problem": "<immutable one-shot finding>",
"problem": "<protected one-shot finding to adjudicate>",
"severity": "Low|Medium|High|Critical",
"severity_explanation": "<proof>"
}
Expand Down Expand Up @@ -89,35 +89,44 @@ For each finding, decide one of:
is genuinely wrong, KEEP it - filtering is for clear false positives,
not for taste.

`baseline_findings` contains the protected one-shot review. It is read-only:
never rewrite or replace a baseline finding. DROP a regular candidate when it
reports the same underlying problem as a baseline finding, even if the wording
differs. Do not drop a candidate merely because it shares a location, function
name, or terminology with the baseline; distinct failure modes at the same line
are novel findings.

`baseline_false_positive_challenges` contains specialist proposals, not
trusted conclusions. Independently inspect the reviewed commit with repository
tools and decide whether each proof establishes with certainty that the
complete copied baseline finding is false. Confirm a challenge only when all
of its verified facts are independently established and the contradiction
makes the reported failure impossible. Missing evidence, lower severity,
plausibility, inability to reproduce, or an alternative interpretation is not
enough. If any assumption, ambiguity, or uncertainty remains, do not confirm
it. Return confirmed challenges verbatim under `confirmed_false_positives`;
never construct, rewrite, or strengthen a challenge. An empty array preserves
the baseline. You MUST execute repository tools before returning any confirmed
challenge; if tools are unavailable, confirm none.

For every candidate finding or baseline challenge whose conclusion depends on
a function-like macro, expand the complete invocation chain token by token.
`baseline_findings` contains the protected one-shot review. Independently
adjudicate EVERY entry, even when no specialist challenged it. Assign each
entry its host identity `fast-N`, where N is its zero-based array index. Return
exactly one `baseline_adjudications` record per baseline entry, in the same
order. Copy `baseline_id` and the complete finding object exactly. Every record
must contain repository-tool-verified proof: make `proof.finding_claim` equal
the finding's `problem` exactly, list concrete `verified_facts`, and explain in
`assessment` why those facts support or disprove the complete finding. Use
`verdict: "KEEP"` with `proof.conclusion: "supported"` unless the checked-out
tree conclusively proves the reported failure impossible. Only then use
`verdict: "DROP"` with `proof.conclusion: "false_positive"`. Missing evidence,
lower severity, plausibility, inability to reproduce, or an alternative
interpretation is not enough to DROP. If any assumption, ambiguity, or
uncertainty remains, KEEP. You MUST execute repository tools while adjudicating
the baseline. If tools are unavailable, KEEP every entry and state the concrete
facts available from the supplied commit material.

DROP a regular candidate when it reports the same underlying problem as a
surviving baseline finding, even if the wording differs. Do not drop a candidate
merely because it shares a location, function name, or terminology with the
baseline; distinct failure modes at the same line are novel findings.

`baseline_false_positive_challenges` contains optional specialist evidence, not
trusted conclusions and not the complete set of baseline findings to inspect.
Verify each proposal independently with repository tools. When its exact proof
is correct, it may inform the corresponding adjudication. When you independently
disprove an unchallenged baseline finding, construct the same strict proof
yourself. Never copy or strengthen an unverified specialist claim.

For every candidate or baseline finding whose conclusion depends on a
function-like macro, expand the complete invocation chain token by token.
At each level, bind formal parameters to actual arguments, substitute every
matching preprocessing token in the replacement list, and rescan for nested
expansion. Punctuation or member-access operators do not make a matching
parameter token literal. Account for stringification, token pasting, and
variadic arguments when present. KEEP a candidate, or reject a baseline
challenge, only according to the final expanded token stream rather than the
unexpanded spelling of an intermediate macro body.
variadic arguments when present. Adjudicate the finding only according to the
final expanded token stream rather than the unexpanded spelling of an
intermediate macro body.

Repository-verifiable absence/linkage claims are not matters of taste. Before
KEEP or TIGHTEN of a claim that a declaration, definition, export, stub,
Expand All @@ -138,9 +147,11 @@ Hard rules:
correspond to one in the input (by `location` and substance).
- Emit only surviving entries from `findings`. Never emit entries from
`baseline_findings`; the caller unions the result with the protected
baseline after applying only your confirmed challenges.
- Emit only verbatim entries from `baseline_false_positive_challenges` under
`confirmed_false_positives`. Omission is rejection and preserves the finding.
baseline after applying your exact, tool-verified adjudications.
- Adjudicate every `baseline_findings` entry, including entries absent from
`baseline_false_positive_challenges`. Emit exactly one ordered record under
`baseline_adjudications` for each entry. Omission, duplication, reordering,
or an inexact finding copy invalidates the complete response.
- A finding must describe a problem that remains in or is introduced by
the reviewed commit. Do NOT keep a finding merely because the parent
version was wrong; the final report is a review of the patch, not a
Expand Down Expand Up @@ -183,15 +194,16 @@ Output shape (strict):
}
// ... or [] if all findings for this commit are false positives
],
"confirmed_false_positives": [
"baseline_adjudications": [
{
"baseline_id": "fast-N",
"finding": { "problem": "..." },
"verdict": "KEEP|DROP",
"proof": {
"finding_claim": "...",
"verified_facts": ["..."],
"contradiction": "...",
"conclusion": "false_positive"
"assessment": "...",
"conclusion": "supported|false_positive"
}
}
]
Expand All @@ -203,5 +215,5 @@ Output shape (strict):

No prose outside the JSON. No markdown fences. Top-level key MUST be
`commits`. Each commit entry MUST carry `sha`, `findings`, and
`confirmed_false_positives` exactly as named. If you receive zero commits with
findings or challenges, return `{"commits": []}`.
`baseline_adjudications` exactly as named. If you receive zero commits with
baseline findings, regular findings, or challenges, return `{"commits": []}`.
Loading
Loading