feat(pr-review): add agent-canvas-automation v2.7+v2.8 with Reviews API and duplicate-review guard by m0nk111 · Pull Request #2 · m0nk111/extensions

m0nk111 · 2026-06-16T22:14:50Z

What

Adds plugins/pr-review/agent-canvas-automation/ to the fork. This is the
source-of-truth copy of the standalone OpenHands Automations tarball
that powers the github-pr-review automation on m0nklabs/cryptotrader
— the tarball uploaded as oh-internal://uploads/6f316eb4-7145-494b-957f-10f80af8b489
and attached to automation 9581078a-7fce-4157-8fb1-04acfd4e718a.

The branch contains two commits that together form the v1 → v2.8 fix:

644f143 — v2.7: introduces the new prompt template (mandates the
GitHub Pull Request Reviews API, requires the ###REVIEW_JSON### JSON
contract), the Reviews-API result-poster, path validation, the
REQUEST_CHANGES → COMMENT downgrade for PR authors, the
re-review guard, the MCP-direct detection, and the brace-counting
JSON parser with "last-marker-wins" rule.
9252f85 — v2.8: adds a duplicate-review guard in both the
"no JSON" and "have JSON" paths, so that when the agent posts a
review via the GitHub MCP AND the script also posts from the parsed
JSON, only one review lands on the PR.

Why

The agent-canvas automation does not use the upstream
plugins/pr-review action directly — it has its own tarball that was
uploaded before the upstream plugin reached a usable state, and the
tarball was never updated. The user reported on 2026-06-16 that the
bot was still posting single-blob reviews instead of inline review
threads. This was traced to two bugs in the v1 tarball:

The prompt template explicitly forbade the agent from using the
Reviews API ("Do not modify files, push commits, approve via the GitHub
API, or request changes via the review API; only produce the final
comment text").
The result-poster called POST /repos/.../issues/{n}/comments (the
issue-comments API) instead of POST /repos/.../pulls/{n}/reviews.

The v1 → v2.8 fix replaces the prompt template and the result-poster.

The v2.7 → v2.8 step adds the duplicate-review guard, which the user
triggered by reporting two reviews on PR 400. Both reviews were in the
new format (priority labels, inline threads, suggestion blocks) — the
"old" one was actually the agent's MCP-direct post at 21:06:09, and the
"new" one was the script's API post at 21:06:21 from the parsed
###REVIEW_JSON###. The v2.7 script's MCP-detection only ran in the
"no JSON" path, so it didn't catch this race. v2.8 fixes it.

What's in this PR

File	Purpose
`plugins/pr-review/agent-canvas-automation/README.md`	Top-level index: what this is, why it lives on the fork (not upstream), and how to apply the v2.8 tarball via the Automations API.
`plugins/pr-review/agent-canvas-automation/v2/README.md`	The v1 → v2.8 changelog. Explains the bugs, the fix, and includes a real-world demonstration table comparing PR 380/381 (old format) with PR 400 (v2.7 race) and PR 402 (v2.8 fix).
`plugins/pr-review/agent-canvas-automation/v2.7/main.py`	The v2.7 source. The intermediate state between the v1 prompt and the v2.8 race-fix.
`plugins/pr-review/agent-canvas-automation/v2.8/main.py`	The current source-of-truth. The deployed tarball is `tar -czf` of this file.

Real-world demonstration

PR	Outcome	Reference
PR 379 (old, before v1 was uploaded)	Used the new format because v1 wasn't yet the prompt	m0nklabs/cryptotrader#379
PR 380, 381 (with v1 tarball)	Old format (single issue-comment blob)	https://github.com/m0nklabs/cryptotrader/issues/4718914883 (the bad blob)
PR 380, 381, 387 (with v2.5/2.6)	New format with inline review threads	m0nklabs/cryptotrader#381 (review)
PR 400 (with v2.7)	New format — but 2 reviews with identical content landed on the PR (agent's MCP-direct post + script's post from the parsed JSON). The v2.7 script's MCP-detection only ran in the "no JSON" path.	m0nklabs/cryptotrader#400
PR 402 (clean e2e test, with v2.8)	New format, exactly 1 review. The v2.8 duplicate-review guard runs in BOTH the "no JSON" and "have JSON" paths.	m0nklabs/cryptotrader#402

What's NOT in this PR (deliberately)

The agent-canvas UI "double replies" symptom the user reported is a
separate problem in either the agent-server's WebSocket event
emission or the agent-canvas frontend's rendering (no deduplicate
in the bundled JS). It is not in the pr-review script and is not in
scope here. The v2.8 script does everything right; if you still see
the agent-canvas UI showing the same reply twice, that bug is in the
agent-server or agent-canvas frontend, not in the script.
The CLI process (PID 1986310, in pts/2 since 11:25) is an idle
background process from earlier in the day; it has no active TCP
connection to the agent-server (verified via ss -tnp). It is not
the source of any "double" behavior.

How to apply this locally

# 1. Pack the v2.8 main.py into a gzipped tarball
cd plugins/pr-review/agent-canvas-automation/v2.8
tar -czf /tmp/pr-reviewer-v2.8.tar.gz main.py

# 2. Upload via the Automations API (port 18001 locally)
curl -s -X POST \
  -H "X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY" \
  -H "Content-Type: application/gzip" \
  --data-binary @/tmp/pr-reviewer-v2.8.tar.gz \
  "http://localhost:18001/api/automation/v1/uploads?name=pr-reviewer-v2.8-reviews-api&description=..."

# 3. PATCH the automation to point at the new tarball
curl -s -X PATCH \
  -H "X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tarball_path": "oh-internal://uploads/<id-from-step-2>"}' \
  "http://localhost:18001/api/automation/v1/9581078a7fce41578fb104acfd4e718a"

Why this isn't upstreamed

OpenHands/extensions is the public extensions registry. The
agent-canvas-automation is specific to one user's agent-canvas
deployment (a single OpenHands Automations instance on one host, with
one cron job and one automation ID). It is not general-purpose plugin
code — it's the wiring that talks to that specific deployment. When the
upstream plugins/pr-review plugin gets a deployment story that doesn't
need a custom tarball, this whole directory can be deleted.

Versions

Version	What changed	When
v1	(Pre-PR) Single-blob issue-comment reviews. Prompt forbade the Reviews API.	2026-06-08
v2.0–v2.6	New prompt + Reviews-API poster. Path validation, REQUEST_CHANGES→COMMENT for PR author, MCP-direct detection in "no JSON" path, re-review guard.	2026-06-16
v2.7	Last-marker parser fix (handles agent prose that mentions `###REVIEW_JSON###`).	2026-06-16
v2.8	Duplicate-review guard in BOTH the "no JSON" and "have JSON" paths — the v2.7 race on PR 400.	2026-06-16

Upstream fix: OpenHands/extensions#339 — fixed the same problem in the upstream plugin.
The local-install .pth shim that points the agent-server at this
fork as the public-skills source:
/home/flip/.cache/uv/archive-v0/Gd11WEMCgEp_rZFU/lib/python3.14/site-packages/openhands_fork_extensions.{py,pth}
(and the CLI twin at
/home/flip/.local/share/uv/tools/openhands/lib/python3.12/site-packages/openhands_fork_extensions.{py,pth}).
The CLI setting: agent_context.load_public_skills: true in
/home/flip/.openhands/settings.json.

Co-authored-by

Generated by an AI agent (OpenHands) on behalf of @m0nk111.

Tracks the standalone OpenHands Automations tarball that powers the github-pr-review automation on m0nklabs/cryptotrader. The v1 tarball (oh-internal://uploads/80f203b3-0678-404a-9b60-401782a0e4b9) told the agent NOT to use the Reviews API and posted results via the issue-comments API, which produced single-blob reviews instead of inline review threads. v2.7 (currently deployed) replaces the prompt template and the result-poster. Concretely: - Prompt mandates the GitHub Pull Request Reviews API and requires the agent to emit a ###REVIEW_JSON### block with the comment list. - Result-poster calls POST /repos/.../pulls/{n}/reviews with comments[]. - Path validation drops comments whose path is not in the PR diff. - REQUEST_CHANGES is downgraded to COMMENT when the bot is the PR author. - Re-review guard: closed (PR, label_event_id) pairs are skipped. - MCP-direct detection via _list_existing_reviews, no double-post. - Brace-counting JSON parser with last-marker-wins rule. Includes the source-of-truth v2.7 main.py and a README that documents the v1 -> v2.7 transition, the bug, the fix, and a real-world demonstration table comparing PR 380/381 (old format) with PR 400 (new format, clean e2e test). Refs OpenHands#339 Co-authored-by: openhands <openhands@all-hands.dev>

…ew guard PR 400 demonstrated a race: the agent posted a review via the GitHub MCP (21:06:09), then the v2.7 script also posted from the parsed ###REVIEW_JSON### block (21:06:21), producing two reviews with identical content. The v2.7 script's MCP-direct detection only ran in the 'no JSON' path, not the 'have JSON' path. v2.8 fixes this: the script now calls _list_existing_reviews and _existing_bot_review_for_commit BEFORE deciding whether to post, regardless of whether the agent emitted a parseable JSON contract or used the GitHub MCP directly. If a m0nk111-post review already exists at the same commit_id, the script closes the state without re-posting. Verified on PR 402 (m0nklabs/cryptotrader): exactly 1 review lands, vs 2 on PR 400 with v2.7. Refs the PR 400 duplicate that the user reported. Co-authored-by: openhands <openhands@all-hands.dev>

…te guard + deployment-neutral config Consolidates the work that was previously proposed as PRs #1, #2 and #3 into a single reviewable change. ## What this does 1. Steers the github-pr-review skill and the agent prompt at the GitHub Pull Request Reviews API (POST /repos/{owner}/{repo}/pulls/{n}/reviews with a comments[] array) instead of the issue-comments API. Inline threads, suggestion blocks, and one-click-apply now actually work. 2. Adds the agent-canvas-automation cron runner that watches one repo for a trigger label, forks a fresh OpenHands conversation per (PR, label_event_id), waits for the conversation to finish, parses a ###REVIEW_JSON### contract out of the final response, and posts the result as a single Pull Request Review. v2.7 and v2.8 are shipped together; the difference between them is the duplicate-review guard (see below). 3. Makes the duplicate-review guard robust: before posting a parsed payload, the script queries GitHub for an existing review by the bot user at the same commit_id. If one is found, the script closes the state without re-posting — that fixes the race where the agent posts via the GitHub MCP and the script also posts from the parsed JSON, producing two reviews with identical content. 4. Removes every hardcoded reference to the original test deployment from this repo. The cron runner is generic now: REPO is an empty string with a TODO pointing at /pr-reviewer:setup, the bot-user check is resolved at runtime via GET /user with the same token that posts the review, and the SKILL.md / README references are deployment-neutral. ## Files M plugins/pr-review/scripts/prompt.py M skills/github-pr-review/SKILL.md M skills/index.js A plugins/pr-review/agent-canvas-automation/README.md A plugins/pr-review/agent-canvas-automation/v2/README.md A plugins/pr-review/agent-canvas-automation/v2.7/main.py A plugins/pr-review/agent-canvas-automation/v2.8/main.py ## Behavioural impact - New env vars: none. - One extra GET /user per cron tick (cached thereafter). That's well within the 5000/hr PAT budget. - Until REPO is filled in by /pr-reviewer:setup, _verify_token_and_repo will fail with a 404 against an empty path — fail-fast by design. ## Verification - python3 -c 'import ast; ast.parse(...)' on v2.7 + v2.8 main.py -> OK - grep -rnE 'm0nklabs|cryptotrader|m0nk111-post|pull/379|PR 379|PR OpenHands#379' on the whole tree -> zero hits (the previous test-deployment literal that used to live in REPO = ... is gone; REPO is now empty). - node scripts/build-skills-catalog.mjs regenerated skills/index.js cleanly. Co-authored-by: openhands <openhands@all-hands.dev>

openhands-agent added 2 commits June 16, 2026 21:11

m0nk111 merged commit 85f6fa6 into main Jun 16, 2026

m0nk111 deleted the agent-canvas-pr-reviewer-v2.7 branch June 16, 2026 22:16

m0nk111-post mentioned this pull request Jun 17, 2026

chore: roll up PRs #1 + #2 + strip deployment refs into one reviewable branch #4

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pr-review): add agent-canvas-automation v2.7+v2.8 with Reviews API and duplicate-review guard - #2

feat(pr-review): add agent-canvas-automation v2.7+v2.8 with Reviews API and duplicate-review guard#2
m0nk111 merged 2 commits into
mainfrom
agent-canvas-pr-reviewer-v2.7

m0nk111 commented Jun 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

m0nk111 commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

What's in this PR

Real-world demonstration

What's NOT in this PR (deliberately)

How to apply this locally

Why this isn't upstreamed

Versions

Related

Co-authored-by

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

m0nk111 commented Jun 16, 2026 •

edited

Loading