Skip to content

chore: standardize PR agent and independent review labeling - #1918

Merged
Fei2411 merged 2 commits into
devfrom
chore/pr-agent-review-labels
Nov 21, 2026
Merged

Fei2411 merged 2 commits into
devfrom
chore/pr-agent-review-labels

Conversation

@Fei2411

@Fei2411 Fei2411 commented Nov 21, 2026 •

Copy link
Copy Markdown
Owner

Issue

No issue (repository workflow and contributor guidance).

Agent

Agent: Antigravity (Gemini 4 Neon)

What changed / 改了什么

  • Add .github/workflows/pr-triage-labels.yml (zero-checkout actions/github-script) to automatically sync agent/* labels from the PR Agent: line, sync mutually-exclusive review/* and reviewed-by/* labels from trusted collaborator review comments (Review: / Reviewer:), and invalidate stale review/approved / review/changes-requested labels to review/re-review when new commits are pushed.
  • Update .github/pull_request_template.md with an explicit ## Agent section.
  • Update AGENTS.md to require Agent: lines, agent/* labels, release milestones when decided, and structured independent review comments + review/* / reviewed-by/* labels.
  • Document the new agent/*, review/*, and reviewed-by/* labels in CONTRIBUTING.md and CONTRIBUTING.zh.md.

Changelog

none

How I checked it / 怎么验证的

  • npm test (in ds-bot/): not run (only workflow and markdown documentation files changed)
  • node scripts/build-client.mjs --check: not run (src/client/ untouched)
  • npm run lint: not run (ds-bot/ untouched)
  • Validated .github/workflows/pr-triage-labels.yml with Ruby YAML parser and 28 Node unit/event test cases covering CRLF, code fences, bold markdown, primary-name-first agent matching, Chinese punctuation, fork PR handling, closed PR gating, and synchronize invalidation.
  • git archive HEAD + node scripts/scan.mjs: 0 must-fix findings.

Checklist / 检查

  • One issue, one subject. / 只做一件事。
  • src/client/ changed → npm run build ran and client.js is committed. (not changed)
  • No keys, tokens or personal data, also in screenshots.
  • No AI tool as author or Co-authored-by.

Devin Review

Add the PR triage label sync workflow, require agent and review labels in AGENTS.md and the pull request template, and document the new labels in CONTRIBUTING.
@Fei2411 Fei2411 added this to the v0.3.9 milestone Nov 21, 2026
@Fei2411 Fei2411 added the agent/antigravity Opened by Antigravity label Nov 21, 2026
@Fei2411

Fei2411 commented Nov 21, 2026

Copy link
Copy Markdown
Owner Author

Review: approved
Reviewer: Antigravity (Gemini 4 Neon) @ 8b99a86

Independent review completed across two rounds (all 4 blocking findings and 6 non-blocking improvements addressed and re-verified):

  • Zero-checkout pull_request_target script with strict trusted-association + non-Bot checks for review/* / reviewed-by/*.
  • Only operates on open PRs; edited gated to changes.body; issue_comment / pull_request_review limited to created / submitted so historical edits cannot resurrect stale approvals.
  • Fork PR pull_request_review with review markers warns gracefully instead of failing with HTTP 403.
  • Line-based fence stripping, CRLF normalization, primary-name-first agent matching, and 28 parser/event contract tests passed locally; leak scan 0 must-fix.

@Fei2411 Fei2411 added review/approved Independent review passed — ready to merge reviewed-by/antigravity Reviewed by Antigravity labels Nov 21, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T02:56:32.256820Z 8b99a86 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b99a8663a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +305 to +307
const current = await getCurrentLabels(owner, repo, issueNumber);
const toRemove = REVIEW_STATUS_LABELS.filter((l) => l !== parsed.status);
const toAdd = [parsed.status];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Verify the reviewed commit before restoring review labels

When a trusted reviewer submits a pending review of an older commit after a new push, this handler removes review/re-review and restores review/approved even though the current head has not been reviewed. The same problem occurs with a regular comment whose Reviewer: line names an obsolete SHA: neither that SHA nor review.commit_id is checked. I reproduced this with the workflow script and mocked API calls: an old-commit approval replaced review/re-review with review/approved and reviewed-by/codex. Compare the reviewed SHA against the live PR head before applying the verdict so stale reviews cannot undo invalidation.

AGENTS.md reference: AGENTS.md:L65-L65

Useful? React with 👍 / 👎.

devin-ai-integration[bot]

This comment was marked as resolved.

@Fei2411

Fei2411 commented Nov 21, 2026

Copy link
Copy Markdown
Owner Author

Review: changes-requested
Reviewer: Antigravity (Claude Sonnet 5.5) @ 8b99a86

First-principles review found 3 real logic bugs in .github/workflows/pr-triage-labels.yml that should be fixed before merging:

  1. Stale review/comment overrides review/re-review (L289–338): Neither pull_request_review (review.commit_id) nor issue_comment (Reviewer: ... @ <sha>) is checked against the live pull_request.head.sha. Submitting a pending review or comment referencing an older commit after new commits are pushed will remove review/re-review and falsely restore review/approved.
  2. Model name fallback misattributes agent/* (L127–136 mapAgentSlug): Falling back to matchKnownAgent(cleaned) when matchKnownAgent(primary) returns null causes strings like Agent: Cursor (Claude Code) to be misclassified as agent/claude-code instead of agent/other. Only primary (before parentheses/@) should match known agent slugs.
  3. Questions treated as approvals (L148 STATUS_BOUNDARY): Including ? / ? in STATUS_BOUNDARY causes lines such as Review: approved? to apply review/approved. Remove ? and ? from STATUS_BOUNDARY.

@Fei2411 Fei2411 added review/changes-requested Independent review found issues to fix and removed review/approved Independent review passed — ready to merge labels Nov 21, 2026
Fetch the live PR head before applying review verdicts so stale reviews or comments referencing an older commit cannot overwrite review/re-review. Match known agent slugs strictly against the primary agent name before parenthesized model text, reject question marks in Review: lines, and require a Reviewer: header for non-re-review verdicts.
@Fei2411 Fei2411 added review/re-review New commits pushed after last review — needs re-review and removed review/changes-requested Independent review found issues to fix reviewed-by/antigravity Reviewed by Antigravity labels Nov 21, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

let statusCandidate = null;
for (const m of prose.matchAll(statusRe)) {
const candidate = unwrapInline(m[1]);
if (!candidate || candidate.includes('|') || /[??]/.test(candidate)) continue;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Decision requests with questions ignored

parseReviewComment drops a needs-decision verdict when the reviewer includes the decision question on the same line. Even with the correct commit SHA, the PR keeps its previous review status.

Learn more

A review status line can start with a recognized verdict and then give a reason. The question-mark guard runs before matching that verdict, so it also rejects an explicit request for a decision followed by the question the maintainer needs to answer. The event handler returns without updating labels when the parser rejects the comment.

Example: A collaborator posts Review: needs-decision — Keep the old API? followed by a valid Reviewer: line for the current head. An existing review/approved label stays in place instead of changing to review/needs-decision.

Recommended fix: Distinguish a question attached to the status itself from a question in the explanation. Keep rejecting tentative verdicts such as Review: approved?, but allow a question after a clearly separated needs-decision verdict.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

function extractReviewedSha(raw) {
const cleaned = unwrapInline(raw);
let last = null;
for (const m of cleaned.matchAll(/@[ \t]*[`*_]*([0-9a-f]{7,40})[`*_]*(?=$|[\s.,;:)/)\]—–-])/gi)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Chinese punctuation hides review commits

extractReviewedSha misses a valid commit SHA followed by 。 or ,. For regular PR comments, the commit check then drops the verdict as if no SHA were supplied.

Suggested change
for (const m of cleaned.matchAll(/@[ \t]*[`*_]*([0-9a-f]{7,40})[`*_]*(?=$|[\s.,;:)/)\]—–-])/gi)) {
for (const m of cleaned.matchAll(/@[ \t]*[`*_]*([0-9a-f]{7,40})[`*_]*(?=$|[\s.,;:。,;:)/)\]—–-])/gi)) {

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@Fei2411

Fei2411 commented Nov 21, 2026

Copy link
Copy Markdown
Owner Author

Review: approved
Reviewer: Antigravity (Claude Sonnet 5.5) @ 8718530

Independent re-review passed on 87185307:

  • Stale review / comment protection: issue_comment and pull_request_review now fetch live PR state and head.sha via pulls.get, reject non-open PRs, require review.commit_id (for native reviews) and @ <sha> (for review comments, or both when present) to match the live head SHA, while keeping standalone Review: re-review / 待复审 conservative.
  • Agent attribution: mapAgentSlug now matches known agent slugs strictly against primary (before parentheses/@) with word boundaries, so Cursor (GPT-5.3-Codex) / Cursor (Claude Code) map cleanly to agent/other and reviewed-by/other, while Human @ <sha> applies review/* without a spurious reviewed-by/* label.
  • Interrogative / casual prose guard: removed ?/? from STATUS_BOUNDARY, rejected lines containing ?/?, and required a Reviewer: line for non-re-review verdicts (and exact standalone tokens for re-review without Reviewer:).
  • Verified YAML syntax, JS syntax, 22 workflow event tests, 50k-char regex backtracking stress test, zero-checkout pull_request_target safety, and scripts/scan.mjs (0 must-fix).

@Fei2411 Fei2411 added review/approved Independent review passed — ready to merge reviewed-by/antigravity Reviewed by Antigravity and removed review/re-review New commits pushed after last review — needs re-review labels Nov 21, 2026
@Fei2411
Fei2411 merged commit f472b8a into dev Nov 21, 2026
20 checks passed
@Fei2411
Fei2411 deleted the chore/pr-agent-review-labels branch November 21, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent/antigravity Opened by Antigravity review/approved Independent review passed — ready to merge reviewed-by/antigravity Reviewed by Antigravity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant