Skip to content

docs: require explicit pull request review verdicts - #347

Merged
zengchuang-hw merged 4 commits into
InferMatrixLab:mainfrom
zengchuang-hw:codex/inferrsi-review-skill
Nov 4, 2026
Merged

zengchuang-hw merged 4 commits into
InferMatrixLab:mainfrom
zengchuang-hw:codex/inferrsi-review-skill

Conversation

@zengchuang-hw

@zengchuang-hw zengchuang-hw commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Governing Issue and delivery slice

Refs #346

  • Accepted scope / delegation: Prepare and propose the InferRSI reviewer skill and its collaboration-guide entry.
  • What this PR delivers: A repository-local review-pr skill. Every completed review gives PASS or DO NOT PASS for a specific revision. PASS maps to Approve with LGTM alone on the first non-empty line of the same review. DO NOT PASS maps to Request changes with blockers and expected corrections or evidence.
  • Integration: The guide records the active two-approval ruleset. The integrator checks the review bodies and the current approvals. The skill binds authorized GitHub submissions to the reviewed commit and verifies the recorded review.

Changed artifacts and evidence

  • .agents/skills/review-pr/SKILL.md defines the review and re-review procedure, with findings grounded in the actual diff and applicable evidence.
  • docs/agent-collaboration.md routes reviewers to the skill and explains the formal verdict and integration requirements.
  • The vLLM-Omni review skill informed the revision, diff and evidence checks. The verdict format follows InferRSI's requested convention.

Checks and dependencies

  • Skill frontmatter validation, ruff check src tests tools, git diff --check and 25 PR-reference tests passed.
  • Independent walkthrough covered pass, blocked or missing evidence, stale revision and unavailable posting authorization. The pre-submission review identified a revision race. The revision-bound submission fix was re-reviewed.
  • The local Windows full suite did not pass. One reproduced setup error is pytest exceeding the Windows environment variable length limit for a large parameterized case. Current-head Ready CI passed.
  • Review publication was exercised as a walkthrough. Live approval and body validation are outside this slice.

Remaining work and Issue closure

The PR is ready for independent GitHub review of the current revision. Issue acceptance and closure remain with the authorized owner.

AI assistance: OpenAI Codex authored the patch and prepared the submission.

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>
@zengchuang-hw

Copy link
Copy Markdown
Contributor Author

Dong1017 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

PR review verdict: PASS for the proposed documentation slice. No blocking findings in the two-file diff against #346.

Reviewed base 663f3b327998526a13aa31e72ad729025c18983c → head eef7177380d0fe6b3d901d3f8f4acbbd8f27f20c.

  • The skill covers actual-diff evidence, missing evidence, re-review, independent review, authorization, and revision-bound submission/read-back. Its repository-relative guide links resolve.
  • The integration section matches the live protect-main ruleset on two approvals and stale-approval dismissal. LGMT body inspection remains an explicit integrator responsibility, consistent with body-validation automation being out of scope.

Limits: static inspection only; I did not run the reported local tests or exercise live review submission. At this check, the Ready run still has both full-check test jobs running, so this is not a full-suite pass or a merge-ready determination. Acceptance remains with the Issue's authorized owner.

AI-assisted conversation review; no formal GitHub approval submitted.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scoped documentation-contract review (base 663f3b32 → head eef71773): one actionable ambiguity in the new review skill, called out inline. The proposed conversation verdict and the required GitHub approval-body prefix need distinct, explicit formats; otherwise a reviewer following the first instruction in a formal approval can produce a body that fails the stated integrator check. This is advisory COMMENTED feedback, not a PASS/DO NOT PASS gate decision, approval, or independent test result. I inspected the two-file diff and governing Issue #346; I did not run the reported checks or verify live branch-protection behavior.

Comment thread .agents/skills/review-pr/SKILL.md Outdated

## Give one clear conclusion

Lead the report with **PR review verdict — PASS** or **PR review verdict — DO NOT

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — ambiguous first-line contract for formal approvals. This says to lead the report with PR review verdict — PASS, but line 34 and the new collaboration-guide section require the first non-empty line of the same GitHub approval review body to be exactly LGMT. If a reviewer uses this report format as their GitHub APPROVE body, an otherwise eligible approval will not meet the documented integrator check. Could you distinguish the conversation summary from the submitted GitHub review body and give an explicit example (LGMT first, verdict/evidence below)? It would also help to clarify that non-voting COMMENTED incremental feedback is not a formal PASS/DO NOT PASS approval.

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Dong1017
Dong1017 previously approved these changes Nov 4, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT

PR review verdict — PASS for the two-file documentation change at f27feceb.

  • The first-line ambiguity is cleared: conversation summaries and formal review bodies now have separate instructions, an explicit approval example, and a distinct non-voting COMMENTED path.
  • The complete skill/guide diff preserves independent review, posting authorization, exact-commit submission and read-back. The documented two-account/stale-approval requirements match the active ruleset; body-prefix inspection remains an integrator responsibility.
  • Current Ready and per-head policy checks pass. Existing author @zengchuang-hw can hand this documentation slice to the integrator; this approval supplies one account's review and does not grant merge permission or Issue #346 acceptance.
Evidence and limits

Reviewed head: f27feceb4f8b6147e86dff196f0f747f3f88fc3d.
API base and actual main: 663f3b327998526a13aa31e72ad729025c18983c.

Inspected both complete files, the eef71773 → current-head correction, governing Issue #346, linked collaboration/evidence requirements, prior reviews/comments, and active ruleset 25528020. No outstanding source blocker found in this scope. I did not contribute implementation changes.

All six Ready jobs passed. The Python 3.11 job records Ruff success, 3,544 passed / 13 skipped / 284 subtests. Its tested merge e9c37bc07702c40c44581f5884ca3fef67fa2eaa has zero file differences from the reviewed head. Current-head policy succeeds. The current SkillEval workflow's path filter does not cover these two documentation/agent-skill paths; no SkillEval or efficacy pass is claimed.

Static source/document review and existing hosted evidence only; no local test run, live submission walkthrough, model/device execution or body-validation automation test. The proposed LGMT literal follows the Issue's requested convention. This review does not adopt the policy repository-wide or resolve another reviewer's thread.

Copy link
Copy Markdown
Collaborator

Review at head f27feceb: not approving yet; a few points need clarification/changes before this convention is adopted.

  1. LGMT vs LGTM (docs/agent-collaboration.md:80, :87; .agents/skills/review-pr/SKILL.md:41-45, :71): LGMT reads like a typo of the common LGTM. Is the spelling intentional? If so, please state explicitly that only LGMT counts and that LGTM does not (or that both are accepted), otherwise integrators and reviewers will disagree on which bodies qualify.
  2. Removed anti-self-approval rule (docs/agent-collaboration.md, old line 74): the sentence "the parent cannot substitute its own approval for independent findings" was dropped. "independent, eligible accounts" (L87) does not clearly cover the parent/accountable agent approving its own delegated work. Please restore that sentence or an equivalent explicit rule.
  3. "alone" inconsistency: L80 says LGMT on the first non-empty line "of the same review body", while L86-87 require LGMT alone on that line. Please use one rule in both places (and in the skill, L41).
  4. Mermaid diagram still uses the old model: docs/agent-collaboration.md:64 still has R -->|"Findings and recommendation"| A, and L37 still says "Review recommendation is not merge permission", while the Reviewer row (L24) now gives a verdict. Please update these to the PASS / DO NOT PASS verdict wording.
  5. "eligible" / "authorized" undefined: L87 "eligible accounts", L90 "Authorized reviewers may act under standing delegation", and skill L59 / L68 "eligible independent account" are not defined. Who is eligible (write access only? not the author or the parent?) and where is a standing delegation recorded? Please define or link it.
  6. No flow for clearing Request changes: L81-82 require Request changes for DO NOT PASS and L88 says outstanding blocking reviews must be resolved, but there is no rule on who clears a Request changes review (re-review by the same reviewer with Approve? dismissal by whom, with what record?). Please document it.

I did not verify the ruleset settings linked at L84 (requires repo admin view).

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>

Copy link
Copy Markdown
Contributor Author

Thanks for the six review points. The updated wording is in 55696ce.

  • LGMT is intentional per task: define an explicit verdict for InferRSI PR reviews #346. The guide and skill now require that exact case-sensitive token alone on the first non-empty line of an Approve review.
  • The guide explicitly excludes the author, material contributors, and the accountable parent from independent approval. It defines eligible accounts and where to record the agent's review delegation. The skill links to those rules.
  • The collaboration diagram and prose now use the PASS / DO NOT PASS verdict. The guide also explains how the original reviewer can clear Request changes after re-review and how an authorized, independent maintainer can document a dismissal if that reviewer is unavailable.

The skill format check, 25 PR-reference tests, and independent review passed locally. Current-head Ready CI is running. The revised wording is ready for re-review.

amy-why-3459
amy-why-3459 previously approved these changes Nov 4, 2026

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT

PR review verdict — PASS
Reviewed base 663f3b32 and head 55696ce5 (delta f27feceb..55696ce5), plus both complete files at head.

All six points from my earlier comment are resolved: exact case-sensitive LGMT token (guide L81-83, skill L42-44); author/material contributors/accountable parent excluded from independent approval (guide L86-90); LGMT alone used consistently; diagram L65 and L37-38 use the verdict model; eligibility (write access, independence, counted once) and delegation recording defined (guide L86-93); Request changes clearing via re-review or documented dismissal (guide L101-107). No new blockers found in the delta.

Limits: static doc review only; I did not verify ruleset 25528020 settings or CI results. Agent-submitted under direct delegation from the account owner (@amy-why-3459) for this PR.

@zengchuang-hw

Copy link
Copy Markdown
Contributor Author

@Dong1017 PTAL again

@Dong1017 Dong1017 added the area:process Repo-wide process, governance, privacy and security rules label Nov 4, 2026 — with ChatGPT Codex Connector
Dong1017
Dong1017 previously approved these changes Nov 4, 2026

@Dong1017 Dong1017 left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS for the complete two-file documentation change at 55696ce5.

  • Rechecked all six clarification points: exact LGMT spelling and standalone first line; explicit author/contributor/accountable-parent exclusions; verdict wording in the diagram/prose; review eligibility and recorded delegation; and the re-review/dismissal path for Request changes. No blocking finding remains in this scope.
  • Both complete files and the delta from f27feceb agree with Issue #346. The active ruleset still requires two eligible account approvals and dismisses stale approvals. This replaces my dismissed approval of the earlier revision.
  • Current Ready and per-head policy checks pass. @zengchuang-hw remains the contributor; integration authority and Issue adoption/acceptance remain separate.
Evidence, delegation and limits

Reviewed head: 55696ce50a9ea2edec6cabf0f82eae84d12d57a3.
API base and actual main: 663f3b327998526a13aa31e72ad729025c18983c.

Acting reviewer: Codex in the existing maintenance-review conversation, using account Dong1017. The account owner directly requested this re-review and renewed LGTM; this conversation's standing delegation permits formal APPROVE/REQUEST_CHANGES for independently assessed PRs. Scope: PR #347's documentation/skill change. I did not author, edit, or act as the accountable implementation parent for this delivery.

Read the current diff, both complete files, governing Issue, all five conversation comments, submitted reviews, the inline thread and ruleset 25528020. The earlier format finding and the six later clarification points are cleared in source; no thread was resolved or historical review dismissed by this task.

All six Ready jobs passed. The Python 3.11 log records Ruff success, 3,544 passed / 13 skipped / 284 subtests. Tested merge 51fa801fcc9f1a01fee16abd5d525bf83b335963 has zero file differences from the reviewed head. Current policy succeeds. These paths remain outside the current automatic SkillEval path filter.

Static documentation review and existing hosted evidence only; no local test/model/device run, live automation-body enforcement test, merge or Issue acceptance. The exact LGMT token follows this proposal's requested convention.

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>
@zengchuang-hw
zengchuang-hw dismissed stale reviews from Dong1017 and amy-why-3459 via 03bef4c November 4, 2026 08:10

Copy link
Copy Markdown
Collaborator

Incremental review 55696ce5..03bef4cc (one commit, docs: use LGTM as the approval token). Not approving yet, only because the governing Issue is now out of sync.

The delta itself is fine: LGMT is replaced by LGTM consistently in .agents/skills/review-pr/SKILL.md (L42-44, L47, L74) and docs/agent-collaboration.md (L81-85), and no LGMT remains in either file at this head. Switching to the common spelling resolves my original point 1 more cleanly than the earlier "intentional LGMT" wording.

One thing to fix before approval:

  1. Issue task: define an explicit verdict for InferRSI PR reviews #346 still specifies LGMT. Its "Goal and acceptance" section says an authorized reviewer submits Approve "with LGMT on the first non-empty line", and its current-state summary still describes "intentional case-sensitive LGMT alone". The earlier reply on this PR also said LGMT was intentional per task: define an explicit verdict for InferRSI PR reviews #346. Since the PR now requires exactly LGTM and says only that spelling qualifies, please update task: define an explicit verdict for InferRSI PR reviews #346's acceptance text (and state that the token changed from LGMT to LGTM, with the reason), so integrators checking review bodies against the Issue and the guide don't reach different answers.

Minor (not blocking): reviews already submitted with LGMT on earlier heads of other PRs would no longer match the exact-token rule; a one-line note on whether those need to be resubmitted would avoid confusion during the transition.

CI at 03bef4cc: declarations, pr-references, installed-package (3.10/3.11) and kv-task-qualification passed; full-check (3.10/3.11) was still running when I looked. Static doc review only; I did not verify ruleset 25528020.

Copy link
Copy Markdown
Contributor Author

Thanks for flagging the mismatch in the incremental review. Issue #346 and this PR’s description now match 03bef4cc: the requester chose the conventional LGTM spelling, alone on the first non-empty line of an Approve review. Earlier approvals are recorded against their original revisions. For this PR, the current revision needs fresh approvals using LGTM. Ready CI for 03bef4cc passed.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS for the two-file documentation change at 03bef4cc.

  • Rechecked the complete change and the 55696ce5..03bef4cc delta. The skill, guide, examples and exact-match wording now consistently use LGTM; the six earlier clarification points remain resolved.
  • The requester’s recorded governing decision, updated Issue #346 and the author's synchronization reply now agree. No blocking source finding remains in this reviewed scope.
  • Current Ready and per-head policy checks pass. @zengchuang-hw remains the contributor; this refresh replaces my now-dismissed earlier-head approval. Integration authority, the two-account gate and Issue adoption/closure remain separate.
Evidence, delegation and limits

Head: 03bef4cc3f1072ba5c2b84f3f058691b2a325239.
API base/actual main: 663f3b327998526a13aa31e72ad729025c18983c.

Acting reviewer: Codex maintenance reviewer using Dong1017, under the account owner's direct request for renewed LGTM and standing authority to submit formal independent PR reviews. Scope: this PR's skill/guide changes. I did not author, materially edit, or act as accountable implementation parent for this delivery.

All six Ready jobs pass. The Python 3.11 job reports Ruff success, 3,544 passed / 13 skipped / 284 subtests. Tested merge e4347343a087450aee9dfd5887398fbb579f4b6d has zero file differences from this head. Latest head policy succeeds. The live ruleset was checked for its two-approval/stale-dismissal requirements; these paths remain outside the automatic SkillEval path filter.

Existing hosted evidence and static documentation inspection only. No local test/model/device run, body-enforcement automation test, merge, thread resolution or Issue acceptance.

@zengchuang-hw

Copy link
Copy Markdown
Contributor Author

@amy-why-3459 PTAL again

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed delta 55696ce5..03bef4cc: the skill, guide and examples now consistently use the exact LGTM token, matching the recorded decision on Issue #346. The six earlier clarification points remain resolved; no new blockers.

Limits: static doc review only. Agent-submitted under direct delegation from the account owner (@amy-why-3459).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:process Repo-wide process, governance, privacy and security rules

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants