Skip to content

feat(vibe): Encode review lessons from the first two vibe feature PRs - #305

Merged
shafty023 merged 2 commits into
mainfrom
feat/vibe-review-lessons
Oct 16, 2026
Merged

shafty023 merged 2 commits into
mainfrom
feat/vibe-review-lessons

Conversation

@shafty023

Copy link
Copy Markdown
Collaborator

Purpose

Operator ruling, 2026-10-16: encode the mistakes found reviewing the first two vibe feature pull requests in symphony-alpha (#8885 grouped list sections, #8886 bulk add tag), from both builders' own reports and both independent reviews. Every lesson lands in the agent or reference section that owns the behavior. Where a rule already existed and was not followed, the owner now runs a concrete check and quotes its evidence instead of restating the rule.

What changed

  • Guardrails reviewer (agents/vibe-guardrails-reviewer.md). Check 1 greps packages/app/shared (shared/auth included) and existing use-* hooks before a new hook or inline query. Check 4 blocks retyped wording a shared helper renders, and any docs change. Check 7 blocks casts (stories included) and host key literals. Check 8 adds slot-without-owner (the ISS-12137 selection red flag), private copies, and a leftovers sweep over every deleted, renamed or extracted file. Check 9 blocks unasked gates and widened shared test helpers, tests named after deleted modules, and production-code oracles. Check 10 reads compile-only repairs word by word, allows one Version History entry at the next version with no duplicate-key special case, blocks whole-page Composites, and blocks lost story IDs and stale references to them. New check 13: capability removal on every main-sync merge. New check 14: flag retirement reads the linked tickets' comments, and a recorded hold is a blocking engineering decision.
  • Adversarial reviewer: criteria per surface, every selected row accounted for, host-capability gates with web and Desktop parity, per-row enum parsing.
  • Storybook decomposer: PRD states on the consumed component, paging-helper fixtures, honest story names; handoff mode reports lost IDs with rg hits and Version History entries.
  • Change worker: the matching build rules, main-sync conflicts never drop main's capability, and pull-request tests in their owning lanes with literal oracles, criterion lineage and real host adapters (same-route link and Back).
  • Verify worker / handoff: Storybook preview sort-order tests, touched e2e specs through dagger call e-2-e, and .github/pr-test-event-tiers.md before any CI claim. Handoff step 7 runs check 13 on its own main sync.
  • quality-loop.md: each acceptance criterion has a Surfaces: line and is met only per surface.
  • No agent-written public docs. The 2026-10-13 docs-draft exception is retired; handoff-inventory.mjs reports any docs change as outside the allowed paths in every phase, and the ticket's Feature docs line only lists pages for a human to update.
  • PR body files are named per ticket and read back before gh pr create/edit (environment worker, seed PR worker, cl-execute).
  • Merged main (feat(vibe): Offer visual QA, check precedent and plugin version #304): unspecified selection handling points at "Precedent before asking"; an outdated plugin's version question now comes first, alone, and the endings line follows in the first reply after the answer.
  • Release: vibe 1.0.54 (both manifests), CHANGELOG entry.

Commands run

  • uv run pytest plugins/: 2367 passed, 3 skipped. uv run ruff check . and uv run pyright: clean.
  • node --test over every plugin test file: 412 tests; one cl-sweep black-box test hit an EPIPE while evals ran in parallel and passes alone on this branch and on main.
  • tools/plugin-manifests: typecheck and 6 tests pass.
  • test_vibe_review_lessons_contract.py: 18 cases, 17 fail on 1.0.53. The inventory docs test fails on 1.0.53.
  • Evals, one run each, --tag review-lessons --tag version-check: Claude Sonnet 12 of 12; Codex runner 9 of 9 on the new and version cases. On 1.0.53 the docs case fails on both harnesses; the other six review cases also pass there because the prompt hands the reviewer the evidence the new checks collect.

- Guardrails reviewer gains concrete checks with evidence: main-sync
  capability removal on every merge, flag retirement against a recorded
  hold, leftovers after an extraction, slots without a shared owner,
  private copies, host key literals, casts, Version History, prose in
  compile-only repairs, lost and stale story IDs, and checks nobody asked
  for.
- Adversarial reviewer checks acceptance criteria per surface, selection
  accounting, host-capability gates and their web and Desktop parity, and
  per-row enum parsing.
- Storybook decomposer, change and verify workers carry the matching
  build rules: one Version History entry, story states on the consumed
  component, paging-helper fixtures, test lanes, oracles and lineage,
  the sort-order test, containerized e2e and the CI tier doc.
- Nothing agent-written goes into public docs: the docs-draft exception
  is retired and the inventory reports any docs change in every phase.
- PR body files are named per ticket and read back before gh.
- Seven review-lessons eval cases for Claude and Codex, and a contract
  test file for the rules.

Testing: vibe pytest (291 pass), inventory node tests (21 pass), Claude
evals 7 of 7
Risks: Longer handoff reviews
- Keep both eval paragraphs in the README and count both sets of cases
  in the Codex runner test and README.
- Point unspecified selection handling at "Precedent before asking".
- An outdated plugin's version question comes first, alone; the new
  session's endings line goes in the first reply after the answer.
- Release vibe 1.0.54.

Testing: vibe pytest (303 pass), full pytest (2367 pass), ruff, pyright,
node suites, manifest tests; Claude and Codex evals on the new cases
Risks: None identified
@shafty023
shafty023 merged commit ce6df4b into main Oct 16, 2026
10 checks passed
@shafty023
shafty023 deleted the feat/vibe-review-lessons branch October 16, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant