diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fb59fa0..60696d82 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ All notable changes to the claude-plugins project will be documented in this fil The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Entries are listed newest-first; each plugin section is treated as released when merged to `main`. +### vibe v1.0.46 + +#### Fixed +- Codex now gets the writer's rules. Codex reads a skill through shell output, which keeps the start and end of a long file and drops the middle; the vibe SKILL.md is about 14,400 tokens, so one read lost the build loop, and a Codex context doing the writer's work never opened `vibe-change-worker.md` or `quality-loop.md`. On main `642d235` that is why all four visual-cues Codex evals failed: a render with no reference image, criteria that paraphrased the person's words with no source or result, and no `Reference read` lines or token-mapped criteria for a reference. A new harness note near the top of SKILL.md tells Codex to read long files to their last line in parts of at most 200 lines, and in both harnesses whatever context does the writer's work reads the writer's definition and the "Acceptance criteria before ready" and "Visual references" sections first. The writer's own definition tells a Codex writer the same. +- `quality-loop.md`, "Visual references", now opens with "Recording it in the table": the exact line shapes for a `Reference read` (one per reference image on the session, a screenshot of our page as it is now included, on a correction too), a criterion per reference property naming its token or component (a shadow, a type size and a type weight are three criteria), the reference check after the last edit, and, with no reference, a criterion quoting the person's words and never a render. Criterion lines sit above `### Checks`. +- Eval fixtures: the stand-in screenshot tool read a CSS file's `@import` line as part of the first selector and silently dropped that rule, so a card that imported the tokens rendered with no shadow; it now strips `@import` lines and comments. The stand-in catalog names the selected-state contract the renderer and graders already used (`aria-selected="true"` or `is-selected`). The hover and selected graders accept a grouped selector (`.meeting-card:hover, .meeting-card:focus-visible { ... }`), which applies the same token on hover, and the outside-product criteria grader fails only when one of its four required criteria lacks a token, not when another criterion records something the design system has no token for. +- Coverage: a new contract test file (`test_vibe_codex_reading_contract.py`, 8 cases, all failing on 1.0.44). Codex evals for the four visual-cues cases, one run each: 0 of 4 on 1.0.44, 4 of 4 here. Claude (`claude plugin eval`, Opus, one run each): 1 of 4 on 1.0.44, 3 of 4 here; `pm-two-screenshot-page` still fails its criteria grader on Claude because in that case Claude opened no plugin file in any run, on 1.0.44 or here. + ### vibe v1.0.45 #### Fixed diff --git a/plugins/vibe/.claude-plugin/plugin.json b/plugins/vibe/.claude-plugin/plugin.json index 26718d7a..672fbd8e 100644 --- a/plugins/vibe/.claude-plugin/plugin.json +++ b/plugins/vibe/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "vibe", "description": "Vibe sessions, handoff, seed refresh, and ClosedLoop ticket workflows", - "version": "1.0.45", + "version": "1.0.46", "dependencies": ["closedloop-core"], "author": {"name": "ClosedLoop", "email": "support@closedloop.ai"} } diff --git a/plugins/vibe/.codex-plugin/plugin.json b/plugins/vibe/.codex-plugin/plugin.json index 4f7ed42f..9133dda1 100644 --- a/plugins/vibe/.codex-plugin/plugin.json +++ b/plugins/vibe/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "vibe", - "version": "1.0.45", + "version": "1.0.46", "description": "Vibe sessions, handoff, seed refresh, and ClosedLoop ticket workflows", "author": {"name": "ClosedLoop", "email": "support@closedloop.ai"}, "homepage": "https://github.com/closedloop-ai/claude-plugins", diff --git a/plugins/vibe/agents/vibe-change-worker.md b/plugins/vibe/agents/vibe-change-worker.md index 69075103..814e69b1 100644 --- a/plugins/vibe/agents/vibe-change-worker.md +++ b/plugins/vibe/agents/vibe-change-worker.md @@ -39,6 +39,10 @@ upfront summary, technical question or plan to approve. Read `../skills/vibe/references/quality-loop.md`, `closedloop-graph.md`, `design-pass.md`, `guardrails.md`, `annotations.md` when relevant, and `ticket-template.md`, resolved from this agent's owning plugin as briefed. +In Codex, read each, and this file, to its last line in parts of at most 200 +lines (`sed -n '1,200p'`, then `'201,400p'`, and so on): one read of a long +file loses its middle. In `quality-loop.md` that includes "Acceptance criteria +before ready" and "Visual references", past its first 400 lines. Read the root and nearest owning AGENTS.md before editing. Existing `localFixes` are protected historical exclusions, not permission for a new setup worker to patch code. Preserve other people's existing work. @@ -244,8 +248,10 @@ image never stands in for that read. A correction ("it still doesn't look like it") reads the reference again the same way. Add each visual and UX property the reference shows (shadow and elevation, type size and weight, color, spacing, radius, borders, alignment, icons and visible states) to the -`## Acceptance Criteria` section, with the design token or catalog component -that delivers it, and check it like any other criterion. On a surface that +`## Acceptance Criteria` section, one property per criterion, with the design +token or catalog component that delivers it, and check it like any other +criterion. Record each line in the shapes `quality-loop.md`, "Recording it in +the table", gives. On a surface that does not bundle the design-system tokens, use the named token's exact value, cite the token, and log `Engineering needed (tokens not bundled): ; uses values`; diff --git a/plugins/vibe/evals/pm-outside-product-cards/graders/criteria-mapped.md b/plugins/vibe/evals/pm-outside-product-cards/graders/criteria-mapped.md index bcb52cb5..a6d429a5 100644 --- a/plugins/vibe/evals/pm-outside-product-cards/graders/criteria-mapped.md +++ b/plugins/vibe/evals/pm-outside-product-cards/graders/criteria-mapped.md @@ -6,4 +6,4 @@ PASS only if all of these hold in the session decision table: 1. The Acceptance Criteria section has criteria, sourced to the reference image, for: the card's resting shadow, the bold larger title, a visible hover state, and a visible selected state. 2. Each of those criteria names the design token or catalog component that delivers it (for example --shadow-card, --shadow-raised, --text-lg or --text-xl, --weight-bold, --border-selected, Card, CheckBadge). 3. The Checks log records that the reference image was read, naming its path. -FAIL if any criterion is missing or any lacks a token or component. +FAIL if any of those four criteria is missing or lacks a token or component. Another criterion for something the reference shows that has no design-system equivalent, recorded as such, does not fail this. diff --git a/plugins/vibe/evals/pm-outside-product-cards/graders/hover-state.md b/plugins/vibe/evals/pm-outside-product-cards/graders/hover-state.md index fc3f5dce..301dedc5 100644 --- a/plugins/vibe/evals/pm-outside-product-cards/graders/hover-state.md +++ b/plugins/vibe/evals/pm-outside-product-cards/graders/hover-state.md @@ -1,6 +1,6 @@ --- type: regex target: { source: file, path: session/src/meeting-card.css } -pattern: "\\.meeting-card:hover\\s*\\{[^}]*var\\(--shadow-raised\\)" +pattern: "\\.meeting-card:hover\\s*(,[^{}]*)?\\{[^}]*var\\(--shadow-raised\\)" --- Hovering raises the card with the raised elevation token. diff --git a/plugins/vibe/evals/pm-outside-product-cards/graders/selected-state.md b/plugins/vibe/evals/pm-outside-product-cards/graders/selected-state.md index d7b8636c..310319ff 100644 --- a/plugins/vibe/evals/pm-outside-product-cards/graders/selected-state.md +++ b/plugins/vibe/evals/pm-outside-product-cards/graders/selected-state.md @@ -1,6 +1,6 @@ --- type: regex target: { source: file, path: session/src/meeting-card.css } -pattern: "\\.meeting-card(\\[aria-selected=\"true\"\\]|\\.is-selected)\\s*\\{[^}]*var\\(--border-selected\\)" +pattern: "\\.meeting-card(\\[aria-selected=\"true\"\\]|\\.is-selected)\\s*(,[^{}]*)?\\{[^}]*var\\(--border-selected\\)" --- The selected card uses the selected border token. diff --git a/plugins/vibe/evals/shared/stand-in-repo.sh b/plugins/vibe/evals/shared/stand-in-repo.sh index 5beba902..75803e3d 100755 --- a/plugins/vibe/evals/shared/stand-in-repo.sh +++ b/plugins/vibe/evals/shared/stand-in-repo.sh @@ -55,7 +55,8 @@ CSS - `Card`: a surface. Elevation `--shadow-none`, `--shadow-card` (resting) or `--shadow-raised` (hover or dragged). Radius `--radius-lg`. Selected state - uses `--border-selected` and the `CheckBadge` component. + (`aria-selected="true"` or the `is-selected` class) uses `--border-selected` + and the `CheckBadge` component. - `CheckBadge`: the small accent check shown on a selected item. - `Heading`: sizes `--text-md`, `--text-lg`, `--text-xl`; weights `--weight-regular`, `--weight-medium`, `--weight-bold`. @@ -135,7 +136,7 @@ def font(size: int, weight: str | None) -> ImageFont.FreeTypeFont: def render(html_path: Path, out: Path, width: int, state: str | None) -> None: root = html_path.resolve().parents[1] table = tokens(root) - css = html_path.with_suffix(".css").read_text() + css = re.sub(r"/\*.*?\*/|@import[^;]*;", "", html_path.with_suffix(".css").read_text(), flags=re.S) rules = styles(css, table, state) elements = ELEMENT.findall(html_path.read_text()) box = rules.get(elements[0][1].split()[0], {}) diff --git a/plugins/vibe/skills/vibe/SKILL.md b/plugins/vibe/skills/vibe/SKILL.md index 2175c0f8..7627881b 100644 --- a/plugins/vibe/skills/vibe/SKILL.md +++ b/plugins/vibe/skills/vibe/SKILL.md @@ -65,6 +65,19 @@ This skill only works in a `closedloop-ai/symphony-alpha` checkout. ## Harness notes +- Codex reads files through shell output, which keeps the start and end of a + long file and drops its middle; one read of this file loses the build loop. + Read this file, each agent file and each reference to its last line, in + parts of at most 200 lines (`sed -n '1,200p'`, then `'201,400p'`, and so + on), never with one `cat`. `grep -n '^#'` on a file gives a named + section's lines. +- In both harnesses, whatever context does the writer's work, a spawned + `vibe-change-worker` or this one when it is told to do that work itself, + reads all of `../../agents/vibe-change-worker.md` and the "Acceptance + criteria before ready" and "Visual references" sections of + `references/quality-loop.md` (both past its first 400 lines) before it + plans or builds, and follows them: they set what the session decision table + records and when a render is allowed. - Codex: invoke as `$vibe`. Workers are this plugin's agents in `agents/` (`../../agents/.md` from this file). Codex cannot load them as registered agents, so spawn a subagent with the file's body as its diff --git a/plugins/vibe/skills/vibe/references/quality-loop.md b/plugins/vibe/skills/vibe/references/quality-loop.md index 1c028e36..684c7c46 100644 --- a/plugins/vibe/skills/vibe/references/quality-loop.md +++ b/plugins/vibe/skills/vibe/references/quality-loop.md @@ -519,6 +519,34 @@ X") gets the reference saved, read, turned into criteria and checked against a screenshot before ready. Someone who describes what they want in words gets criteria from their words, checked as before, with no render or compare. +### Recording it in the table + +The rules below leave their evidence in the session decision table, one line +each, in these shapes. Criterion lines sit under `## Acceptance Criteria`, +above `### Checks`; only check lines go below it. + +- `### Checks`: `Reference read : : `, + one per reference image on the session, every one the person gave and not + only the one the latest request names (a screenshot of our page as it is + now included), before any criterion or code for that request, a correction + included. +- `## Acceptance Criteria`: `AC-n (source: , reference ): ; `. One property per criterion: a shadow, + a type size and a type weight are three criteria, never one "match the + reference". A correction ("it still doesn't look like it") adds a criterion + for each property the reference shows that no criterion names yet, even + when a broad one such as "make ours like the second one" exists, before it + fixes anything. +- `### Checks`: `, AC-n, AC-m: reference check, compared + with after the last edit: ; met`, or `not met yet` with the difference that remains. + +With no reference image, a criterion from the person's words quotes them, +`AC-n (source: ): ""`, and is checked with the +cheapest honest evidence ("Acceptance criteria before ready"), never a +render. + ### Saving the reference The writer is a separate context that receives text, so an image the person diff --git a/plugins/vibe/tools/python/test_vibe_codex_reading_contract.py b/plugins/vibe/tools/python/test_vibe_codex_reading_contract.py new file mode 100644 index 00000000..94eefc00 --- /dev/null +++ b/plugins/vibe/tools/python/test_vibe_codex_reading_contract.py @@ -0,0 +1,132 @@ +"""Contract checks for how the writer's rules reach it, and how it records them. + +Codex loads a skill by reading its file through shell output, and a long +output keeps its start and end and drops the middle. The vibe SKILL.md runs +to about 14,400 tokens, so a single read lost the build loop, and a context +doing the writer's work never opened the writer's definition or +`quality-loop.md`. Told to read in parts, it stopped after the first 200 +lines, before the rules it needed. On main `642d235` the four visual-cues +Codex evals failed: a render with no reference image, criteria that +paraphrased the person's words without a source or result, and no +`Reference read` lines or token-mapped criteria for a reference (vibe 1.0.44 +Codex baseline). Claude on 1.0.44 missed the same records in three of the +four cases. + +1. The Codex harness note says to read long files to their last line in + parts, and it sits near the top of SKILL.md, inside the part a single read + keeps. +2. In both harnesses, whatever context does the writer's work reads the + writer's definition and the two named sections of `quality-loop.md` + before it plans or builds. +3. `quality-loop.md` gives the exact line shapes for a `Reference read`, a + criterion per reference property with its token, the reference check, and + a criterion quoting the person's words with no render; the writer points + at them. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +PLUGIN_ROOT = Path(__file__).resolve().parents[2] +VIBE_SKILL = PLUGIN_ROOT / "skills" / "vibe" / "SKILL.md" +QUALITY = PLUGIN_ROOT / "skills" / "vibe" / "references" / "quality-loop.md" +WRITER = PLUGIN_ROOT / "agents" / "vibe-change-worker.md" +NOTE = "Codex reads files through shell output" +RECORDING = "### Recording it in the table" +# A single Codex read of SKILL.md kept roughly its first 19,000 characters; +# stay well inside that so the note itself is never the part that drops. +KEPT_HEAD_CHARACTERS = 12_000 + + +def flat(text: str) -> str: + return re.sub(r"\s+", " ", text) + + +def harness_notes() -> str: + text = VIBE_SKILL.read_text() + start = text.index("## Harness notes\n") + end = text.index("\n## ", start + 1) + return flat(text[start:end]) + + +def recording() -> str: + text = QUALITY.read_text() + start = text.index(RECORDING + "\n") + end = text.index("\n### ", start + 1) + return flat(text[start:end]) + + +def test_the_codex_reading_note_is_in_the_part_a_single_read_keeps() -> None: + text = VIBE_SKILL.read_text() + assert NOTE in text + assert text.index(NOTE) < KEPT_HEAD_CHARACTERS + + +def test_codex_reads_long_files_to_the_end_in_parts_never_one_cat() -> None: + notes = harness_notes() + for phrase in ( + "keeps the start and end of a long file and drops its middle", + "one read of this file loses the build loop", + "Read this file, each agent file and each reference to its last line, in parts of at most 200 lines", + "never with one `cat`", + "`grep -n '^#'` on a file gives a named section's lines", + ): + assert phrase in notes, phrase + + +def test_whoever_does_the_writers_work_reads_its_rules_first_in_both_harnesses() -> None: + notes = harness_notes() + for phrase in ( + "In both harnesses, whatever context does the writer's work, a spawned `vibe-change-worker` or this one when it is told to do that work itself", + "reads all of `../../agents/vibe-change-worker.md`", + '"Acceptance criteria before ready" and "Visual references" sections of `references/quality-loop.md` (both past its first 400 lines) before it plans or builds', + "when a render is allowed", + ): + assert phrase in notes, phrase + + +def test_the_writer_reads_its_references_to_the_end_in_codex() -> None: + writer = flat(WRITER.read_text()) + read_first = writer[writer.index("## Read first") : writer.index("## Plan before code")] + for phrase in ( + "In Codex, read each, and this file, to its last line in parts of at most 200 lines", + "one read of a long file loses its middle", + '"Acceptance criteria before ready" and "Visual references", past its first 400 lines', + ): + assert phrase in read_first, phrase + + +def test_the_table_records_each_reference_read_criterion_and_check_in_one_shape() -> None: + assert RECORDING in QUALITY.read_text() + section = recording() + for phrase in ( + "`Reference read : : `, one per reference image on the session, every one the person gave and not only the one the latest request names (a screenshot of our page as it is now included), before any criterion or code for that request, a correction included", + "`AC-n (source: , reference ): ; `", + 'a shadow, a type size and a type weight are three criteria, never one "match the reference"', + 'A correction ("it still doesn\'t look like it") adds a criterion for each property the reference shows that no criterion names yet, even when a broad one such as "make ours like the second one" exists, before it fixes anything', + "reference check, compared with after the last edit", + "Criterion lines sit under `## Acceptance Criteria`, above `### Checks`; only check lines go below it", + ): + assert phrase in section, phrase + + +def test_the_record_shapes_come_first_in_visual_references() -> None: + # A Codex writer that reads "Visual references" from its first line stops + # before the section's end often enough that the shapes must lead it. + text = QUALITY.read_text() + visual = text.index("## Visual references\n") + assert visual < text.index(RECORDING) < text.index("### Saving the reference") + + +def test_with_no_reference_a_criterion_quotes_the_words_and_nothing_renders() -> None: + section = recording() + assert '`AC-n (source: ): ""`' in section + assert "never a render" in section + + +def test_the_writer_records_one_property_per_criterion_in_those_shapes() -> None: + writer = flat(WRITER.read_text()) + assert "one property per criterion, with the design token or catalog component that delivers it" in writer + assert 'Record each line in the shapes `quality-loop.md`, "Recording it in the table", gives' in writer