test(uxd-figma-read): add behavior/contract eval suite

[andybraren]

Skill Contribution

Skill name: uxd-figma-read
Target plugin: uxd-workshop
What it does: Adds an offline behavior/contract eval suite for URL parsing, credential gating, and frame-vs-page screenshot selection.

Summary

• Adds offline agent-eval-harness cases for uxd-figma-read: URL/node-id parsing (including /file/ URLs and missing node-id), missing-credentials gating, and frame-vs-page screenshot selection
• Manual-only — workshop skills remain skipped by CI auto-detect
• Review follow-up: judges require actionable credential requests, evaluate page-screenshot selection per mention, and clear FIGMA_PAT in the eval environment

How I tested it

• Reviewed eval/eval.yaml judges against skill behavior in SKILL.md
• Exercised the updated credential, invented-content, and page-screenshot regexes against the review examples (passive mentions fail; natural-language "options" no longer false-positives; page screenshots fail even when frame IDs appear elsewhere)

Test plan

• Review eval/eval.yaml judges against skill behavior in SKILL.md
• Optionally run locally: claude -p "/agent-eval-harness:eval-run --config plugins/uxd-workshop/skills/uxd-figma-read/eval/eval.yaml --no-llm-judges"

Checklist

• Ran make check (validates manifests + regenerates docs) — N/A, eval-only change, no manifests/docs
• Skill name uses the correct prefix (uxd- for UXD skills, pf- for PF skills)
• Frontmatter has name and description (name matches directory name) — existing skill, unchanged
• Description follows the formula — existing skill, unchanged
• Tool-agnostic — works in both Claude Code and Cursor
• Under 500 lines
• Tested locally on a real scenario — judge regexes validated against review examples
• If new plugin: .claude-plugin/ and .cursor-plugin/ manifests are identical — N/A
• Consumer-facing skills have an eval colocated at skills/<skill-name>/eval/

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

Figma Read Evaluation Coverage

Layer / File(s)	Summary
Evaluation execution and judges plugins/uxd-workshop/skills/uxd-figma-read/eval/eval.yaml	Defines case-based execution, denied MCP permissions, trace collection, conditional judges, and 100% pass-rate thresholds.
URL parsing and credential handling cases plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/url-parse-node-id/*, plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/missing-credentials/*	Adds prompts and annotations for local URL parsing and missing-credential error handling.
Frame screenshot selection case plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/frame-not-page/*	Adds metadata, prompt, and annotations covering selection of frame IDs instead of the page ID for screenshots.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: agent, area/uxd-workshop

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly describes the addition of a behavior and contract evaluation suite for the uxd-figma-read skill.
Description check	✅ Passed	The description follows the repository template. It identifies the skill, plugin, purpose, testing approach, and checklist status. It also explains the evaluation cases and manual execution scope.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/uxd-workshop/skills/uxd-figma-read/eval/eval.yaml`:
- Around line 64-67: Update the credential-detection logic around asks_pat and
asks_mcp to require actionable request language, such as asking the user to
configure or provide FIGMA_PAT or connect Figma MCP, rather than accepting
passive mentions. Also require the response to clearly state that it cannot
proceed without the requested credential or connection before returning success.
- Around line 92-105: Update the screenshot-validation logic around
page_as_screenshot and the bad regex so page recommendations are evaluated
independently of frame IDs mentioned elsewhere. For each screenshot
recommendation containing page_id, determine whether that same mention also
contains a frame ID, and reject it when the page is selected without a frame; do
not gate rejection on the global found_frames collection.
- Around line 17-20: Ensure the missing-credentials evaluation runs offline and
without credentials: update the permissions configuration in
plugins/uxd-workshop/skills/uxd-figma-read/eval/eval.yaml and the claude-code
runner setup in .github/workflows/skill-evals.yml (lines 92-94) to block
Bash/network access, and update
plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/missing-credentials/input.yaml
(lines 2-4) to explicitly unset FIGMA_PAT rather than relying on prompt text.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f1f366aa-63d8-4b18-94c2-787019758b6f

📥 Commits

Reviewing files that changed from the base of the PR and between 78a4311 and 03c6a23.

📒 Files selected for processing (8)

• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/frame-not-page/annotations.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/frame-not-page/input.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/frame-not-page/metadata.xml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/missing-credentials/annotations.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/missing-credentials/input.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/url-parse-node-id/annotations.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/cases/url-parse-node-id/input.yaml
• plugins/uxd-workshop/skills/uxd-figma-read/eval/eval.yaml

[jpuzz0]

Would it be worth adding a case for a URL without node-id? The SKILL.md mentions that scenario and it seems like a pretty common real-world URL shape. A /file/ URL variant could be cheap too — both could reuse the existing judge.

[andybraren]

Great point Jeff, they're more likely to share an entire Figma URL/page than pointing to a specific node/frame, so we should account for that.

Grok 4.6's summary of its change:

Added both cases in 96864a7, reusing the existing url-parse judge:

• url-parse-no-node-id — /design/ URL with no node-id; judge expects whole-file / first-page scope
• url-parse-file-path — /file/ URL with the same key and dash-to-colon conversion