Redesign build agent eval verdicts

[alandotcom]

This PR replaces aggregate eval scoring with evidence-backed verdicts. Deterministic judges now evaluate completion, graph semantics, tool evidence, edit safety, recovery, and validation against the production workflow rules.

Review guide

Start with packages/evals/src/agent/trajectory.ts and packages/evals/src/agent/completion-facts.ts. Then review the focused judges under packages/evals/src/agent/judges/. The main policy seam is packages/core/src/backend/agent/publication-validation.ts, which now shares save-shape and publication checks with production.

Focus on whether each judge owns one policy and whether raw trace evidence supports every verdict. The browser stream contract remains unchanged.

What changed

• Stores normalized raw eval evidence and builds an ordered trajectory from the internal agent trace.
• Splits deterministic evaluation into completion, grounding, semantics, evidence-use, edit-safety, recovery, and validation judges.
• Keeps intent alignment and efficiency budgets advisory.
• Uses production graph-save and publication checks for agent validation, including integration requirements.
• Adds exhaustive action and Lifecycle Event expectations, revision-aware write checks, and refusal-recovery checks.
• Strengthens guidance for capability discovery, unavailable channels, Start Filters, required text fields, and publication blockers.
• Runs independent model scenarios with four-way concurrency while each scenario keeps serialized tool execution.

Validation

• pnpm run type-check
• pnpm run lint
• pnpm run test (3,779 passed; 38 PostgreSQL tests skipped because no database is configured)
• pnpm run build
• pnpm run knip
• pnpm run skills:validate
• pnpm run fix
• git diff --check
• Original model baseline: 28/28 passed
• PR 1 model baseline: 28/28 passed
• Full PR 2 runs exposed stochastic agent failures and evaluator false positives. Each concrete failure was reproduced or covered with deterministic tests. The final two affected live scenarios passed together after the last fixes. The rebased Cancel Filter scenario also passed.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
🔒 Security Review	✅ Completed 2026-09-03T17:31:26.219124Z	ca11983	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[pkg-pr-new]

Open in StackBlitz

pnpm add https://pkg.pr.new/@wfgraph/client@25084ff

pnpm add https://pkg.pr.new/@wfgraph/core@25084ff

pnpm add https://pkg.pr.new/@wfgraph/plugins@25084ff

commit: 25084ff