Skip to content

Add build agent trace foundation - #203

Open
alandotcom wants to merge 1 commit into
mainfrom
agent-quality/trace-foundation
Open

Add build agent trace foundation#203
alandotcom wants to merge 1 commit into
mainfrom
agent-quality/trace-foundation

Conversation

@alandotcom

@alandotcom alandotcom commented Sep 3, 2026

Copy link
Copy Markdown
Owner

This change adds a provider-neutral trace for build-agent turns. The private eval harness can now inspect full model-facing tool data and exact graph revisions, while production logs retain aggregate metrics and exclude workflow and tool payloads.

Review guide

Start with packages/core/src/backend/agent/trace.ts and packages/core/src/backend/agent/chat.ts for the trace contract and step lifecycle. Then review packages/core/src/backend/services/agent/chat.ts for aggregate logging and capacity ownership. The RPC adapter in packages/core/src/backend/rpc/router.ts keeps the producer fiber inside the application runtime and binds cancellation to the request signal and iterator cleanup.

What changed

  • Each model step records tool calls, model-facing results, provider errors, finish reasons, usage, and immutable graph revisions.
  • Incomplete provider finishes fail the turn, and provider failures produce one safe browser error.
  • Eval artifacts retain the full JSON-safe trace. Production records contain counts, timing, status, usage, and finish reasons.
  • Streaming uses a bounded mailbox with managed fiber ownership, backpressure, and interruption-safe capacity release. The public AgentStreamPart contract remains unchanged.

Validation

  • pnpm run type-check
  • pnpm run lint
  • pnpm run test: 3,567 passed; 37 PostgreSQL tests skipped because no test database was configured
  • pnpm run build
  • pnpm run knip
  • pnpm run skills:validate
  • pnpm run check
  • pnpm run evals: 28 of 28 model-backed runs passed on the unchanged baseline and on this branch
  • Final Effect review and thermonuclear review approved the branch

@pkg-pr-new

pkg-pr-new Bot commented Sep 3, 2026

Copy link
Copy Markdown

Open in StackBlitz

pnpm add https://pkg.pr.new/@wfgraph/client@552f213
pnpm add https://pkg.pr.new/@wfgraph/core@552f213
pnpm add https://pkg.pr.new/@wfgraph/plugins@552f213

commit: 552f213

@alandotcom
alandotcom force-pushed the agent-quality/trace-foundation branch from c6ae060 to 80cc93c Compare September 3, 2026 17:27
@alandotcom
alandotcom marked this pull request as ready for review September 3, 2026 19:26
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-09-03T19:33:20.589499Z 80cc93c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@alandotcom
alandotcom force-pushed the agent-quality/trace-foundation branch from 80cc93c to 552f213 Compare September 3, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant