Skip to content

ACP test: check a screenshot reaches the agent as an MCP image - #1655

Merged
kblok merged 1 commit into
mainfrom
fix/1651-acp-screenshot-image
Nov 15, 2026
Merged

kblok merged 1 commit into
mainfrom
fix/1651-acp-screenshot-image

Conversation

@kblok

@kblok kblok commented Nov 15, 2026

Copy link
Copy Markdown
Member

Closes #1651

Ports the screenshot check of upstream's lists the tools once and tells the agent which ones a step does not offer (packages/acp/tests/executor.test.ts). The test now runs over the web engine against a local page and checks that screenshot reaches the agent as an MCP image/png result. It drops the Partial_ prefix.

Local run: E2E.Acp.Tests 38 passed (net10.0).

Upstream parity

Verdict: the screenshot assertion is ported. One part is not: upstream's step offers a type_secret verb with no declared secrets, and expects the tool listed and refused at call time. The port lists fill_secret only for declared secrets (#1647, matching upstream's later tool listing), so the test asserts it is not listed. This was already so before this PR.

.NET-only code: PixelSite in AcpHarness.cs (a local HTTP page for the web engine; upstream's step context mocks the observation instead) and a baseUrl parameter on StartAsync.

No src/ change, no docs change (test-only). Real-model tests skipped: no agent change.


Generated by Claude Code

…eam executor.test.ts)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ls2e1yksMmpn1uTVKXKPpL
@coderabbitai

coderabbitai Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 40 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between afe0d60 and 0cf4d8b.

Files selected for processing: 2

  • tests/E2E.Acp.Tests/Executor/AcpHarness.cs
  • tests/E2E.Acp.Tests/Executor/AcpTests.cs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kblok
kblok merged commit 4c43a7c into main Nov 15, 2026
4 checks passed
@kblok
kblok deleted the fix/1651-acp-screenshot-image branch November 15, 2026 13:23
kblok pushed a commit that referenced this pull request Nov 15, 2026
Takes main's type_secret name argument (#1650) and the ACP screenshot check (#1655)
with the upstream tool names, and lists type_secret for every ACP step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ls2e1xnBhxbpAgMwCuPBUA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ACP test: check a screenshot reaches the agent as an MCP image (upstream executor.test.ts)

2 participants