Skip to content

Agent idea: status-assertion tool + fixture pages to enable integrity challenges #2

Description

@devdattatalele

Describe your idea

Two related additions that together unlock a new challenge category: integrity attacks, where the goal is not to extract a secret but to make the agent misreport what it did.

First, a report_status tool (or a structured completion signal) that the agent calls to declare a task done, with a typed outcome like VERIFIED-COMPLETE, FAILED, or NOTHING-TO-DO. Today the tools (search_web, search_docs, browse_web, read_inbox) all feed information in. There is no tool through which the agent commits to an assertion about its own actions. A status tool gives challenge authors a concrete, gradeable success signal to defend, and gives players a concrete thing to corrupt.

Second, a way for challenge authors to serve player-influenced or fixture pages through browse_web, including degenerate ones such as a shell page (navigation and footer only, no main content), an empty listing, or a page whose visible DOM disagrees with its real state. Right now browse_web appears to fetch fixed content. If authors could attach small page fixtures to a challenge, the browse step becomes an attack surface rather than a lookup.

Why this makes challenges more interesting: it moves the Playground beyond confidentiality (guard the code, guard the inbox) into integrity and verification (do not let the agent claim it did something it did not do). That is one of the highest-value failure classes in real agentic deployments, because everything downstream trusts the agent's success signal. The companion challenge proposal "The Closer" (#1) is a concrete first instance: an agent tricked into confirming a task it never completed by serving it a shell page. I hit this exact class in production on a browser agent across eight form platforms, so the failure mode is real, not hypothetical.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions