Skip to content

[Improve] Check the work before the agent reports to a person or ships, not only at turn end - #3085

Merged
mrubens merged 5 commits into
developfrom
feat/completion-check-triggers
Sep 22, 2026
Merged

mrubens merged 5 commits into
developfrom
feat/completion-check-triggers

Conversation

@mrubens

@mrubens mrubens commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The completion check from #3071 runs once, when the OpenCode session goes idle. On an autonomous task that is the very end: implement, validate, proof, push, open the pull request, report. A 30-minute turn is common. So a wrong claim mid-turn is not caught until everything downstream of it has happened, and on the delivery turn the check lands after the push, where a flag can only become a follow-up commit.

Change

The same check now also runs at the two moments that matter, and at those moments it can hold the action:

Trigger What is checked On a flag
The agent is about to report to a person (report_to_parent_session, send_chat_reply, send_chat_message) The tool's own text as the report, against the request, checklist, diff, and recorded commands The tool call fails once with the reasons; the agent fixes or explains, then calls it again
The agent is about to ship (git push, gh pr create / ready / edit, glab mr create, manage_source_control create/update pull request) The agent's latest message as the report Same
Turn end As before Reopens the turn once, as before

How it holds a call. A new OpenCode plugin (roomote-completion-gate.js, written next to the existing Slack, tool-safety, and identity plugins) implements tool.execute.before. For candidate tools it calls a new sandbox-server procedure, commands.checkCompletionBeforeTool, under the run token; the harness runs the check and answers allowed or a denial with the reasons; the plugin throws the reason, which the agent reads as the tool result. The classification of tools lives in the harness, not the plugin, so it is unit-tested TypeScript.

Never stuck, never doubled. A denial is issued at most once per piece of work (request generation plus diff identity); the next call for the same work goes through. A check at report or ship time records the diff it saw, so the turn-end check does not repeat for an unchanged diff. Concurrent tool calls share one evaluation. Any failure (server unreachable, timeout, check skipped) allows the call. The plugin is inert unless the run carries ROOMOTE_COMPLETION_GATE=true, the same flag that turns on the turn-end check.

Agent instructions and docs describe the three moments and that a held call goes through on the second attempt.

Turn end on delegated tasks. On nightly, a delegated UI task (01az6027gjdzb) ended three of its four turns with an empty assistant message: the report went out through report_to_parent_session and nothing followed it. The turn-end check requires a report, so it never ran for that task. It now falls back to the agent's last non-empty message when the closing message is empty. The report trigger above covers the report itself.

Validation

  • Classifier: report tools by flattened MCP name, git push with global options, gh pr forms, platform PR creation; non-triggers (git status, test runs, PR comments, reads).
  • Harness: a turn ending with an empty message is checked against the agent's last message; a flagged report is held once with the tool's text as the report, the retry passes without a second platform request; a pre-push check uses the latest message and the turn-end check does not repeat for the same diff; non-trigger tools and ineligible runs pass.
  • Procedure: hands the call to the harness; harnesses without the method allow everything.
  • Plugin: loaded from disk as OpenCode would load it, against a local HTTP server: denial throws the reason with the right path, auth, and superjson body; allowed and non-candidate calls do not throw; unreachable server and gate-off both allow.
  • pnpm lint:fast, pnpm check-types:fast, pnpm knip; worker sandbox-server and run-task suites (the three failures are the known local-only ones: command-executor UTF-8 tail, tail-file-path symlink, OpenRouter variant flake).

Not in this PR

  • The ship-time report is the agent's latest message, which may be narration rather than a claim; the diff and command questions still apply in full.
  • No live run yet. Worth watching the first held call on nightly: the harness logs completion check held a <trigger> tool call.

@roomote-community

roomote-community Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • apps/worker/src/sandbox-server/lib/harnesses/opencode-server/completion-gate.ts recognizes the source-control tool's create_or_update_pull_request delivery action.
  • apps/worker/src/sandbox-server/lib/harnesses/opencode-server/completion-gate.ts leaves reopen_pull_request outside shipping triggers.

Reviewed 6b283de

'send_chat_reply',
'send_chat_message',
]);
const SHIP_MCP_ACTIONS = /^(create|update)_pull_request$/;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manage_source_control does not emit either action this regex accepts: its public action is create_or_update_pull_request. The plugin will call the procedure, but this classifier returns null, so the default autonomous PR delivery path is allowed through without the pre-ship completion check. Include the actual action here and cover it with a regression test.

const SHIP_MCP_ACTIONS = new Set([
'create_or_update_pull_request',
'update_pull_request',
'reopen_pull_request',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reopen_pull_request is not a shipping action: it only changes the existing PR's state and neither pushes code nor creates/updates its content. Including it here makes an unrelated reopen wait for, and potentially be denied by, the completion check; this also contradicts the preceding comment. Restrict the set to the actual create/update delivery actions.

@mrubens
mrubens marked this pull request as ready for review September 22, 2026 03:12
@mrubens
mrubens merged commit ab00a75 into develop Sep 22, 2026
18 checks passed
@mrubens
mrubens deleted the feat/completion-check-triggers branch September 22, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant