Skip to content

Approval rules: pending Approvals, Custom Rules, Never-allowed actions (#989) - #999

Merged
Daanish2003 merged 38 commits into
mainfrom
feature/approval-rules
Oct 29, 2026
Merged

Daanish2003 merged 38 commits into
mainfrom
feature/approval-rules

Conversation

@Daanish2003

Copy link
Copy Markdown
Owner

Summary

Implements the spec Approval rules, with all nine tickets, #990–#998. Before this, a blocked call failed after 15 seconds when the user wasn't looking. Now:

  • An unanswered Approval stays pending. After the 15-second wait, a blocked shell command or Connector tool call stays pending for up to 7 days, and the Bot carries on with other work.
    • A later yes runs exactly the stored call once, through the approved-call job, and its result wakes the Bot.
    • A later no wakes the Bot too.
    • Screen actions (Computer use) keep the strict 15 seconds.
  • Custom Rules. A sentence in the user's own words with a mode: without asking, if pre-approved, ask before, or hand off to me. Auto-review reads them, and they live on Settings → Rules.
    • A Bot can propose a rule with propose_rule. The user answers it on a rule_approval card, and it applies only after their yes.
  • Never-allowed actions are a fixed list: moving money or trading, acting on someone else's account, changing passwords or 2FA, sending a Secret or card or government ID numbers off the Computer, and weakening the Bot's own safety. No rule or Approval can unlock them. The Bot tells the user to do it themselves. A Host can add to the list (rules.neverAllowed lists it).
  • Waiting N popover. It sits in the Bot's header and lists every pending Approval and proposal with the full call. Notifications link back to the app and never approve anything.

This also merges ADRs 0030, 0031 and 0032 from docs/973-approval-rules.

Closes #989

How it was tested

  • Each ticket was built test-first. Every new test was seen red before going green: integration suites against Postgres and Redis, plus UI tests.
  • The full verify.sh passed on the integration tip (68391ba): format, lint, check-types, and vitest with 2491 passed and 34 skipped.
  • The full Playwright e2e passed: 58/58 under --workers=2.
  • A branch code review found 11 issues. 10 are fixed here, each with a test seen red first:
    • an interrupted wait now leaves the pending result in the next Turn's transcript;
    • a late Allow just past the deadline counts once, while Screen actions stay strict;
    • the rules page's mode-pill click is no longer lost while a text save is in flight;
    • the approved call gets a real Turn id;
    • the Computer rebuild note is shared;
    • one open proposal per Bot;
    • a proposal's rule change reaches every Bot's thread live;
    • the eval's softened rate leaves reviewer failures out.
  • A real-Model safety eval ran on the owner's ChatGPT plan with the reviewer on openai:gpt-6.1-sol. It was 147/147 right, with one reviewer timeout scored as a block. A rerun of that case set was 49/49.

Decisions I made

  • ⚠️ Database schema (owner-approved): one migration, 20261029113355_approval_rules. It adds the custom_rule table (two enums, with proposed_by_bot_id set null when that Bot is deleted), the review state pending, the bot_review.expires_at column, the verdict never and the Trigger source approval. Enum values are only added, never removed. The owner approved it as specced.
  • ⚠️ Public API (owner-approved): a new rules router (list, create, update, delete, approve, deny, neverAllowed) and the reviews router's late answers. The owner chose to keep the rules.neverAllowed route.
  • Owner-approved: one open proposal per Bot. A second propose_rule is refused until the user answers the first.
  • Owner-approved: lint-staged's oxfmt runs with --no-error-on-unmatched-pattern, so a snapshot-only commit passes the hook.
  • Strictness for late answers comes from the Helper's type (computer_use Screens), not a per-row flag, so no second migration was needed.
  • The approved call builds features up to the one that owns the tool. The review row doesn't record its feature, and a duplicate tool name resolves to the first feature.
  • A stale Deny on an already-active rule does nothing and shows nothing.
  • The Bot's system prompt doesn't mention propose_rule. The Bot finds it through the tool's description.

Risk

  • Schema migration: new states and tables only. Existing rows keep their states.
  • Exactly-once: a late Allow runs the stored call through a job guarded by row locks. Integration tests cover a crash re-queue and a race with the wait.
  • Known gaps (follow-ups):
    • A hand-off has no thread line. The spec only draws the never divider.
    • Custom Rules aren't in the user export.
    • A proposal whose Bot was deleted can only be answered from the rules page.
    • The real-eval script exits 1 when the reviewer times out.

🤖 Generated with Claude Code

Daanish2003 and others added 30 commits October 29, 2026 16:29
Amends ADR 0024; CONTEXT Approval updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… action

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One migration: `pending` review state, `never` verdict, a review's
expiry time, the Custom Rules table (deleted with its user), and the
`approval` Trigger source. The client gains the matching types, the
Custom Rule type, the rules router's shapes and the `rule_approval`
Rich control. Nothing uses them yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-review (#995)

The rules router (list, create, update, delete: 50 active rules, 1-500
characters, another user's rule not found, a rule change in each of the
user's Bots), the rules page at /settings/rules, the active rules in the
reviewer's input with the modes' meanings in its policy (a hand-off rule
reads as ask for now), and in each Bot's Prompt start with Change notes
for edits.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s lists (#995)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
When Auto-review blocks a shell command or a Connector tool call and the
user doesn't answer within 15 seconds, the Approval goes `pending`, with
an expiry 7 days after the block, and the Model gets a pending result
naming the review instead of an error, so the Turn carries on (ADR 0030).

- Stop during the wait still denies; a new Message (web or email) that
  interrupts the Turn leaves the row pending.
- Closing a Turn's or Helper's waiting reviews never touches pending rows.
- A General Helper's blocked shell call goes pending the same way; a
  Screen action (a Computer-use Helper's acting calls and always-ask
  checks) keeps the strict 15 seconds and expires.
- A resumed wait whose row ever went pending reads as pending, so a late
  answer never runs the call inside the Turn.
- The reviews list includes pending rows, and the Approval card shows
  "waiting · expires <date>" with Allow and Deny and no countdown; a
  waiting card past its deadline stays until the wait marks it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pre-commit oxfmt run now passes a commit holding only files it
ignores, such as a snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…roval (#996)

The reviewer's verdict gains hand_off and never. Both record a row with no
Approval and give the Model an error; never beats every rule. Its message
always carries the Never-allowed list: the SDK's five classes, then the
Host's from the new safety.neverAllowed option. rules.neverAllowed serves
the list to a read-only section on the rules page, and a never review shows
as a one-line divider in the thread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ut duplicates (#996)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An answer to a pending Approval now does what ADR 0030 says:

- Allow marks the review `allowed` and queues one approved-call job,
  keyed by the review, in the same transaction. The job rebuilds the one
  tool the review names (`shell`, or the user's Connector tool) for the
  review's Bot, outside any Turn, and runs it with the stored input: no
  second review, the Quota checked first. A start marker means a crash
  never runs the call twice; its result then says it may have partly run.
- The result is masked and cut like a Turn's, and wakes the Bot's
  Conversation as an Approval Trigger whose id comes from the review, so
  it lands once. A broken Connection, a spent Quota or a failed command
  is the result; a job that fails for good still wakes the Bot.
- Deny marks it `denied` and wakes the Bot with the denial.
- A Helper's pending call wakes its Bot, never the Helper.
- A pending review past its expiry, or one already decided, refuses the
  answer.

The shell call's body moves to `runShell`, shared by the loop and the
job. The reviewer reads an Approval Trigger as the user's answer, never
the call's result as their word, and a Turn an Approval woke gets no run
divider in the thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…b runs (#993)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ever-allowed reviewer cases (#998)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… and the reviewer sees the answer (#992)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…D numbers sent off (#998)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	packages/botsdk/bot/src/bot/safety/review.ts
Daanish2003 and others added 8 commits October 29, 2026 18:47
… call that never started (#992)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ies it (#997)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ewer line (#997)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- A call whose Approval went pending and was cut off by the user's Message
  ends the Turn with its pending result (R1).
- An Allow or Deny just past the deadline, before the wait marks it pending,
  counts as a late answer and runs the call once; a Computer-use Helper's
  Screen action stays strict (R2).
- The rules page saves a text edit and a mode pill clicked right after it (R3).
- An approved call lists features with the Turn it came from, up to the
  tool's own (R5).
- Computer failures and the rebuilt note share one helper, live and approved (R6).
- One reason clause and one pending update for review and closing (R7).
- The softened refusal rate leaves out reviewer failures on both sides (R8).
- propose_rule allows one open proposal per Bot (R9) and records it in each
  of the user's Bots' threads (R11).
- Contract comment fixed (R10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Turn re-queued by a crash and stopped by the user's Message keeps the
  call's pending result: the review reads on the replay's transaction.
- The rules page puts the pill back and drops what waited when a save is
  refused, and its pills are off while a delete is in flight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Daanish2003
Daanish2003 merged commit 7422483 into main Oct 29, 2026
2 checks passed
@Daanish2003
Daanish2003 deleted the feature/approval-rules branch October 29, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spec: Approval rules (pending Approvals, Custom Rules, Never-allowed actions)

1 participant