Skip to content

TypeScript SDK: Let validators require human confirmation, fail-closed - #46

Open
murthy-g wants to merge 9 commits into
modelcontextprotocol:mainfrom
murthy-g:ts-validator-confirmation
Open

murthy-g wants to merge 9 commits into
modelcontextprotocol:mainfrom
murthy-g:ts-validator-confirmation

Conversation

@murthy-g

@murthy-g murthy-g commented Oct 16, 2026 •

Copy link
Copy Markdown

Implements validator confirmation from #45 in the TypeScript SDK.

A validator can block a call until a person approves it. The approval is tied to the exact call (event, phase, target and payload), so it can't be reused for a different one.

  • Validators call requireConfirmation(params, { reason, secret }). It passes if this exact call was approved, and otherwise asks for approval.
  • Invokers call runValidationPhase(...). It asks the person, re-runs the validator with the approval, and blocks if anything goes wrong: no way to ask, a denial, a timeout, or another validator blocking.
  • Invokers call isConfirmedCall(...) after any later mutation. If the payload changed, the approval no longer applies.

Audit-mode validators never ask and never block.

Implements the "Requiring Confirmation" proposal from modelcontextprotocol#45:

- Wire types: ValidationResult.confirmation (ConfirmationRequest) and
  invocation context.target / context.confirmations.
- RFC 8785 (JCS) canonicalization and the call binding hash
  ("sha256:" + hex SHA-256 of { event, phase, target, payload }).
- Validator helpers: confirmationRequired, findConfirmation,
  requireConfirmation, and self-verifying HMAC tokens bound to id,
  callHash, sessionId and expiry.
- Invoker: runValidationPhase asks only when every blocking result
  carries a confirmation, blocks on denial/unreachable/expiry or a
  request not bound to this call, re-validates each validator with its
  own confirmation, and never asks for audit-mode validators.
  isConfirmedCall re-checks the payload after later mutations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 23:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved failure-handling and confirmation-binding gaps undermine the intended fail-closed guarantees.

6 open issues
What changed in this PR

Adds human-confirmation support to the placeholder TypeScript SDK, implementing the proposal in #45 without changing the specification.

Changes:

  • Adds confirmation wire types, call hashing, and signed tokens.
  • Implements validation, approval, re-validation, and final payload checks.
  • Adds unit tests and usage documentation.
File Description
typescript/​sdk/​src/​validation.ts Runs validation and confirmation flows.
typescript/​sdk/​src/​validation.test.ts Tests invoker approval and blocking behavior.
typescript/​sdk/​src/​types.ts Defines validation and confirmation wire types.
typescript/​sdk/​src/​jcs.ts Implements canonical JSON serialization.
typescript/​sdk/​src/​index.ts Exports the new SDK APIs.
typescript/​sdk/​src/​confirmation.ts Adds hashing, tokens, and validator helpers.
typescript/​sdk/​src/​confirmation.test.ts Tests canonicalization, hashes, tokens, and helpers.
typescript/​sdk/​README.md Documents confirmation integration.

Review effort: Balanced

Comment thread typescript/sdk/src/jcs.ts Resolved
Comment thread typescript/sdk/src/validation.ts Resolved
Comment thread typescript/sdk/src/confirmation.ts Resolved
Comment thread typescript/sdk/src/jcs.ts Resolved
Comment thread typescript/sdk/src/validation.ts Resolved
Comment thread typescript/sdk/README.md Resolved
- JCS: reject array holes and non-plain objects (e.g. Date) instead of
  collapsing them, so a call hash cannot hide a wire-visible change.
- Tokens: include the issuing validator's name in the MAC, so a
  confirmation cannot be relabeled for another validator sharing a secret.
- Validation phase: track validator failure separately from the error
  value, so rejecting with undefined still fails closed.
- Validation phase: recheck every request's expiry before re-validating,
  since an earlier approval can expire while a later prompt is pending.
- README: check outcome.allowed in the example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 17, 2026 02:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Call-binding gaps, missing-result handling, and unenforced validator deadlines undermine the fail-closed guarantees.

3 open issues
2 previously missed issues

In code that hasn't changed since last review

Medium severity Treat undefined or null validator results as failures

typescript/​sdk/​src/​validation.ts:159

If a JavaScript validator or transport adapter resolves to undefined, this records neither a failure nor a result. blocks() then returns false, allowing the phase even with failOpen disabled. Treat a missing result as an invocation failure so the default remains fail-closed; handle null here too rather than letting it throw during result classification.

Medium severity Enforce timeouts for unsettled validator invocations

typescript/​sdk/​src/​validation.ts:162

timeoutMs is only forwarded to the callback. If invoke never settles, either invocation batch can wait forever despite a configured timeout—even for an audit or fail-open validator. Bound each invocation's wait and record expiration as a failed attempt so the existing mode and failure policy can run. The underlying adapter still needs to honor the forwarded timeout to cancel its work.

6 resolved since last review

Review effort: Balanced

Comment thread typescript/sdk/src/jcs.ts Resolved
Comment thread typescript/sdk/src/jcs.ts Resolved
Comment thread typescript/sdk/src/validation.ts Resolved
Add a check for custom toJSON methods in objects to prevent serialization errors.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 17, 2026 02:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing validator results bypass fail-closed handling, and configured invocation timeouts are not enforced.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Validator timeout is not enforced for pending callbacks

typescript/​sdk/​src/​validation.ts:162

timeoutMs is forwarded to the callback but never bounds this await. A validator that never settles leaves the phase pending indefinitely, even when a timeout is configured and the validator is audit-only or fail-open. This also affects confirmation retries. Enforce a local deadline for each invocation and record expiry as a failed attempt so the existing mode/failOpen policy applies. The deadline can stop waiting without requiring cancellation support from the callback.

3 resolved since last review

Review effort: Balanced

Comment thread typescript/sdk/src/validation.ts Resolved
Ensure validation result is defined before returning.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 18, 2026 04:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Configured validator deadlines are not enforced, so an invocation that never settles can hang the entire validation phase.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread typescript/sdk/src/validation.ts Resolved
Implement timeout handling for validation invocation.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 18, 2026 22:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Payload hashing can miss array mutations, and same-name validator entries receive unrelated confirmations.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Match validator confirmations by name and request ID

typescript/​sdk/​src/​validation.ts:258

When two entries share a validator name, this filter attaches both requests' confirmations to each retry, rather than only the confirmation that entry requested. A validator expecting its pending request ID can consequently receive another request's approval. Match both the name and request ID so each retry receives only its own confirmation.

1 resolved since last review

Review effort: Balanced

Comment thread typescript/sdk/src/jcs.ts Resolved
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
@astrogilda

Copy link
Copy Markdown

@murthy-g, I build test cases for agent approval flows, and you wrote the confirmation step in this PR that they exercise. My question is about that step: is a confirmation meant to be single use? As far as I can tell nothing records that a confirmation token was used. So before expiresAt the same approved call can be confirmed again, for example when the invoker retries.

If it is meant to be single use, how should the id of a used confirmation be recorded? The SDK keeps no state between calls.

The test cases are a public set of JSON fixtures (https://github.com/probityai/agent-evidence-vectors). A one-line answer is plenty. If it is meant to be single use, I can send the JSON fixture that shows the second use getting through. You could then add it to this PR's tests.

requireConfirmation takes an optional UsedConfirmations store. With it, a
confirmation passes once; a retry gets a new confirmation request instead
of reusing the old one. MemoryUsedConfirmations keeps accepted ids, keyed
by validator, until they expire. Without a store the SDK stays stateless.
The payload is shared by reference across the awaited confirmation
handler. Recheck its call hash before re-validating, so confirmedCallHash
never names a payload other than the one that was approved.
Copilot AI balanced review requested due to automatic review settings October 22, 2026 23:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@murthy-g

Copy link
Copy Markdown
Author

@murthy-g, I build test cases for agent approval flows, and you wrote the confirmation step in this PR that they exercise. My question is about that step: is a confirmation meant to be single use? As far as I can tell nothing records that a confirmation token was used. So before expiresAt the same approved call can be confirmed again, for example when the invoker retries.

If it is meant to be single use, how should the id of a used confirmation be recorded? The SDK keeps no state between calls.

The test cases are a public set of JSON fixtures (https://github.com/probityai/agent-evidence-vectors). A one-line answer is plenty. If it is meant to be single use, I can send the JSON fixture that shows the second use getting through. You could then add it to this PR's tests.

Good question! No, it's not single-use. The approval is tied to the exact call and expires at expiresAt, so retrying the same call before then works on purpose. The SDK doesn't keep state, so if you need one-time approvals, the invoker should track used confirmation ids itself. I added a note to the README so that's clearer.

@murthy-g

Copy link
Copy Markdown
Author

Correction to my answer above: a confirmation is meant to be single use. The SEP text in #45 now says so (validators SHOULD record accepted confirmation ids until they expire and reject a repeat), and this PR adds it to the SDK: requireConfirmation takes a used store (MemoryUsedConfirmations is included), so a retry gets a new confirmation request instead of passing again. Without a store the SDK stays stateless. The README note describes this.

Please do send the fixture for the second-use case, and I'll add it to the tests.

@astrogilda

Copy link
Copy Markdown

@murthy-g thanks for adding the used store! At commit 50d3d31, allow-bound expects allow and approval-already-consumed expects deny with reason approval_replayed; they differ only in consumedApprovalIds, the ids already in the used store.

In your SDK that deny is the retry returning valid false, and a TS test can copy the JSON records and keep these digests:

# sha256 of vectors-approval-binding/cases/<case>/record.json at 50d3d312974688297569d62eebe5f45c3139053d
edf9debc8dd96359f8865324d47ac9ae8bc400cfadfc4ce524d33efae9bf11f8  allow-bound/record.json
9f14975c6032d782a096fc15388ab26d9d42574b206098627ba4675e12c23bd6  approval-already-consumed/record.json

# Python, the whole approval-binding set, which is newer than PyPI 0.25.0, so pin the commit:
pip install "agent-evidence-vectors @ git+https://github.com/probityai/agent-evidence-vectors@50d3d312974688297569d62eebe5f45c3139053d"
agent-evidence-vectors --corpus vectors-approval-binding

This reply was written with AI assistance.

Adds the allow-bound and approval-already-consumed records from
probityai/agent-evidence-vectors at 50d3d31 (Apache-2.0), unchanged and
checked by sha256. They differ only in consumedApprovalIds, which the test
loads into the used store: the first passes, the second asks again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 01:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@murthy-g

Copy link
Copy Markdown
Author

@murthy-g thanks for adding the used store! At commit 50d3d31, allow-bound expects allow and approval-already-consumed expects deny with reason approval_replayed; they differ only in consumedApprovalIds, the ids already in the used store.

In your SDK that deny is the retry returning valid false, and a TS test can copy the JSON records and keep these digests:

# sha256 of vectors-approval-binding/cases/<case>/record.json at 50d3d312974688297569d62eebe5f45c3139053d
edf9debc8dd96359f8865324d47ac9ae8bc400cfadfc4ce524d33efae9bf11f8  allow-bound/record.json
9f14975c6032d782a096fc15388ab26d9d42574b206098627ba4675e12c23bd6  approval-already-consumed/record.json

# Python, the whole approval-binding set, which is newer than PyPI 0.25.0, so pin the commit:
pip install "agent-evidence-vectors @ git+https://github.com/probityai/agent-evidence-vectors@50d3d312974688297569d62eebe5f45c3139053d"
agent-evidence-vectors --corpus vectors-approval-binding

This reply was written with AI assistance.

Thanks! Added both records unchanged in 5d70c1e (typescript/sdk/testdata/approval-binding/), checked by sha256. The test loads consumedApprovalIds into the used store: allow-bound passes, approval-already-consumed is asked to confirm again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants