You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
TypeScript SDK: Let validators require human confirmation, fail-closed - #46
Implements validator confirmation from #45 in the TypeScript SDK.
A validator can block a call until a person approves it. The approval is tied to the exact call (event, phase, target and payload), so it can't be reused for a different one.
Validators call requireConfirmation(params, { reason, secret }). It passes if this exact call was approved, and otherwise asks for approval.
Invokers call runValidationPhase(...). It asks the person, re-runs the validator with the approval, and blocks if anything goes wrong: no way to ask, a denial, a timeout, or another validator blocking.
Invokers call isConfirmedCall(...) after any later mutation. If the payload changed, the approval no longer applies.
Implements the "Requiring Confirmation" proposal from modelcontextprotocol#45:
- Wire types: ValidationResult.confirmation (ConfirmationRequest) and
invocation context.target / context.confirmations.
- RFC 8785 (JCS) canonicalization and the call binding hash
("sha256:" + hex SHA-256 of { event, phase, target, payload }).
- Validator helpers: confirmationRequired, findConfirmation,
requireConfirmation, and self-verifying HMAC tokens bound to id,
callHash, sessionId and expiry.
- Invoker: runValidationPhase asks only when every blocking result
carries a confirmation, blocks on denial/unreachable/expiry or a
request not bound to this call, re-validates each validator with its
own confirmation, and never asks for audit-mode validators.
isConfirmedCall re-checks the payload after later mutations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- JCS: reject array holes and non-plain objects (e.g. Date) instead of
collapsing them, so a call hash cannot hide a wire-visible change.
- Tokens: include the issuing validator's name in the MAC, so a
confirmation cannot be relabeled for another validator sharing a secret.
- Validation phase: track validator failure separately from the error
value, so rejecting with undefined still fails closed.
- Validation phase: recheck every request's expiry before re-validating,
since an earlier approval can expire while a later prompt is pending.
- README: check outcome.allowed in the example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Treat undefined or null validator results as failures
typescript/sdk/src/validation.ts:159
If a JavaScript validator or transport adapter resolves to undefined, this records neither a failure nor a result. blocks() then returns false, allowing the phase even with failOpen disabled. Treat a missing result as an invocation failure so the default remains fail-closed; handle null here too rather than letting it throw during result classification.
Enforce timeouts for unsettled validator invocations
typescript/sdk/src/validation.ts:162
timeoutMs is only forwarded to the callback. If invoke never settles, either invocation batch can wait forever despite a configured timeout—even for an audit or fail-open validator. Bound each invocation's wait and record expiration as a failed attempt so the existing mode and failure policy can run. The underlying adapter still needs to honor the forwarded timeout to cancel its work.
Add a check for custom toJSON methods in objects to prevent serialization errors.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Validator timeout is not enforced for pending callbacks
typescript/sdk/src/validation.ts:162
timeoutMs is forwarded to the callback but never bounds this await. A validator that never settles leaves the phase pending indefinitely, even when a timeout is configured and the validator is audit-only or fail-open. This also affects confirmation retries. Enforce a local deadline for each invocation and record expiry as a failed attempt so the existing mode/failOpen policy applies. The deadline can stop waiting without requiring cancellation support from the callback.
Ensure validation result is defined before returning.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Implement timeout handling for validation invocation.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
Match validator confirmations by name and request ID
typescript/sdk/src/validation.ts:258
When two entries share a validator name, this filter attaches both requests' confirmations to each retry, rather than only the confirmation that entry requested. A validator expecting its pending request ID can consequently receive another request's approval. Match both the name and request ID so each retry receives only its own confirmation.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Murthy G <31370582+murthy-g@users.noreply.github.com>
@murthy-g, I build test cases for agent approval flows, and you wrote the confirmation step in this PR that they exercise. My question is about that step: is a confirmation meant to be single use? As far as I can tell nothing records that a confirmation token was used. So before expiresAt the same approved call can be confirmed again, for example when the invoker retries.
If it is meant to be single use, how should the id of a used confirmation be recorded? The SDK keeps no state between calls.
The test cases are a public set of JSON fixtures (https://github.com/probityai/agent-evidence-vectors). A one-line answer is plenty. If it is meant to be single use, I can send the JSON fixture that shows the second use getting through. You could then add it to this PR's tests.
requireConfirmation takes an optional UsedConfirmations store. With it, a
confirmation passes once; a retry gets a new confirmation request instead
of reusing the old one. MemoryUsedConfirmations keeps accepted ids, keyed
by validator, until they expire. Without a store the SDK stays stateless.
The payload is shared by reference across the awaited confirmation
handler. Recheck its call hash before re-validating, so confirmedCallHash
never names a payload other than the one that was approved.
@murthy-g, I build test cases for agent approval flows, and you wrote the confirmation step in this PR that they exercise. My question is about that step: is a confirmation meant to be single use? As far as I can tell nothing records that a confirmation token was used. So before expiresAt the same approved call can be confirmed again, for example when the invoker retries.
If it is meant to be single use, how should the id of a used confirmation be recorded? The SDK keeps no state between calls.
The test cases are a public set of JSON fixtures (https://github.com/probityai/agent-evidence-vectors). A one-line answer is plenty. If it is meant to be single use, I can send the JSON fixture that shows the second use getting through. You could then add it to this PR's tests.
Good question! No, it's not single-use. The approval is tied to the exact call and expires at expiresAt, so retrying the same call before then works on purpose. The SDK doesn't keep state, so if you need one-time approvals, the invoker should track used confirmation ids itself. I added a note to the README so that's clearer.
Correction to my answer above: a confirmation is meant to be single use. The SEP text in #45 now says so (validators SHOULD record accepted confirmation ids until they expire and reject a repeat), and this PR adds it to the SDK: requireConfirmation takes a used store (MemoryUsedConfirmations is included), so a retry gets a new confirmation request instead of passing again. Without a store the SDK stays stateless. The README note describes this.
Please do send the fixture for the second-use case, and I'll add it to the tests.
@murthy-g thanks for adding the used store! At commit 50d3d31, allow-bound expects allow and approval-already-consumed expects deny with reason approval_replayed; they differ only in consumedApprovalIds, the ids already in the used store.
In your SDK that deny is the retry returning valid false, and a TS test can copy the JSON records and keep these digests:
# sha256 of vectors-approval-binding/cases/<case>/record.json at 50d3d312974688297569d62eebe5f45c3139053d
edf9debc8dd96359f8865324d47ac9ae8bc400cfadfc4ce524d33efae9bf11f8 allow-bound/record.json
9f14975c6032d782a096fc15388ab26d9d42574b206098627ba4675e12c23bd6 approval-already-consumed/record.json
# Python, the whole approval-binding set, which is newer than PyPI 0.25.0, so pin the commit:
pip install "agent-evidence-vectors @ git+https://github.com/probityai/agent-evidence-vectors@50d3d312974688297569d62eebe5f45c3139053d"
agent-evidence-vectors --corpus vectors-approval-binding
Adds the allow-bound and approval-already-consumed records from
probityai/agent-evidence-vectors at 50d3d31 (Apache-2.0), unchanged and
checked by sha256. They differ only in consumedApprovalIds, which the test
loads into the used store: the first passes, the second asks again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@murthy-g thanks for adding the used store! At commit 50d3d31, allow-bound expects allow and approval-already-consumed expects deny with reason approval_replayed; they differ only in consumedApprovalIds, the ids already in the used store.
In your SDK that deny is the retry returning valid false, and a TS test can copy the JSON records and keep these digests:
# sha256 of vectors-approval-binding/cases/<case>/record.json at 50d3d312974688297569d62eebe5f45c3139053d
edf9debc8dd96359f8865324d47ac9ae8bc400cfadfc4ce524d33efae9bf11f8 allow-bound/record.json
9f14975c6032d782a096fc15388ab26d9d42574b206098627ba4675e12c23bd6 approval-already-consumed/record.json
# Python, the whole approval-binding set, which is newer than PyPI 0.25.0, so pin the commit:
pip install "agent-evidence-vectors @ git+https://github.com/probityai/agent-evidence-vectors@50d3d312974688297569d62eebe5f45c3139053d"
agent-evidence-vectors --corpus vectors-approval-binding
This reply was written with AI assistance.
Thanks! Added both records unchanged in 5d70c1e (typescript/sdk/testdata/approval-binding/), checked by sha256. The test loads consumedApprovalIds into the used store: allow-bound passes, approval-already-consumed is asked to confirm again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements validator confirmation from #45 in the TypeScript SDK.
A validator can block a call until a person approves it. The approval is tied to the exact call (event, phase, target and payload), so it can't be reused for a different one.
requireConfirmation(params, { reason, secret }). It passes if this exact call was approved, and otherwise asks for approval.runValidationPhase(...). It asks the person, re-runs the validator with the approval, and blocks if anything goes wrong: no way to ask, a denial, a timeout, or another validator blocking.isConfirmedCall(...)after any later mutation. If the payload changed, the approval no longer applies.Audit-mode validators never ask and never block.