Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ await server.start();

An agent wraps an LLM identity: model, system prompt, context strategy, and tool permissions. Multiple agents can coexist, each with independent context and inference state.

Agents can opt into the [tool result guard](docs/tool-result-guard.md) through
`agent_settings` with `{"action":"update","tool_result_guard":true}`. The
setting persists across restarts; withheld results remain recoverable in
Chronicle's audit history.

```typescript
{
name: 'researcher',
Expand Down
6 changes: 6 additions & 0 deletions changelog.d/tool-result-guard.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- Add the opt-in, durable `agent_settings.tool_result_guard` setting (recipe:
`toolResultGuard`). On a provider refusal after tool output, withhold the
latest result batch and retry inference once without rerunning tools or
automatically rewinding older messages. Full originals remain in an
append-only Chronicle audit log; pending output stays out of speculative
compression, and disabling the guard does not restore withheld results.
81 changes: 81 additions & 0 deletions docs/tool-result-guard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Tool result guard

The tool result guard is off by default. An agent can enable it through
`agent_settings`:

```json
{"action":"update","tool_result_guard":true}
```

Use `{"action":"get"}` to inspect the effective boolean and its source.
The setting persists across turns and process restarts. Set it to `false`
to disable it, or explicitly reset `tool_result_guard` to restore the recipe
default. Recipes can set `AgentConfig.toolResultGuard: true`.
Disabling the setting never restores previously withheld output.

## Behavior

The guard applies to the batch of results returned together by the agent's
latest tool round, including text, images, and errors. It recognizes a
structured provider `stopReason: 'refusal'`, not keywords in output, ordinary
provider errors, or natural-language refusal text.

On that signal, every result in the batch is withheld. Tool calls, result IDs,
and error flags remain intact; each entire payload becomes:

> Tool result withheld by the guard. The tool has already executed.

The refused attempt's partial assistant output is discarded. Inference is
retried once on the same model, within the same logical turn; executed tools
are never automatically run again. Refusal details stay in operational logs,
outside the agent-facing notice and setting description.

The guard takes precedence over Membrane's unchanged-input refusal retries
for a pending batch and its recovery attempt. A second refusal stops this
recovery without automatic rewind of older exchanges or human messages.
Explicit operator `/unstick` remains a separate action. A later clean tool
round can stage a new batch with its own single recovery allowance.

Normal successful rounds admit the preceding results to memory. This works
for framework yielding streams (including ephemeral agents and context-budget
restarts) and the backward-compatible direct `Agent.runInference` API.

## Chronicle and memory

Withholding is non-destructive. Before submitting a guarded batch, the host
appends a `staged` record to the Chronicle append-log state
`framework/tool-result-guard`. It contains the full `originals` (including
pre-truncation data, error strings, and image bytes), the serialized history
`content`, and the `wireResults`. A `linked` record connects its `batchId` to
the context message's `messageId`; later `accepted` or `withheld` records
record the outcome. No guard operation deletes or overwrites these records.
Payload fields larger than 10 KB use Chronicle blobs (`{blobId}`), following
the inference log convention; resolve them with `store.getBlob(blobId)` and
parse the JSON. The append-log snapshots retain the blob references.

The context manager initially receives only placeholders, so speculative
compression cannot incorporate output that is subsequently withheld. Raw
pending output goes directly to the provider. A clean following response
promotes the history payload through Chronicle's versioned message-edit API.
On a refusal, the placeholders remain. The audit slot is not a context or
compression source.

If the process stops before acceptance, placeholders remain after restart;
the full pending originals are still available in the audit. This is
deliberately conservative: an interrupted submission does not establish that
the output was accepted. Similarly, if context compilation omits a pending
exchange, its unsubmitted payload is not admitted to memory.

For example, an operator can inspect records without changing the agent's
view:

```ts
const store = framework.getStore();
const records = store.getStateJson('framework/tool-result-guard');
// For large logs, use getStateLen/getStateItemJson instead of loading all.
const historical = store.getStateJsonAt('framework/tool-result-guard', sequence);
```

This is reactive recovery, not pre-submission screening: the provider sees
the original batch once before returning the signal. The setting is not
retroactive and does not rewrite results already accepted into memory.
53 changes: 46 additions & 7 deletions src/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import type { Membrane, NormalizedMessage, NormalizedRequest, ContentBlock, Yiel
import { isAbortedResponse } from '@animalabs/membrane';
import { createHash } from 'node:crypto';
import type { CacheWireReceipt, KvUnifiedRequestHooks } from './kv-unified-wire.js';
import { ToolResultGuard, TOOL_RESULT_GUARD_NOTICE } from './tool-result-guard.js';
import {
toolResultDataToHistoryString,
truncateForHistory,
Expand Down Expand Up @@ -82,6 +83,7 @@ export class Agent {
readonly thinking: AgentConfig['thinking'];
/** Refusal auto-rewind policy (see AgentConfig.refusalHandling). */
readonly refusalHandling: AgentConfig['refusalHandling'];
readonly toolResultGuard: ToolResultGuard;
/** Prose delivery mode (see AgentConfig.proseRouting). Default 'locus'. */
readonly proseRouting: 'locus' | 'explicit' | 'hybrid' | 'disabled';
/** Exact whole-response known-tool wrapper containment (default off). */
Expand Down Expand Up @@ -143,6 +145,7 @@ export class Agent {
this.temperature = config.temperature;
this.thinking = config.thinking;
this.refusalHandling = config.refusalHandling;
this.toolResultGuard = new ToolResultGuard(config.name, contextManager, config.toolResultGuard);
this.proseRouting = config.proseRouting ?? 'locus';
this.toolWrapperProseGuard = config.toolWrapperProseGuard ?? false;
this.cacheTtl = config.cacheTtl ?? '1h';
Expand Down Expand Up @@ -582,17 +585,28 @@ export class Agent {
// Filter tools to only allowed ones
const tools = availableTools.filter((t) => this.canUseTool(t.name));

// The direct Agent API uses the same admission rules as framework-driven
// streams. Stage before compiling so compression only sees placeholders.
const guardedResults = this._state.status === 'ready' && this.toolResultGuard.enabled;
if (guardedResults && this._state.status === 'ready') {
const content = this.buildToolResultMessages(this._state.toolResults)[0].content;
const wireResults = content.flatMap((block) => block.type === 'tool_result'
// buildToolResultMessages serializes every payload to a string.
? [{ toolUseId: block.toolUseId, content: block.content as string, isError: block.isError }] : []);
this.toolResultGuard.storeResults(content, wireResults, this._state.toolResults);
}

// Compile context (with optional injections)
const { messages, systemInjections } = await this.compileWithInjections(budget, injections);

// If we have pending tool results, add them
if (this._state.status === 'ready') {
if (this._state.status === 'ready' && !guardedResults) {
const toolResultMessages = this.buildToolResultMessages(this._state.toolResults);
messages.push(...toolResultMessages);
}

const request: NormalizedRequest = {
messages,
messages: this.toolResultGuard.prepareRequest(messages, true),
system: this.buildSystemPrompt(systemInjections),
config: {
model: this.model,
Expand Down Expand Up @@ -648,6 +662,7 @@ export class Agent {
} catch (error) {
// On error, go back to idle
this._state = { status: 'idle' };
this.toolResultGuard.recovering = false;
throw error;
}
}
Expand Down Expand Up @@ -773,7 +788,7 @@ export class Agent {
}

return {
messages,
messages: this.toolResultGuard.prepareRequest(messages),
system: this.buildSystemPrompt(systemInjections),
config: {
model: this.model,
Expand Down Expand Up @@ -830,6 +845,7 @@ export class Agent {
this.lastStreamOutputTokens = 0;

const request = await this.buildActivationRequest(availableTools, injections, budget);
request.messages = this.toolResultGuard.prepareRequest(request.messages, true);

const receiptAware = (this.contextManager as unknown as { getStrategy?: () => unknown })
.getStrategy?.() as {
Expand All @@ -854,6 +870,7 @@ export class Agent {
};
}

const agent = this;
const stream = this.membrane.streamYielding(request, {
emitTokens: true,
emitBlocks: false,
Expand All @@ -863,7 +880,13 @@ export class Agent {
// (cache-warm), where a framework-level requeue would recompile and
// land a different window. The framework's driveStream handles the
// resulting `retrying` event by discarding the abandoned attempt.
...(this.refusalHandling?.retries ? { refusalRetries: this.refusalHandling.retries } : {}),
// Membrane reads this per physical round, including after settings
// tools run. A guarded result must reach the host on its FIRST refusal;
// never spend retries resubmitting unchanged guarded content.
get refusalRetries() {
return agent.toolResultGuard.hasPending || agent.toolResultGuard.recovering
? 0 : agent.refusalHandling?.retries ?? 0;
},
});

this._state = { status: 'streaming', stream };
Expand Down Expand Up @@ -1041,9 +1064,21 @@ export class Agent {
request: NormalizedRequest,
signal?: AbortSignal
): Promise<InferenceResult> {
const response = await this.membrane.stream(request, { signal });
let response = await this.membrane.stream(request, { signal });
if (!isAbortedResponse(response) && response.stopReason === 'refusal') {
const ids = this.toolResultGuard.withhold('unknown');
if (ids) {
const withheld = new Set(ids);
request = { ...request, messages: request.messages.map((message) => ({
...message, content: message.content.map((block) => block.type === 'tool_result' && withheld.has(block.toolUseId)
? { type: 'tool_result', toolUseId: block.toolUseId, content: TOOL_RESULT_GUARD_NOTICE, isError: block.isError } : block),
})) };
response = await this.membrane.stream(request, { signal });
}
}

if (isAbortedResponse(response)) {
this.toolResultGuard.recovering = false;
const partialContent = response.partialContent ?? [];
const { toolCalls, speechContent } = this.extractToolCallsAndSpeech(partialContent);
return {
Expand All @@ -1056,10 +1091,14 @@ export class Agent {
};
}

const { toolCalls, speechContent } = this.extractToolCallsAndSpeech(response.content);
const guardedRefusal = response.stopReason === 'refusal' && this.toolResultGuard.recovering;
if (response.stopReason !== 'refusal') this.toolResultGuard.accept();
this.toolResultGuard.recovering = false;
const content = guardedRefusal ? [] : response.content;
const { toolCalls, speechContent } = this.extractToolCallsAndSpeech(content);

// Add assistant response to context
this.contextManager.addMessage(this.name, response.content);
if (!guardedRefusal) this.contextManager.addMessage(this.name, content);

return {
toolCalls,
Expand Down
Loading
Loading