Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions changelog.d/feat-failure-notices.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
- Opt-in `failureNotices`: when a turn fails outright (provider error,
refusal, over-budget), post a short automatic notice to the agent's locus on
the first failure of a streak and every 5th after, so people can tell a
dead turn from a quiet one.
3 changes: 3 additions & 0 deletions src/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ export class Agent {
readonly refusalHandling: AgentConfig['refusalHandling'];
/** Prose delivery mode (see AgentConfig.proseRouting). Default 'locus'. */
readonly proseRouting: 'locus' | 'explicit' | 'hybrid' | 'disabled';
/** Post a room notice when a turn fails (see AgentConfig.failureNotices). */
readonly failureNotices: boolean;
/** Exact whole-response known-tool wrapper containment (default off). */
readonly toolWrapperProseGuard: boolean;
/** Prompt-cache TTL forwarded to the provider (see AgentConfig.cacheTtl). */
Expand Down Expand Up @@ -144,6 +146,7 @@ export class Agent {
this.thinking = config.thinking;
this.refusalHandling = config.refusalHandling;
this.proseRouting = config.proseRouting ?? 'locus';
this.failureNotices = config.failureNotices === true;
this.toolWrapperProseGuard = config.toolWrapperProseGuard ?? false;
this.cacheTtl = config.cacheTtl ?? '1h';
this.promptCaching = config.promptCaching ?? true;
Expand Down
26 changes: 26 additions & 0 deletions src/framework.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11531,6 +11531,7 @@ export class AgentFramework {

const streak = (this.refusalStreak.get(agentName) ?? 0) + 1;
this.refusalStreak.set(agentName, streak);
this.postFailureNotice(agentName, streak, `the model declined to respond (safety category: ${category})`);

this.logFailure({
agent: agentName,
Expand Down Expand Up @@ -11591,6 +11592,30 @@ export class AgentFramework {
this.opsAlert(kind, agentName, message, { data });
}

/**
* AgentConfig.failureNotices: tell the ROOM when a turn produced nothing.
* The chronicle marker already tells the resident; without this, people
* see only silence and can't tell a dead turn from a quiet one. Posted on
* the first failure of a streak and every 5th after, so a stuck loop can't
* flood the channel. Best-effort: a notice that can't be delivered is
* dropped (the stderr/failures.log records remain).
*/
private postFailureNotice(agentName: string, streak: number, what: string): void {
const agent = this.agents.get(agentName);
if (!agent?.failureNotices || !this.channelRegistry) return;
if (streak !== 1 && streak % 5 !== 0) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Mixed failures bypass notice limit Refusals and provider errors use separate streak counters. A refusal also resets the provider-error counter because it is a completed inference. If the two failure types alternate, each provider error qualifies as a first failure, so notices can be posted every other turn instead of being limited to the first and every fifth failure.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11606

Comment:
**Mixed failures bypass notice limit** Refusals and provider errors use separate streak counters. A refusal also resets the provider-error counter because it is a completed inference. If the two failure types alternate, each provider error qualifies as a first failure, so notices can be posted every other turn instead of being limited to the first and every fifth failure.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

const locus = this.channelRegistry.resolveLocus(agentName) ?? null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Notice can reach wrong room When a heartbeat turn starts, its speaking room is pinned. If an unrelated message changes the default channel before that turn fails, this fresh lookup can send the failure notice and its reason to the other room rather than the failed turn’s room.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11607

Comment:
**Notice can reach wrong room** When a heartbeat turn starts, its speaking room is pinned. If an unrelated message changes the default channel before that turn fails, this fresh lookup can send the failure notice and its reason to the other room rather than the failed turn’s room.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +11605 to +11607

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Background notice bypasses explicit routing A subconscious agent inherits failureNotices: true from its primary agent, but its speech must use an explicit destination. When a background turn fails without one, this lookup can fall back to the last incoming channel and post an unsolicited notice there.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11605-11607

Comment:
**Background notice bypasses explicit routing** A subconscious agent inherits `failureNotices: true` from its primary agent, but its speech must use an explicit destination. When a background turn fails without one, this lookup can fall back to the last incoming channel and post an unsolicited notice there.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

if (!locus) return;
const oneLine = what.replace(/\s+/g, ' ').trim();
const reason = oneLine.length > 160 ? `${oneLine.slice(0, 157)}…` : oneLine;
const text =
`⚠️ [automatic notice] ${agentName}'s reply failed to generate` +
`${streak > 1 ? ` (${streak} in a row)` : ''}: ${reason}. ` +
'Nothing was lost from the conversation; it will see this and your messages on its next turn.';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Notice promises removed messages On a refusal with auto-rewind enabled, the notice posts before the rewind can remove the latest incoming human message. The next turn then sees a marker describing withheld content, not the message itself. Telling participants the agent will see their messages may discourage them from resending input it no longer has.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11614

Comment:
**Notice promises removed messages** On a refusal with auto-rewind enabled, the notice posts before the rewind can remove the latest incoming human message. The next turn then sees a marker describing withheld content, not the message itself. Telling participants the agent will see their messages may discourage them from resending input it no longer has.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

void this.channelRegistry.routeSpeech(agentName, text, locus)
.catch((err) => console.error(`[inference-failed] failure notice not delivered for ${agentName}:`, err));
Comment on lines +11615 to +11616

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Failed notice creates false marker An undeliverable notice normally makes routeSpeech resolve to null, so this catch does not handle it. The ordinary speech-failure callback then records a [discord-send-failed] marker saying the agent’s previous reply was saved to its archive, although the failed notice was not an archived reply. This gives the agent an inaccurate account of what was sent.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11615-11616

Comment:
**Failed notice creates false marker** An undeliverable notice normally makes `routeSpeech` resolve to `null`, so this catch does not handle it. The ordinary speech-failure callback then records a `[discord-send-failed]` marker saying the agent’s previous reply was saved to its archive, although the failed notice was not an archived reply. This gives the agent an inaccurate account of what was sent.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

}

private noteInferenceExhausted(
agentName: string,
reason: string,
Expand All @@ -11603,6 +11628,7 @@ export class AgentFramework {

// (1) Durable stderr line — works in headless/daemon mode with no client.
console.error(`[inference-failed] agent=${agentName} consecutive=${streak}: ${reason}`);
this.postFailureNotice(agentName, streak, reason);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Notice overlooks delivered narration A tool-using turn can publish narration during one round and encounter a provider error in a later round. This notice still says the reply failed to generate, although the room received part of it. That wording can mislead participants about what they have already seen.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 11631

Comment:
**Notice overlooks delivered narration** A tool-using turn can publish narration during one round and encounter a provider error in a later round. This notice still says the reply failed to generate, although the room received part of it. That wording can mislead participants about what they have already seen.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.


// (1b) Machine-greppable durable record, independent of journald/unit log
// redirects: logs/failures.log under the host's working directory. This is
Expand Down
7 changes: 7 additions & 0 deletions src/types/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,13 @@ export interface AgentConfig {
*/
proseRouting?: 'locus' | 'explicit' | 'hybrid' | 'disabled';

/**
* Post a short automatic notice to the agent's speaking room when a turn
* fails outright (model call error, refusal, over-budget) — first failure
* of a streak and every 5th after. Default false.
*/
failureNotices?: boolean;

/**
* Fail-closed containment for a text response whose entire visible prose is
* an invocation-shaped wrapper naming a tool registered on that exact turn.
Expand Down
21 changes: 21 additions & 0 deletions test/inference-failure-observability.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,24 @@ test('per-agent isolation: one agent failing does not flag another', () => {
assert.equal(fw.consecutiveInferenceFailures.get('cairn'), 2);
assert.equal(fw.consecutiveInferenceFailures.get('lena') ?? 0, 0);
});

test('failureNotices: room notice on failure 1 and every 5th; off by default', async () => {
const { fw, restore } = makeHarness();
const posted: Array<{ text: string; locus: string | null }> = [];
fw.channelRegistry = {
resolveLocus: () => 'discord:g:salon',
routeSpeech: async (_agent: string, text: string, locus: string | null) => { posted.push({ text, locus }); return { delivered: true }; },
};
const addMessage = () => {};
fw.agents.set('cairn', { failureNotices: true, getContextManager: () => ({ addMessage }) });
fw.agents.set('quiet', { failureNotices: false, getContextManager: () => ({ addMessage }) });
try {
for (let i = 0; i < 5; i++) fw.noteInferenceExhausted('cairn', '400 image/webp label but PNG bytes');
fw.noteInferenceExhausted('quiet', 'boom');
} finally { restore(); }
await new Promise((r) => setImmediate(r));
assert.equal(posted.length, 2, 'failure 1 and failure 5 only');
assert.equal(posted[0].locus, 'discord:g:salon');
assert.match(posted[0].text, /^⚠️ \[automatic notice\] cairn's reply failed to generate: 400 image\/webp/);
assert.match(posted[1].text, /\(5 in a row\)/);
});
Loading