Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions changelog.d/fix-67-end-turn-accounting.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
- **A turn ended by a tool result (`endTurn`, such as `skip_reply`) is recorded as a completed
turn** (#67). It emits `inference:completed` with the stream's cumulative usage, then
`inference:turn_ended`, writes a successful inference-log entry (`stopReason: 'turn_ended'`),
and adds its usage once to the session totals (`usage:updated`, `getSessionUsage()`). Like a
completion, it resets the consecutive-failure streak and the poison-history rewind budget, ends
a refusal-rewind episode (a forced `/unstick` reports that the model responded), and records a
provider-cooldown recovery. In hosts such as connectome-host, consumers that end a turn's display
on `inference:completed` (a typing indicator, notices that close on success, a voice relay) now
end it on these turns too, and a consumer that totals `inference:completed`'s `tokenUsage` now
counts them too; one that also adds each `inference:usage` sample counts them twice.
- **Every stream's billed rounds reach the session totals once, however the stream ends.** A
context-budget or physical-window restart is counted and logged where it is decided
(`stopReason: 'context_budget'` or `'physical_window'`), without `inference:completed`. A stream
that errors, aborts or throws after finishing rounds counts them, and its failure entry in the
inference log carries them. A turn abandoned by an operator quiesce, a stream superseded by a
newer one, one whose agent was disposed, or one that ended with no final response and no other
end, which wrote no entry before, now writes a failure entry naming what ended it and carrying
its rounds, and `inferenceCount` counts it, when it had finished a round; one that hadn't has
nothing to carry and writes none, as before. If the store fails while a stream's end is
recorded, the error is reported on stderr and what comes after it still runs: the rest of a
tool-ended turn's end, a restart's next stream, and the stream's teardown. Exactly one
entry of each stream carries its usage, so totals summed from the inference log count each
stream once, as the session's do. A health snapshot's token totals read the log's newest 1,000
entries, so with these new entries that window spans fewer turns.
- An inference-log entry for a stream the framework ended at a tool boundary (a tool-ended turn or
a restart) records a note in place of the compiled request, so these frequent entries don't each
add a whole-context blob to the store. They are still new entries: the inference log keeps no
retention, so it grows by one small entry for every tool-ended turn and restart, and by one
entry, with its request, for each abandoned, superseded or disposed stream that had finished a
round.
311 changes: 221 additions & 90 deletions src/framework.ts

Large diffs are not rendered by default.

16 changes: 14 additions & 2 deletions src/types/framework.ts
Original file line number Diff line number Diff line change
Expand Up @@ -413,18 +413,30 @@ export interface InferenceLogEntry {
success: boolean;
/** Error message if failed */
error?: string;
/** Request data or blob ID if large */
/** Request data or blob ID if large. A stream the framework ended at a
* tool boundary (stopReason 'turn_ended', 'context_budget' or
* 'physical_window') records a note in place of the body. */
request: unknown | { blobId: string };
/** Response data or blob ID if large (only if successful) */
response?: unknown | { blobId: string };
durationMs: number;
/** The stream's usage: its final response's, or, for a stream that ended
* without one, the provider rounds it finished before it ended. Exactly
* one entry of a stream carries it, the one its end wrote; a stream whose
* end writes none (a quiesce-abandoned turn, a superseded stream, a
* disposed agent's stream) gets a failure entry that carries it, when it
* had finished a round. One that hadn't has nothing to carry and gets no
* entry. */
tokenUsage?: {
input: number;
output: number;
cacheCreation?: number;
cacheRead?: number;
};
/** Stop reason from the model */
/** Stop reason from the model, or why the framework ended the stream at a
* tool boundary: 'turn_ended' (a tool result with endTurn), or
* 'context_budget' / 'physical_window' (a context-budget or
* physical-window restart). */
stopReason?: string;
}

Expand Down
3 changes: 2 additions & 1 deletion src/types/module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,8 @@ export interface ModuleContext {
* Modules use this to react to agent lifecycle without polling — e.g. an
* adapter that wants to show a "typing" indicator can bracket it between
* `inference:started` and the terminal events: `inference:completed` (the
* normal end), `inference:turn_ended` (a tool result ended the turn),
* model answered and the turn ended, with a final response or with a tool
* result that ended it, which `inference:turn_ended` then follows),
* `inference:aborted`, `inference:failed` and `inference:exhausted`, all of
* which carry `agentName`. A turn's tool-call cycles stay inside its
* stream, so one bracket spans them. A context-budget restart emits
Expand Down
6 changes: 6 additions & 0 deletions src/types/trace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ export type TraceEvent =
silent?: boolean;
})
| (TraceEventBase & {
/**
* The model answered and the turn ended: with the stream's final
* response, or with a tool result carrying `endTurn`, which
* `inference:turn_ended` then follows. `tokenUsage` is the stream's
* usage, cumulative across its rounds.
*/
type: 'inference:completed';
agentName: string;
durationMs: number;
Expand Down
Loading
Loading