Repository navigation
feat(prose): autoContinue — continue a turn the model ends itself until skip_reply #744
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
4c0bdd4
feat(prose): autoContinue — continue a turn the model ends itself unt…
antra-tess 900fc6d
fix(prose): review of #744 — notice at the continuation's start, its …
antra-tess 89a8413
fix(prose): review of #744 — refusal names the kind, never the value;…
antra-tess a676cf8
fix(prose): review of #744 — a silent tick doesn't restart a run; ref…
antra-tess be1ecee
fix(prose): nissa-seru review of #744 — row at the stream's start, na…
antra-tess e05a311
fix(prose): review of #744 — a bounced close is the bounce's; the sha…
antra-tess File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| - **Hosts:** `AgentConfig.autoContinue` (default off): a turn the model ends | ||
| itself, with prose or with nothing, is followed by another turn straight | ||
| away, until the agent calls `skip_reply` (or another tool that ends the | ||
| turn), ends on a `>>skip_reply` / `>>>skip_reply` envelope, or reaches | ||
| `maxConsecutive` continuations in a row (default 3, 1 to 20). The count | ||
| starts again on any wake with a cause of its own (a silent heartbeat tick | ||
| isn't one). As the next turn starts it reads a short `[continue]` row | ||
| naming the agent, saying another turn starts, how to stop and that | ||
| unmarked words are sent as usual (and, on the last continuation, that it | ||
| waits for the next event after it); the row is written only when the | ||
| continuation is that turn's whole cause, never for a wake that doesn't run. | ||
| The next turn keeps the turn's locus unless a batched message names | ||
| another channel. Its wake has reason `auto-continue`. Not continued: | ||
| refusals, `max_tokens` cut-offs, the stall guard's ends, silent heartbeat | ||
| ticks, a turn whose closing prose bounced, ephemeral runs, conversation forks, an | ||
| agent offered no way to stop, and a quiesce (which drops a queued | ||
| continuation); a ` !` continuation runs instead. Each continuation is a | ||
| full inference: an agent that answers and ends its turn gets at least one | ||
| more per answer. A malformed setting is refused when the agent is built, | ||
| naming the value's kind, never the value. New exported types | ||
| `AutoContinueConfig` and `AutoContinueRun` (`InferenceRequest.autoContinueRun`). | ||
| See docs/auto-continue.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| # Continuing until skip_reply (`autoContinue`) | ||
|
|
||
| `autoContinue` is a default-off per-agent setting. With it on, a turn the | ||
| model ends itself is followed by another turn straight away, instead of the | ||
| usual pause until the next event (a message, a heartbeat, a timer). The agent | ||
| keeps going until it says it's done: by calling `skip_reply`, or another tool | ||
| that ends the turn, such as `sleep`. | ||
|
|
||
| `true` turns it on with a cap of 3 continuations in a row: | ||
|
|
||
| ```json | ||
| { "agent": { "autoContinue": true } } | ||
| ``` | ||
|
|
||
| An object sets the cap, a whole number from 1 to 20: | ||
|
|
||
| ```json | ||
| { "agent": { "autoContinue": { "maxConsecutive": 5 } } } | ||
| ``` | ||
|
|
||
| ## What it costs | ||
|
|
||
| An agent that answers and simply ends its turn gets at least one more full | ||
| inference per answer: the continuation it then ends with `skip_reply`. One | ||
| that never calls `skip_reply` gets up to `maxConsecutive` more, and in locus | ||
| and hybrid modes each of those turns' prose is published. Each continuation's | ||
| rows also stay in the window. Keep the cap low, and expect roughly one extra | ||
| inference per turn for an agent that does skip. | ||
|
|
||
| ## Which turns continue | ||
|
|
||
| A turn continues when the model ended it (stop reason `end_turn` or | ||
| `stop_sequence`), whether its last response is prose or empty, unless: | ||
|
|
||
| - a tool ended it: `skip_reply`, `sleep`, any endTurn tool; | ||
| - its last prose envelope is a skip: `>>skip_reply` in explicit mode, or a | ||
| `>>>skip_reply` in hybrid mode whose hold is still standing at the end; | ||
| - it asked for a ` !` continuation (`>>#ops !`), which runs instead, with no | ||
| row, and counts toward nothing; | ||
| - its closing prose bounced (explicit or hybrid mode): the bounce, with the | ||
| bounce-wake cap, is that turn's answer, and continuing once the cap is | ||
| spent would restart the loop the cap breaks; | ||
| - it was a silent heartbeat tick; | ||
| - it ended with a refusal, a `max_tokens` cut-off, or the stall guard's | ||
| `no_progress` / `round_limit`; | ||
| - the agent is an ephemeral run (a subagent) or a conversation fork, or the | ||
| framework is stopping; | ||
| - the agent has no way to stop that it's offered: in locus or disabled mode, | ||
| `skip_reply` must be offered (a channel registry provides it) and allowed | ||
| (`allowedTools`). Explicit and hybrid agents can always end with a skip | ||
| envelope. | ||
|
|
||
| A turn that sent with a send tool and then closed in prose (held, not sent, | ||
| because the turn had already sent) still continues: its closing words can't | ||
| be told from unfinished work. The row tells the agent those words were held. | ||
|
|
||
| ## What the agent reads | ||
|
|
||
| As the continuation's turn starts, after every gate that could still stop it, | ||
| a short system row joins the window as the newest of the turn's inputs (after | ||
| the last turn's `[delivered]` receipt, outside explicit mode, whose envelopes | ||
| acknowledge themselves and write no receipt): | ||
|
|
||
| ``` | ||
| [continue] lena, another turn starts now. If you're done, call skip_reply; anything you write without it is sent as usual. | ||
| ``` | ||
|
|
||
| It names the agent, since residents read one shared message slot. For the | ||
| same reason it is never written while another resident's turn is alive: the | ||
| continuation waits for that turn to end, as a clock row does. When the | ||
| last turn's closing words were held because that turn had already sent, it | ||
| adds `Your closing words last turn were held, not sent: that turn had already | ||
| sent with a send tool.` The last continuation of a run adds `This is the last | ||
| continuation in a row: after this turn you wait for the next event either | ||
| way.` An agent that can't call `skip_reply` is told to end with a | ||
| `>>skip_reply` (explicit) or `>>>skip_reply` (hybrid) envelope instead. | ||
|
|
||
| The wording is neutral on purpose: a row that reads as a lapse ("you ended | ||
| without…") or as a budget ("2 of 3, one more left") invites an apology or a | ||
| "done here" reply, which in locus and hybrid modes would itself be published. | ||
| It is short and event-shaped: longer instruction blocks about output, sent as | ||
| user-role messages, have tripped provider classifiers. | ||
|
|
||
| The row is never written for a turn that doesn't run: a wake the inference | ||
| policy drops, or that a quiesce drops, writes none and gives back its count, | ||
| as does one a resumed turn (a budget restart) takes in; a row that can't be | ||
| stored gives back its count, and withdraws a turn the continuation alone | ||
| caused. A write that throws after its row landed counts as written. It is appended before the turn compiles and never | ||
| retracted, so the prompt-cache prefix stays stable. | ||
|
|
||
| ## The next turn | ||
|
|
||
| The continuation's wake has source `framework` and reason `auto-continue`. In | ||
| locus and hybrid modes it carries the turn's locus, so the next turn's plain | ||
| prose lands in the same locus rather than the process-global last-inbound | ||
| channel. (A hybrid `>>>` destination is a pin within a turn: the next turn | ||
| starts with none, as any turn does.) A turn whose wake named a channel that | ||
| resolves to none (it had no channel and took no fallback) passes that channel | ||
| on, so its continuation takes no fallback either. | ||
|
|
||
| A message that arrives meanwhile joins the same turn, as any batched wake | ||
| does, and then that turn is the message's: its channel outranks the | ||
| continuation's (ambient included, though the continuation's wake was queued | ||
| later), no `[continue]` row is written, and the run's count starts again. A | ||
| silent heartbeat tick batched with a continuation is not a cause of its own: | ||
| the continuation keeps its row and its count. | ||
|
|
||
| A host quiesce drops a queued continuation: at resume, possibly hours later, | ||
| "another turn starts now" would no longer be true. The next real event wakes | ||
| the agent as usual. | ||
|
|
||
| ## The cap | ||
|
|
||
| At most `maxConsecutive` continuations run in a row (default 3). The count | ||
| starts again when a turn starts with a cause of its own (anything but a | ||
| framework wake or a silent heartbeat tick), and when a tool ends a turn. A | ||
| silent tick batched with a continuation neither restarts the run nor takes | ||
| its row, so a resident whose ticks come faster than its turns still reaches | ||
| the cap. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| /** | ||
| * autoContinue (AgentConfig.autoContinue, docs/auto-continue.md): a turn the | ||
| * model ends itself is followed by another, unless it ended on a skip. This | ||
| * module holds what doesn't need the framework: which ends continue, and the | ||
| * row the next turn reads. | ||
| */ | ||
|
|
||
| /** The reason on a continuation's wake (InferenceRequest.reason). */ | ||
| export const AUTO_CONTINUE_REASON = 'auto-continue'; | ||
|
|
||
| /** | ||
| * Stop reasons of a turn the model ended itself. The others don't continue: | ||
| * a refusal has its own handling (retries, rewind, the reaction), a | ||
| * `max_tokens` end was cut off rather than chosen, and `no_progress` and | ||
| * `round_limit` are the stall guard's verdict that the turn wasn't moving. | ||
| */ | ||
| export const AUTO_CONTINUE_STOP_REASONS: ReadonlySet<string> = new Set(['end_turn', 'stop_sequence']); | ||
|
|
||
| /** | ||
| * The row a continuation's turn reads. It names the agent (residents read | ||
| * one shared message slot), says another turn starts, says how to stop | ||
| * (`stop`, an imperative such as "call skip_reply") and that anything written | ||
| * without it is sent as usual, and, on the last continuation of a run, that | ||
| * the agent waits for the next event after it. When the last turn's closing | ||
| * words were held because that turn had already sent, it says so. Short, | ||
| * neutral and event-shaped on purpose: it rides as a system row in the | ||
| * window, where instruction blocks about output have tripped classifiers, | ||
| * and wording that reads as a lapse or a budget invites a published reply. | ||
| */ | ||
| export function autoContinueNotice(opts: { | ||
| agentName: string; | ||
| stop: string; | ||
| last: boolean; | ||
| heldAfterSend?: boolean; | ||
| }): string { | ||
| return `[continue] ${opts.agentName}, another turn starts now.` + | ||
| (opts.heldAfterSend | ||
| ? ' Your closing words last turn were held, not sent: that turn had already sent with a send tool.' | ||
| : '') + | ||
| ` If you're done, ${opts.stop}; anything you write without it is sent as usual.` + | ||
| (opts.last ? ' This is the last continuation in a row: after this turn you wait for the next event either way.' : ''); | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.