From d45bbfc6b4857684e4d51dc710837bb2f8bfa0a5 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Sat, 10 Oct 2026 10:47:27 -0400 Subject: [PATCH 1/4] =?UTF-8?q?=E2=9C=A8=20Evaluate,=20preview=20and=20con?= =?UTF-8?q?verse=20as=20a=20trusted=20host=20(#883,=20PR=201=20of=202)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A host that means to discuss something with an agent and show somebody what the reply *says* needs three things core did not have. **Evaluate the reply as a root.** `evaluateGeneratedXmdRoot()` admits source through the same walk, the same durable record and the same pinned resolution a fragment gets, and differs over exactly one construct: a top-level ``, which is what lets a reply select what it renders. The admission's context travels with its policy, so a root admission does not resume as a fragment and a fragment admission does not resume as a root — and because a fragment record states no context, every record already written stays readable and resumable as what it is. The walk's invariant inputs are bundled rather than threaded, so the one difference is read from where the rest of the decision is instead of copying the policy walk per context. It answers with a `Result`, because the two kinds of failure are different things for a host to hold. Source core refused, and work the root's own elements failed at, are the request's and come back as `Err` carrying what the root had rendered. A durability failure, an absent Files provider, a secret-policy refusal and a failed teardown are not, and each keeps its own classification. **Read what an arriving prefix already says.** `previewGeneratedXmdRoot()` is a pure projection with no `Operation`: it reads no file, resolves no name, invokes no component, evaluates no expression and grants no admission. Source still arriving is a success that says so; source the language does not allow where it is written is a refusal. Telling those apart needs the scanner's own walk — it is the only reader that knows how far it got and why it stopped — so the scan now reports the constructs it gave up on, beside the elements it already reported. A host recognizing `` with a regular expression would be a second reading of one syntax. **Hold a conversation open.** `useAgentConversation()` is the conversation a document's `` and `` cannot be: no element to name a placement, no journal already around it, and a history that outlives the process. It is an execution the host keeps open, and three things follow. It is an *element* — a host-declared identity component in a synthetic root of core's own — because a provider and its placement coordinator are installed inside `Execution.document` and a preparation precedes both, so the session is placed under the engine's own claimant-issued identity rather than under anything the host supplied. Its root is never completed, because a recorded terminal replays instead of reopening: the element serves turns until the resource is released, and the halt appends nothing. And each turn is one journaled Prompt under its own durable name, run in a task of its own, so cancelling the call halts the turn, leaves its append unmade, and makes the next turn a new turn rather than that one resumed. A turn whose outcome the history does not hold refuses the conversation before it opens — resuming it would re-send a prompt the provider may already have accepted, and writing a terminal for it would make an unfinished turn look complete. The history is bound to the conversation it belongs to, so one established for another id or another agent refuses rather than reconnecting the wrong chat. `runPrompt` is now shared rather than copied: the component and the host operation both reach the one definition of what a canonical turn is. Covers C1, C2 and C3 of the issue's frozen matrix. --- architecture.md | 132 ++++ packages/core/host.ts | 81 +++ packages/core/src/agent/conversation.ts | 673 +++++++++++++++++ .../core/src/agent/function-components.ts | 14 +- packages/core/src/errors.ts | 2 +- packages/core/src/generated-interpolation.ts | 25 + packages/core/src/generated-xmd-preview.ts | 236 ++++++ packages/core/src/generated-xmd.ts | 540 +++++++++++--- packages/core/src/scanner.ts | 130 +++- .../core/tests/agent-conversation.test.ts | 605 ++++++++++++++++ .../core/tests/generated-xmd-preview.test.ts | 307 ++++++++ .../core/tests/generated-xmd-root.test.ts | 677 ++++++++++++++++++ specs/acp-client-spec.md | 78 ++ specs/executable-mdx-spec.md | 5 +- 14 files changed, 3373 insertions(+), 132 deletions(-) create mode 100644 packages/core/src/agent/conversation.ts create mode 100644 packages/core/src/generated-interpolation.ts create mode 100644 packages/core/src/generated-xmd-preview.ts create mode 100644 packages/core/tests/agent-conversation.test.ts create mode 100644 packages/core/tests/generated-xmd-preview.test.ts create mode 100644 packages/core/tests/generated-xmd-root.test.ts diff --git a/architecture.md b/architecture.md index a8c94800b..b748131ae 100644 --- a/architecture.md +++ b/architecture.md @@ -2469,6 +2469,135 @@ Mutation-proposal admission and workflow-bundled Markdown component admission remain unbuilt. Directory registration is not among them: a workflow Agent is given no directory to register. +### A generated root, and the context that is a ceiling + +A host that shows somebody what an Agent's reply *says* needs the one construct +a fragment has no context for. `evaluateGeneratedXmdRoot()` admits source as a +**root**: a top-level `` selects what the reply renders and everything +around it is work the reply does. `` and `` stay unavailable in +both contexts — generated source has no caller to claim content from and no +value body to answer — and an `` written below the top level declares +nothing in a root either, which is the same lexical rule an authored body is +held to. + +Nothing else differs. The same whole-source preflight, the same `generated_xmd` +durable admission, the same pinned identities and the same refusal classes, so +there is one walk rather than one per context: the context is a term of the +admission the walk is decided against, beside the table and the request +ceilings. + +**Which context an admission was made in is one of its ceilings.** A retained +root admission does not resume as a fragment and a fragment admission does not +resume as a root — either would expand source nobody admitted *this* run for — +so the context travels in the retained policy and is compared whole, exactly as +the class selection and the pinned identities are. A fragment record states no +context, which is what every record written before roots existed says and what +this build still writes for one, so nothing already retained is rewritten or +reinterpreted. A root record carries the term; a build that does not know it +finds a member it has no rules for and refuses the continuation. + +Rendering is the document root's own. A root declaring top-level `` +selects through the same body partition a text root uses, under the same error +mode, so documentation around the regions executes and renders nothing and a +region that fails keeps what it rendered. The result states the canonical +rendering and whether the root declared a region at all, because an empty region +and no region are different answers and a host suppressing the ordinary +whole-body fallback must not have to read the source to tell them apart. + +The operation answers with an Effection `Result`, and the split is deliberate. +Source core refused, and work the root's own elements failed at, are the +request's own outcome: they come back as a failure carrying the rendering the +root had produced, which is what a host shows beside the reason. A failure of +the *run* is not: a journal that stopped describing it, an absent Files +provider, a secret-policy refusal and a failed teardown each keep the +classification they already have, so a host cannot mistake one for a reply +worth correcting. + +### Previewing source that is still arriving + +A host streaming a reply holds a prefix rather than a document, and showing +what it already says is a different question from deciding whether it may run. +`previewGeneratedXmdRoot()` answers the first and nothing else: a pure +projection over the accumulated prefix, with no `Operation`, because it reads no +file, resolves no name, invokes no component, evaluates no expression and grants +no admission. A prefix that projects cleanly has been granted nothing, and the +provider's own completion and the generated-root admission remain the only +things that decide a root runs. + +It is core's because the alternative is a second reading of one syntax. A host +recognizing `` with a regular expression would have to re-decide what a +fence, a quoted `>`, an inline code span and a tag-like expression are, and the +two readings would disagree the first time one of them was wrong. So the scan +reports the constructs it began and could not finish — beside the elements it +already reports, through the same walk — because only that walk knows how far it +got and why it stopped. Source still arriving is a success that says so; source +the language does not allow where it is written is a refusal. The two are +distinct, and a whole prefix is never a claim that the provider has finished. + +Only text whose meaning is already settled projects. An expression, an +interpolation and a nested component wait for evaluation, and a passive fence is +the literal example it looks like, so a tag written inside one creates no +region. One rule decides what reads a binding, shared with admission, because +text one of them called literal and the other called a read would be two +readings of one rule. + +### A conversation the host holds open + +`` and `` are written where a conversation belongs: inside an +expansion, with an element to name the placement and a journal already around +them. A host discussing something with an agent has neither and still needs the +genuine thing, so `useAgentConversation()` opens the conversation as an +execution the host keeps alive over the history stream it supplies, under the +installations it captured through the ordinary `executeInstalled` boundary. +Three facts shape it, and each is forced rather than chosen. + +**It is an element.** A provider and its placement coordinator are installed +inside `Execution.document`, and a `DurablePreparation` precedes both, so a +conversation opened there would have no provider to place a session with and no +coordinator to settle the placement. The conversation is therefore a +host-declared identity component written into a synthetic root of core's own: +canonical resolution selects it and hands its implementation this execution's +claimant, so the session is placed under the engine's own identity for that +invocation (*Capability-backed invocation identity*) rather than under anything +the host supplied. The host's conversation id names the history; it names no +session and grants no claimant. + +**Its root is never completed.** A durable root that records its terminal is +finished — a later run reads that terminal and replays its result without +entering the body — and reopening a conversation is the ordinary case. So the +element serves turns and does not return, and the halt that releases the +resource appends nothing, because a root records a terminal only on its own +success or failure. What the history keeps is a partial continuation, which is +what reopening resumes. + +**Each turn is its own canonical sequence.** A turn is one journaled Prompt +under its own durable name, offered in the conversation's sequence exactly as +`` offers one in a document's, and run in a task of its own so that +cancelling the call halts the turn: its append never lands, the position it +would have taken stays free, and the next turn is a new turn rather than that +one resumed. Reopening re-offers the turns the history retained, in their own +order, which replays each from its record and reaches no provider. + +A turn whose outcome the history does not hold is a fact rather than something +to finish, and a conversation holding one is refused before it opens — resuming +it would re-send a prompt the provider may already have accepted, and writing a +terminal for it would make an unfinished turn look complete. Reconciling it +belongs to the host, with the provider's own account of that turn. The history +is bound to its conversation by a durable record written on the first opening, +so one established for another id or another agent refuses rather than +reconnecting the wrong chat, and an established conversation never changes agent +in place. + +What a turn ran under is the provider's own account. A placement is made per +turn, because a placement settles what it asks before it is routed and is good +for exactly one use: the first is fresh and the provider creates the +conversation, and a later one is established, configured and read back — so a +provider that put the conversation under settings nobody asked for refuses that +turn rather than having them recorded. Inspection of advertised model and effort +choices sends no model prompt and journals nothing. A refusal before the turn +started is that call's answer and leaves the conversation usable; only a +durability failure ends it. + ### Typed candidate recovery around Evaluate Plan composes ordinary `` and ``. There is no deadline, no output @@ -5441,6 +5570,9 @@ Status is measured against main. | history fork | creates a new run from one compatible checkpoint and retained Workspace root, under a new immutable definition and normalized props | built on the #368 stack, Deno provider only | | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and every component keeps its ordinary binding and output behavior | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | +| generated-root evaluation | admits one Agent-generated **root** through the trusted-host seam, under the same whole-source preflight, `generated_xmd` admission, pinned identities and refusal classes a fragment gets, and differing over one construct: a top-level `` selects what the root renders, while `` and `` stay unavailable in both. The context is a term of the retained policy and is compared whole, so a root admission never resumes as a fragment or the reverse, and a fragment record states none — which leaves every record written before roots existed readable and resumable as what it is. Rendering reuses the document root's own body partition, error mode and partial-rendering path, and the result states the canonical rendering beside whether the root declared a region at all. The outcome crosses as an Effection `Result`: source core refused and work the root's elements failed at come back as a failure carrying what the root had rendered, while durability, Files-infrastructure, secret-policy and teardown failures keep their own classification | built on the #369 stack; core owns the mechanics and the CLI Sidekick is the first consumer | +| generated-root prefix preview | projects the literal top-level `` text a response prefix already says, as a pure function over the accumulated source: no `Operation`, no file, no name resolution, no component, no expression and no admission, so a prefix that projects has been granted nothing. Source still arriving is a success that says so and source the language does not allow where it is written is a refusal, told apart by the scan's own report of the constructs it began and could not finish — the same walk that decides what a fence, a quoted `>`, an inline code span and a tag-like expression are, rather than a second reading of one syntax. Only settled meaning projects: an expression, an interpolation and a nested component wait, and a passive fence is the literal example it looks like | built on the #369 and #589 stacks; one interpolation rule is shared with admission | +| retained host conversation | opens one agent conversation as an execution the host keeps alive over the history stream it supplies, under the installations it captured through `executeInstalled`. It is an element rather than a preparation, because a provider and its placement coordinator are installed inside `Execution.document`, so the session is placed under the engine's own claimant-issued identity and the host's conversation id names the history alone. Its root is never completed — a recorded terminal would replay instead of reopening — so the element serves turns, does not return, and the halt that releases it appends nothing. Each turn is one journaled Prompt under its own durable name, run in a task of its own, so cancelling the call halts the turn and leaves its append unmade; reopening re-offers the retained turns, which replay from their records and reach no provider. A turn whose outcome the history does not hold refuses the conversation before it opens, and a durable binding refuses a history established for another conversation or another agent. A placement per turn is configured and read back, so settings nobody asked for refuse that turn; inspection sends no model prompt and journals nothing | built on the #828 and #456 stacks; `runPrompt` is shared with `` rather than copied | | `` | canonical core's own protected component, written where program text the document did not author should run. Core claims the name ahead of every host and author tier, so no registration, repository file, bundle member, declared Markdown component, import handler answer or second loaded copy replaces it; a handler may observe or refuse the import, and only canonical execution answers one. Protection settles which implementation runs and grants nothing: a host supplies the *ceiling* as one `ExecutionInstallation.evaluation` profile, captured by value before any installation runs, and an execution accepts one and refuses two. Its schema is closed on `text` and an optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read`. The two input forms are disjoint: `text` states the program, paired content renders it, and an element stating both is refused rather than resolved by precedence. A paired producer renders under the narrowed syntax reference through an execution-owned one-shot projection that bypasses the public `content()`/`tryContent()` chain, and keeps its own operational permission while doing so. Evaluate renders the fragment's ordinary output and invents no observation or result envelope. Fragment-local `as` bindings suppress their component's output normally, and language constructs plus pure components such as `` remain available regardless of effect selection; the fragment explicitly renders any bound values it wants to expose. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage per invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's self-closing `` read — joined in the ordinary run profile by core's self-closing `` and canonical protected `` — the write table of core's paired ``, the bundled Git Plugin's `` — supplied by the host that assembles the profile, since this package states no directory entry of its own — and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. An entry states what is behind a name in one of exactly two ways, and neither carries a function. A *capability* names an operation core supplies the body for: canonical capture reads the host's own operation off once, binds it behind a revocation the execution owns, and closes core's own body over it, so an admitted element reaches those operations and never `API.Files`, `API.Fetch` or `API.Env`. A capability states what it binds wherever the ordinary component does, so an admitted `` is the value component the ordinary one is — sharing its props, return contract, source rules and sanitized failure sentence, refusing a missing `as` and an unusable pattern before the search, and reaching the captured search operation and captured working directory rather than a provider a document arranged. A name admitted for two spellings that would bind different results refuses at capture. A *component answer* names an implementation the ordinary import chain resolves, and states only the identity a provider must have claimed for it; canonical execution resolves that name once — before the root import and before any document code, through the complete ordinary `Component.importComponent` chain and a private terminal that writes no record — and asks the identity owner about the exact final answer in one call. That call answers with the claim *and* core's own copy of what was claimed, taken when the claim was recorded: the check and the thing kept are one result, so nothing downstream reads the chain's object a second time and an answer whose members read differently on each read cannot pass a check with one reading and be sealed with another. The identity is then compared whole, and the copy — not the answer — is sealed behind the same revocation. One name states one identity, however many forms and tables hold it: two entries under one name are the two spellings of one component and share one lookup and one sealed implementation, while a second identity for that name, or a name held as both a capability and a component answer, refuses at capture rather than letting assembly order decide. Provider installation and the right to answer one occurrence are separate: installation receives a registrar that installs import middleware, and every invocation of that middleware receives a fresh request fixed to the exact name, position, provider installation, origin and resolution-window object it was asked under. Only that request may claim, and it states the answer plus key and revision without restating the name. Canonical execution closes the request synchronously when its handler returns, fails or is cancelled; an outer request remains live while it delegates and may claim a replacement after the inner handler returns. Enclosing resolution still closes in its own `finally` on success, fallback, failure or cancellation. A claim therefore requires the execution, request and exact current window to remain active, the request's fixed name to match that window, and that provider installation not to have stated a different answer in the window. Identification takes the expected window explicitly and accepts only a claim recorded for that exact object and name, so a stale request cannot answer a later same-name resolution or retag itself while another name is live. The installation is reusable: one provider may answer several admitted names and repeated resolutions through distinct requests. All of this state is held in ordinary private closures — no Context, shared symbol, public brand or module-global registry — and an unidentified middleware replacement remains a valid ordinary import answer outside fragment evaluation. A capability entry also states the exact version-1 identity strings it succeeds, which is the only thing a released untagged record reconciles against; a component answer states none. A capability-only profile performs no such lookup at all. Resolution happens at capture and never again: a fragment runs the sealed snapshot, a continuation resolves once more in its own capture and reconciles before any effect, and a provider still answering when a fragment resolves its admitted name is answering a generated import, which only canonical execution answers. A trusted layering control may enter one package bootstrap through an inherited layer and an execution-local one — documentation, registrations and the provider together — and the repeat is additive: the outer entry observes and delegates rather than claiming a second identity for an implementation that already states what it is, and a non-identical documentation overlap on one owner and component still refuses at the child collection boundary, before the root import and therefore before the provider is asked at all. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated effects use occurrence-owned identities in the existing ordinary durable sequence: their records persist and replay normally, with no staging or rollback. The generated projection and its structured teardown complete before Evaluate returns and later parent work begins. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | diff --git a/packages/core/host.ts b/packages/core/host.ts index bc38d08b7..966c7574f 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -327,9 +327,35 @@ export type { * author. */ export { installInvocationAgentProvider } from "./src/agent/launch-install.ts"; +/** + * Evaluating generated source as a **root** rather than as a fragment — see + * `src/generated-xmd.ts`. + * + * The same admission, the same durable record and the same pinned resolution, + * in the one context that supplies ``: a root selects what it renders, + * and the host is told whether it selected anything so an empty region and no + * region at all are different answers. `` and `` stay + * unavailable in both — neither has a caller to claim content from nor a value + * body to answer. + * + * Which context an admission was made in is one of its ceilings, so a retained + * root admission does not resume as a fragment or the reverse. Existing + * `evaluateGeneratedXmd()` callers and every fragment record already written + * keep exactly the behavior they had. + * + * It answers with a `Result` because the two kinds of failure are different + * things for a host to be holding. An ordinary one — source core refused, or + * work the root's own elements failed at — is the request's, and comes back as + * `Err` carrying what the root had rendered when it failed. A failure of the + * *run* does not: a journal that stopped describing it, a Files provider that + * is not there, a teardown that failed, each keep the classification they + * already have and leave this operation as themselves. + */ export { evaluateGeneratedXmd, + evaluateGeneratedXmdRoot, GeneratedXmdError, + GeneratedXmdRootError, pinnedComponent, pinnedFetch, pinnedFileDelete, @@ -344,9 +370,64 @@ export type { GeneratedObservation, GeneratedRequest, GeneratedXmdRequest, + GeneratedXmdRootResult, RetainedFragmentIdentity, } from "./src/generated-xmd.ts"; +/** + * What an arriving generated root already says — see + * `src/generated-xmd-preview.ts`. + * + * A host streaming a reply holds a prefix rather than a document, and showing + * the person what it says is a different question from deciding whether it may + * run. This answers the first and nothing else: a pure projection over the + * accumulated prefix, with no `Operation`, because it reads no file, resolves + * no name, invokes no component, evaluates no expression and grants no + * admission. A prefix that projects cleanly has been granted nothing, and the + * provider's own completion and `evaluateGeneratedXmdRoot()` remain the only + * things that decide a root runs. + * + * It is core's because the alternative is a second reading of one syntax. A + * host recognizing `` with a regular expression would have to re-decide + * what a fence, a quoted `>`, an inline code span and a tag-like expression + * are, and the two readings would disagree the first time one of them was + * wrong. + * + * `incomplete` says the prefix ends in syntax still arriving, which is a + * success rather than a refusal; invalid source — a region the language does + * not allow where it is written — answers `Err`. The two are deliberately + * distinct, and `incomplete: false` says only that this prefix is whole, never + * that the provider has finished or that anything may execute. + */ +export { GeneratedXmdPreviewError, previewGeneratedXmdRoot } from "./src/generated-xmd-preview.ts"; +export type { GeneratedXmdRootPreview } from "./src/generated-xmd-preview.ts"; + +/** + * One retained agent conversation a trusted host holds open — see + * `src/agent/conversation.ts`. + * + * The eighth act of infrastructure, and the longest-lived. A document's + * `` and `` are written where the conversation belongs: inside + * an expansion, with an element to name the placement and a journal already + * around them. A host discussing something with an agent has neither, and still + * needs the genuine thing — canonical placement, journaled turns, verified + * configuration, and an identity the provider reattaches to tomorrow. So the + * conversation is an execution the host keeps open, over the history stream it + * supplies, under the installations it captured through `executeInstalled`. + * + * Kept here for the reason the rest of this module is: a conversation decides + * what a journal holds and which provider it reaches, and nothing a document, a + * component or a middleware package reaches by importing `@executablemd/core` + * can open one. + * + * The handle is provided by a resource owned by the caller's scope. Holding it + * past that scope reaches nothing, because the conversation it addressed was + * cancelled — which is also why it can be reopened: a root that recorded its + * terminal would replay instead. + */ +export { AgentConversationError, useAgentConversation } from "./src/agent/conversation.ts"; +export type { AgentConversation, AgentConversationRequest } from "./src/agent/conversation.ts"; + /** * Where a completed Prompt publishes, for a host that retains something beside * it. diff --git a/packages/core/src/agent/conversation.ts b/packages/core/src/agent/conversation.ts new file mode 100644 index 000000000..1841d21ad --- /dev/null +++ b/packages/core/src/agent/conversation.ts @@ -0,0 +1,673 @@ +/** + * One retained agent conversation a trusted host holds open. + * + * A document's `` and `` are written where a conversation + * belongs: inside an expansion, with an element to name the placement and a + * journal already around them. A host that *discusses* something with an agent + * has neither. It has a person typing into a chat, a history file that outlives + * the process, and no document at all — and it still needs the genuine thing: + * canonical Session placement, journaled turns, verified configuration, and an + * identity the provider will reattach to tomorrow. + * + * So the conversation is an execution the host keeps open, and everything about + * its shape follows from three facts. + * + * ## It is an element, because that is where a conversation can be placed + * + * A provider and its placement coordinator are installed inside + * `Execution.document`. A `DurablePreparation` runs *before* that, so a + * conversation opened there could reach neither — no provider to place a + * session with, and no coordinator to settle the placement. The conversation is + * therefore a host-declared identity component, written into a synthetic root + * of core's own: canonical resolution selects it, hands its implementation this + * execution's claimant, and the session is placed under the engine's own + * identity for that invocation rather than under anything the host supplied. + * The host's conversation id names the history; it names no session. + * + * ## The root is never completed, which is why it can be reopened + * + * A durable root that records its terminal is finished: a later run over the + * same journal reads that terminal and replays its result without entering the + * body at all. A conversation must do the opposite — reopening is the ordinary + * case — so the element does not return. It serves turns until the resource + * that owns it is released, and the halt that releases it appends nothing, + * because `durableRun` writes a close only on its own success or failure. What + * the history keeps is a partial continuation, which is what reopening resumes. + * + * ## A turn is a canonical durable operation, run where it can be cancelled + * + * Each turn is one journaled Prompt under its own durable name, offered in the + * conversation's own sequence exactly as `` offers one in a document's. + * It runs in a task of its own so that cancelling the call cancels the turn: + * its append never lands, the position it would have taken stays free, and the + * next turn is a new turn rather than that one resumed. Reopening re-offers the + * turns the history retained, in their own order, which replays each from its + * record and reaches no provider. + * + * A turn whose outcome the history does not hold is a fact rather than + * something to finish, and a conversation holding one is refused before it + * opens. Both alternatives are worse: resuming it would re-send a prompt the + * provider may already have accepted, and writing a terminal for it would make + * an unfinished turn look complete. Reconciling it belongs to the host, with + * the provider's own account of that turn. + * + * ## What the handle can and cannot reach + * + * The handle is provided by a resource owned by the caller's scope, and it is a + * mailbox rather than a door: a call puts a turn on the queue the element is + * reading and waits for that turn's own answer. The work therefore happens + * inside the conversation's durable root, where it belongs, and a handle kept + * past the resource reaches a queue nobody is reading — so it refuses instead. + * One turn runs at a time and an overlapping call refuses rather than queuing, + * because a conversation with two turns in flight has no single next turn. + */ + +import { + createQueue, + Err, + ensure, + Ok, + race, + resource, + scoped, + spawn, + suspend, + withResolvers, +} from "effection"; +import type { Operation, Queue, Result } from "effection"; +import { createDurableOperation } from "@executablemd/durable-streams"; +import type { + DurableEvent, + DurableStream, + Json as DurableJson, +} from "@executablemd/durable-streams"; + +import { executeInstalled } from "../execute.ts"; +import type { ExecutionInstallation } from "../execute.ts"; +import { durabilityFailure } from "../errors.ts"; +import { isJsonObject, parseJson } from "../json.ts"; +import { retainedSource } from "../root-source.ts"; +import { Agent } from "./agent-api.ts"; +import type { + AgentOptions, + AgentOptionsRequest, + PromptOptions, + Session, + SessionConfiguration, +} from "./agent-api.ts"; +import { AgentInternal } from "./internal.ts"; +import { runPrompt } from "./function-components.ts"; +import type { Carried } from "./function-components.ts"; +import { AGENT_PROMPT, persistPrompt, promptFailureFromRecord } from "./journal.ts"; +import { sessionPlacement } from "./session-request.ts"; +import type { ComponentInvocation, IdentityClaimant } from "../invocation-identity.ts"; +import type { Json } from "../types.ts"; + +/** Why a host conversation cannot be opened, continued or inspected. */ +export class AgentConversationError extends Error { + override name = "AgentConversationError"; +} + +/** What a trusted host asks for when it opens one conversation. */ +export interface AgentConversationRequest { + /** The durable stream holding this conversation's canonical history. */ + readonly history: DurableStream; + /** The stable host-owned identity this conversation history belongs to. */ + readonly id: string; + /** The selected agent name for this conversation. */ + readonly agent: string; + /** The trusted installations for this conversation's execution. */ + readonly installations: readonly ExecutionInstallation[]; +} + +/** One open conversation, for as long as the caller's scope holds it. */ +export interface AgentConversation { + /** Send one journaled turn and return its successfully completed reply. */ + prompt(input: string, configuration?: SessionConfiguration): Operation>; + /** Inspect the selected agent's advertised model and effort choices. */ + options(request?: AgentOptionsRequest): Operation>; +} + +/** + * The name the conversation's synthetic root writes, and the root that writes + * it. + * + * Core's own constant, because a journal binds to the exact root it was opened + * with: a caller-supplied source would make reopening depend on that caller + * spelling the same text again. + */ +const CONVERSATION_COMPONENT = "AgentConversationHost"; +const CONVERSATION_ROOT_PATH = "agent/conversation.md"; +const CONVERSATION_ROOT_SOURCE = `<${CONVERSATION_COMPONENT} />\n`; + +/** The durable effect that binds a history to the conversation it belongs to. */ +const CONVERSATION_IDENTITY = "agent_conversation"; + +/** The durable name of the nth turn of a conversation, counting from one. */ +function turnName(order: number): string { + return `turn:${order}`; +} + +/** Which turn a retained durable name is, or nothing when it is not one. */ +function turnOrder(name: string): number | undefined { + const match = /^turn:([1-9][0-9]*)$/.exec(name); + return match === null ? undefined : Number(match[1]); +} + +/** One thing the caller asked the conversation's own execution to do. */ +type Job = PromptJob | OptionsJob; + +interface PromptJob { + readonly kind: "prompt"; + readonly input: string; + readonly configuration?: SessionConfiguration; + readonly answer: Resolvers>; + readonly cancelled: Resolvers; +} + +interface OptionsJob { + readonly kind: "options"; + readonly request?: AgentOptionsRequest; + readonly answer: Resolvers>; + readonly cancelled: Resolvers; +} + +interface Resolvers { + readonly operation: Operation; + resolve(value: T): void; + reject(error: Error): void; +} + +/** What the conversation's own execution shares with the handle beside it. */ +interface Live { + /** False once the resource that owns this conversation has been released. */ + open: boolean; + /** True while one turn or inspection is in flight. */ + busy: boolean; +} + +/** What a history already holds about the conversation it belongs to. */ +interface Retained { + /** The durable names of its turns, in the order the history holds them. */ + readonly turns: readonly string[]; + /** The order the next turn takes, which is past every retained one. */ + readonly next: number; +} + +/** + * Open one agent conversation and keep it open for the caller's scope. + * + * The installations are the host's, captured through the ordinary + * `executeInstalled` boundary, so which provider this conversation reaches and + * what creation policy it runs under are settled before any of its code runs. + * Opening performs no model turn and places no session: the placement is made + * when the first turn asks for one, which is what leaves restoring a chat pane + * provider-free. + */ +export function useAgentConversation( + request: AgentConversationRequest, +): Operation> { + return resource(function* (provide) { + const jobs = createQueue(); + const ready = withResolvers>(); + const live: Live = { open: true, busy: false }; + + // Before the execution exists, so a caller halted while this resource is + // being acquired closes the handle rather than leaving one that looks open. + yield* ensure(() => { + live.open = false; + }); + + // Read before anything opens, because what it finds decides whether this + // conversation may be opened at all — and a refusal that had opened the + // execution would have appended a terminal to the history it refused. + const retained = yield* openable(request); + if (!retained.ok) { + yield* provide(Err(retained.error)); + return; + } + + // Spawned, so the conversation's root is owned by this resource's scope and + // halted — never completed — when that scope ends. + yield* spawn(() => serve(request, retained.value, jobs, ready)); + + const opened = yield* ready.operation; + if (!opened.ok) { + yield* provide(Err(opened.error)); + return; + } + yield* provide(Ok(handle(request, jobs, live))); + }); +} + +/** The caller-facing handle, which reaches the conversation only through its queue. */ +function handle( + request: AgentConversationRequest, + jobs: Queue, + live: Live, +): AgentConversation { + function admit(): Result { + if (!live.open) { + return Err( + new AgentConversationError( + `the conversation "${request.id}" has been closed, so nothing reaches it — a handle ` + + "kept past the scope that opened it names no conversation", + ), + ); + } + if (live.busy) { + return Err( + new AgentConversationError( + `the conversation "${request.id}" is already running a turn, and one conversation ` + + "runs one turn at a time — a second is refused rather than queued behind it", + ), + ); + } + live.busy = true; + return Ok(undefined); + } + + return { + prompt(input: string, configuration?: SessionConfiguration): Operation> { + return scoped(function* () { + const admitted = admit(); + if (!admitted.ok) { + return admitted; + } + const answer = withResolvers>(); + const cancelled = withResolvers(); + // What makes this cancellation rather than an ordinary return: a call + // that was answered has nothing to cancel, and signalling one anyway + // would make every finished turn look abandoned. + let answered = false; + try { + // Registered before the send, so a caller halted between the two + // leaves no turn running that nothing is waiting for. + yield* ensure(() => { + if (!answered) { + cancelled.resolve(); + } + }); + jobs.add({ + kind: "prompt", + input, + ...(configuration === undefined ? {} : { configuration }), + answer, + cancelled, + }); + const settled = yield* answer.operation; + answered = true; + return settled; + } finally { + live.busy = false; + } + }); + }, + options(inspection?: AgentOptionsRequest): Operation> { + return scoped(function* () { + const admitted = admit(); + if (!admitted.ok) { + return admitted; + } + const answer = withResolvers>(); + const cancelled = withResolvers(); + let answered = false; + try { + yield* ensure(() => { + if (!answered) { + cancelled.resolve(); + } + }); + jobs.add({ + kind: "options", + ...(inspection === undefined ? {} : { request: inspection }), + answer, + cancelled, + }); + const settled = yield* answer.operation; + answered = true; + return settled; + } finally { + live.busy = false; + } + }); + }, + }; +} + +/** + * Run the conversation's own durable root, and serve turns inside it forever. + * + * The root is one element, and that element does not return. A document that + * finished would record the root's terminal, and a recorded terminal replays + * instead of reopening. + */ +function* serve( + request: AgentConversationRequest, + retained: Retained, + jobs: Queue, + ready: Resolvers>, +): Operation { + let announced = false; + const announce = (outcome: Result): void => { + if (!announced) { + announced = true; + ready.resolve(outcome); + } + }; + const installation: ExecutionInstallation = { + components: [ + { + name: CONVERSATION_COMPONENT, + origin: "@executablemd/core", + props: { type: "object", properties: {}, additionalProperties: false }, + description: "Hold one agent conversation open for the host that opened it.", + factory: (claim) => + function* (_props, invocation): Operation { + yield* held(request, retained, jobs, announce, claim, invocation); + return ""; + }, + }, + ], + }; + try { + const execution = yield* executeInstalled( + { + ...retainedSource(CONVERSATION_ROOT_PATH, CONVERSATION_ROOT_SOURCE), + stream: request.history, + }, + [...request.installations, installation], + ); + const settled = yield* execution; + // Reached only when the execution ended on its own terms. For a + // conversation that means it ended before the element could serve + // anything: the ordinary ending is a halt, which never arrives here. + announce( + Err( + settled.ok + ? new AgentConversationError( + `the conversation "${request.id}" ended before it was established`, + ) + : settled.error, + ), + ); + } catch (error) { + const failure = error instanceof Error ? error : new Error(String(error)); + if (announced) { + throw failure; + } + announce(Err(failure)); + } +} + +/** + * The conversation itself, inside its own invocation. + * + * The claimant answers for this invocation and no other, so what the session is + * placed under is the engine's own identity — derived from where the element is + * written, not from anything the host supplied. + */ +function* held( + request: AgentConversationRequest, + retained: Retained, + jobs: Queue, + announce: (outcome: Result) => void, + claim: IdentityClaimant, + invocation: ComponentInvocation, +): Operation { + const identity = yield* claim(invocation); + // The conversation's first durable fact, so a later opening compares what + // this history belongs to against what it is being asked for. + const bound = yield* bindConversation(request); + if (!bound.ok) { + announce(Err(bound.error)); + yield* suspend(); + } + // Every turn the history already holds, offered again in its own order so + // this run is aligned with the journal it is continuing. A retained turn + // replays from its own record and reaches no provider. + for (const name of retained.turns) { + yield* replayTurn(name); + } + let order = retained.next; + announce(Ok(undefined)); + while (true) { + const next = yield* jobs.next(); + if (next.done) { + break; + } + const job = next.value; + if (job.kind === "options") { + yield* inspect(request, job); + continue; + } + yield* deliver(request, identity, job, order); + order += 1; + } + // Nothing closes this queue, and an element that returned would let the root + // complete. The halt that ends this resource is what ends this. + yield* suspend(); +} + +/** + * Bind this history to the conversation it belongs to, once. + * + * Written on the first opening and replayed on every later one, so what comes + * back is what the history was established for rather than what this caller + * asked for. A history established for another conversation, or for another + * agent, is refused here: a misrouted stream must not quietly reconnect the + * wrong chat, and an established conversation never changes agent in place. + */ +function* bindConversation(request: AgentConversationRequest): Operation> { + const stated = { id: request.id, agent: request.agent }; + const stored = yield createDurableOperation( + { type: CONVERSATION_IDENTITY, name: "conversation", input: stated }, + // deno-lint-ignore require-yield + function* (): Operation { + return parseJson(stated); + }, + ); + const retained = parseJson(stored); + if (!isJsonObject(retained) || retained.id !== stated.id) { + return Err( + new AgentConversationError( + `this history belongs to another conversation, so opening it as "${request.id}" would ` + + "reconnect the wrong chat", + ), + ); + } + if (retained.agent !== stated.agent) { + return Err( + new AgentConversationError( + "this conversation was established with another agent, and an established conversation " + + "never changes agent in place — an explicit change is a new conversation", + ), + ); + } + return Ok(undefined); +} + +/** + * One turn the history already holds, offered again so replay stays aligned. + * + * Its live body refuses. Reaching it means the record this offer was aligned + * with is not there after all, and asking the provider again for a turn it may + * already have accepted is the one thing a continuation must not do. + */ +function* replayTurn(name: string): Operation { + yield* persistPrompt({ name, input: "" }, function* (): Operation { + throw new AgentConversationError( + `the conversation's retained turn "${name}" has no recorded outcome, so continuing it ` + + "would ask the provider for a turn it may already have accepted", + ); + }); +} + +/** + * Send one turn and answer the caller. + * + * The turn runs in a task of its own so that cancelling the call cancels the + * turn: its append never lands, the position it would have taken stays free, + * and the next turn is a new turn rather than that one resumed. + */ +function deliver( + request: AgentConversationRequest, + identity: string, + job: PromptJob, + order: number, +): Operation { + // Scoped, so the turn cannot outlive the call that asked for it however this + // returns: a turn still running would hold its durable operation open, and + // the next turn would be a second one open on the same sequence. + return scoped(function* () { + const turn = yield* spawn(() => conversationTurn(request, identity, job, order)); + const outcome = yield* race([ + (function* (): Operation<"finished"> { + try { + yield* turn; + } catch (error) { + // The turn answers its own caller, so what can still arrive here is a + // journal that stopped describing this run, which ends it. + if (durabilityFailure(error) !== undefined) { + throw error; + } + } + return "finished"; + })(), + (function* (): Operation<"cancelled"> { + yield* job.cancelled.operation; + return "cancelled"; + })(), + ]); + if (outcome === "finished") { + return; + } + // Giving up on the answer is not cancelling the turn: the task is its own, + // and halting it is what leaves its append unmade — a fact rather than a + // suffix for the next turn to resume. Observed, because a halt that nobody + // waited for is a turn still unwinding while the next one starts. + yield* turn.halt(); + }); +} + +/** + * One canonical turn: place the conversation under what this turn asks, then + * run the journaled prompt. + * + * A placement per turn, because a placement settles what it asks before it is + * routed and is good for exactly one use. The first turn's is fresh and the + * provider creates the conversation; a later turn's is established, and the + * provider configures it and reports what it verified — which is what makes + * `configuration` a fact about this turn rather than a hope. + */ +function conversationTurn( + request: AgentConversationRequest, + identity: string, + job: PromptJob, + order: number, +): Operation { + return (function* (): Operation { + try { + const issuance = sessionPlacement(identity, request.id); + issuance.configure(job.configuration); + let session: Session; + try { + session = yield* Agent.operations.session(issuance.request); + } finally { + issuance.close(); + } + // Resolved before the turn starts, so an unavailable agent fails as + // itself rather than being journaled as a failed turn. + const agent = yield* Agent.operations.agent(request.agent); + const options: PromptOptions = { agent, session }; + const sequence = yield* AgentInternal.operations.nextPromptSequence(); + const carried: Carried = {}; + const record = yield* persistPrompt( + { name: turnName(order), input: job.input }, + () => runPrompt(job.input, options, sequence, false, carried), + () => carried.association, + () => carried.begun, + ); + const failure = promptFailureFromRecord(record); + job.answer.resolve(failure === undefined ? Ok(record.text) : Err(failure)); + } catch (error) { + // A refusal before the turn started — an unavailable agent, an identity + // the provider would not reattach, a conversation it put under settings + // nobody asked for — is this turn's answer rather than the end of the + // conversation: the host may ask again. A durability failure is not, and + // it is the one thing that leaves here as itself. + if (durabilityFailure(error) !== undefined) { + throw error; + } + job.answer.resolve( + Err(error instanceof Error ? error : new AgentConversationError(String(error))), + ); + } + })(); +} + +/** Inspect the selected agent's advertised choices. Journals nothing. */ +function* inspect(request: AgentConversationRequest, job: OptionsJob): Operation { + const answered = yield* race([ + (function* (): Operation<{ readonly done: Result }> { + try { + return { done: Ok(yield* Agent.operations.options(request.agent, job.request)) }; + } catch (error) { + return { done: Err(error instanceof Error ? error : new Error(String(error))) }; + } + })(), + (function* (): Operation<{ readonly cancelled: true }> { + yield* job.cancelled.operation; + return { cancelled: true }; + })(), + ]); + if ("cancelled" in answered) { + return; + } + job.answer.resolve(answered.done); +} + +/** + * What this history already holds, and whether it may be continued at all. + * + * Read before the execution opens, because a refusal that had opened one would + * append a terminal to the history it refused. A turn is recognized by the + * canonical record it committed; one that committed none was interrupted, and + * continuing it is exactly what a continuation must not do. + */ +function* openable(request: AgentConversationRequest): Operation> { + const events = yield* request.history.readAll(); + if (terminated(events)) { + return Err( + new AgentConversationError( + "this history records its own terminal, so it is a finished run rather than a " + + "conversation to continue", + ), + ); + } + const turns: string[] = []; + let highest = 0; + for (const event of events) { + if (event.type !== "yield" || event.description.type !== AGENT_PROMPT) { + continue; + } + const { name } = event.description; + if (event.result.status !== "ok") { + return Err( + new AgentConversationError( + `this conversation's retained turn "${name}" recorded no outcome, so continuing it ` + + "would ask the provider for a turn it may already have accepted. Reconcile that " + + "turn against the provider's own account of it before sending another.", + ), + ); + } + turns.push(name); + highest = Math.max(highest, turnOrder(name) ?? 0); + } + return Ok(Object.freeze({ turns: Object.freeze(turns), next: highest + 1 })); +} + +/** Whether these events hold the conversation root's own terminal. */ +function terminated(events: readonly DurableEvent[]): boolean { + return events.some((event) => event.type === "close" && event.coroutineId === "root"); +} diff --git a/packages/core/src/agent/function-components.ts b/packages/core/src/agent/function-components.ts index 367e52540..2d86bdbed 100644 --- a/packages/core/src/agent/function-components.ts +++ b/packages/core/src/agent/function-components.ts @@ -391,12 +391,22 @@ interface ConsumedTurn { * What this turn hands the publication beside the record: the association a * host may retain, and the handle that host recognised this exact live turn by. */ -interface Carried { +export interface Carried { association?: AgentPromptAssociation; begun?: AgentPromptHandle; } -function* runPrompt( +/** + * Consume one provider turn into the record the journal holds. + * + * Exported for the trusted host conversation in `conversation.ts`, which is a + * canonical turn by the same definition this is: the same publication, the same + * audit, the same account of what the provider said the turn ran in. A second + * implementation of it is how a programmatic caller and `` came to + * disagree about what a turn *is* — so there is one, and the component and the + * host operation both reach it. + */ +export function* runPrompt( text: string, options: PromptOptions, sequence: number, diff --git a/packages/core/src/errors.ts b/packages/core/src/errors.ts index 616994f2d..84fca2dfb 100644 --- a/packages/core/src/errors.ts +++ b/packages/core/src/errors.ts @@ -420,7 +420,7 @@ function isRecoveredContent(error: object): boolean { * names its own reach, so a rule that belongs to one question cannot silence the * other. */ -function firstCause( +export function firstCause( error: unknown, select: (candidate: unknown) => T | undefined, opaque?: OpaqueFailure, diff --git a/packages/core/src/generated-interpolation.ts b/packages/core/src/generated-interpolation.ts new file mode 100644 index 000000000..12a00ba09 --- /dev/null +++ b/packages/core/src/generated-interpolation.ts @@ -0,0 +1,25 @@ +/** + * Whether a piece of text would read something. + * + * The two interpolation passes a text segment goes through are definitive on + * what a reference is, so this asks them rather than guessing: `\{` is + * protected exactly as expansion protects it, and what remains is matched by + * the same shapes `interpolate()` and `interpolateEvalBindings()` consume. + * Braces that neither pass would read — prose, a JSON sample, a CSS rule — are + * left alone. + * + * Its own module because two decisions depend on it and they must agree. + * Admission refuses generated source that reads a binding through + * interpolation; a streaming preview leaves such text out rather than showing a + * person a brace that the finished root will replace. Text one of them called + * literal and the other called a read would be two readings of one rule. + */ + +const ESCAPED_BRACE_PLACEHOLDER = ""; +const FRONTMATTER_REFERENCE = /\{(meta|props)\.[^}]+\}/; +const BINDING_REFERENCE = /\{[a-zA-Z_$][a-zA-Z0-9_$]*(?:\.[a-zA-Z_$][a-zA-Z0-9_$]*)*\}/; + +export function readsBinding(content: string): boolean { + const protectedEscapes = content.replaceAll("\\{", ESCAPED_BRACE_PLACEHOLDER); + return FRONTMATTER_REFERENCE.test(protectedEscapes) || BINDING_REFERENCE.test(protectedEscapes); +} diff --git a/packages/core/src/generated-xmd-preview.ts b/packages/core/src/generated-xmd-preview.ts new file mode 100644 index 000000000..eac7838b0 --- /dev/null +++ b/packages/core/src/generated-xmd-preview.ts @@ -0,0 +1,236 @@ +/** + * What an arriving generated root already says, read from the source so far. + * + * A host streaming an Agent's reply has a prefix rather than a document, and it + * wants to show the person what the reply *says* while the rest arrives. The + * question "what does this prefix safely say?" has one honest answer and + * several tempting wrong ones, so this module is deliberately narrow. + * + * ## It is a projection, not a run + * + * Nothing here reads a file, resolves a name, invokes a component, evaluates an + * expression, grants an admission or writes a history. It needs no `Operation` + * because it performs no suspended work: the same prefix always projects to the + * same answer, whatever is installed around it. Deciding whether this source + * may *run* is `evaluateGeneratedXmdRoot()`'s, and a prefix that projects + * cleanly here has been granted nothing at all. + * + * ## Incomplete is not invalid + * + * `Hello` is the beginning of something; ` + * ok` is a finished thing the language does not allow. The + * first waits, the second refuses, and conflating them would either hide a real + * mistake behind "still arriving" or report a mistake about text that has not + * been written yet. The scanner's own walk is what tells them apart — it is the + * only reader that knows how far it got and why it stopped — and the structure + * rule is `body-structure.ts`'s, the same one an authored root is held to. + * + * ## The safe reading stops where the text stops being whole + * + * Everything before the first construct the scan could not finish is text the + * scan committed, and it reads the same whether more arrives or not. From there + * on it does not: an `` written inside an `` that has not closed + * looks like a top-level region until the `` arrives and makes it a + * misplaced one. So the projection ends there — with one exception, which is the + * case this whole module exists for: when the unfinished construct *is* a + * top-level `` whose opening tag completed, the literal text already + * inside it is exactly what the person should be reading. + * + * ## Only text whose meaning is already settled + * + * A region's literal text projects. An expression, an interpolation and a + * nested component do not — their meaning is what evaluation decides, and a + * preview that guessed would show the person something the finished root never + * says. What is left out waits rather than being approximated. + */ + +import { Err, Ok } from "effection"; +import type { Result } from "effection"; + +import { readsBinding } from "./generated-interpolation.ts"; +import { outputPropsViolation, validateBodyStructure } from "./body-structure.ts"; +import { scanSegments } from "./scanner.ts"; +import type { UnfinishedConstruct } from "./scanner.ts"; +import type { ComponentElement, Segment } from "./types.ts"; + +/** What a source prefix safely says, and whether it has finished saying it. */ +export interface GeneratedXmdRootPreview { + /** The literal root Output text safely recognized in this source prefix. */ + readonly output: string; + /** Whether this source prefix ends in syntax that needs more input. */ + readonly incomplete: boolean; +} + +/** Source a generated root cannot be read from, however much more arrives. */ +export class GeneratedXmdPreviewError extends Error { + override name = "GeneratedXmdPreviewError"; +} + +/** + * The literal root Output text this prefix safely says. + * + * `source` is the whole prefix received so far rather than the latest chunk: + * the answer is a complete projection of what has arrived, so a caller replaces + * what it was showing rather than appending to it. Two identical prefixes + * always project identically, however the transport happened to split them. + */ +export function previewGeneratedXmdRoot(source: string): Result { + const unfinished: UnfinishedConstruct[] = []; + scanSegments(source, undefined, undefined, undefined, unfinished); + const pending = earliest(unfinished); + // Everything the scan committed whole. Re-read from the exact prefix that + // ends where wholeness does, so nothing half-delivered takes part in the + // structure rule below. + const settled = pending === undefined ? source : source.slice(0, pending.start); + const segments = scanSegments(settled); + + const violation = validateBodyStructure(segments, undefined); + if (violation !== undefined) { + return Err(new GeneratedXmdPreviewError(violation.message)); + } + const declared = regions(segments); + // `validateBodyStructure` reads where regions are written; what one may carry + // is `outputPropsViolation`'s, which expansion asks when it builds the body. + // Both rules belong to a root, so a preview asks both. + for (const region of declared) { + const props = outputPropsViolation(region.element); + if (props !== undefined) { + return Err(new GeneratedXmdPreviewError(props)); + } + } + const tail = unfinishedRegion(source, pending); + if (tail !== undefined && tail.violation !== undefined) { + return Err(new GeneratedXmdPreviewError(tail.violation)); + } + + const projected = declared.map((region) => literal(region.element.children)).join(""); + return Ok( + Object.freeze({ + output: tail === undefined ? projected : projected + tail.output, + incomplete: pending !== undefined, + }), + ); +} + +/** + * The construct the scan gave up on earliest. + * + * Earliest rather than first reported, because the walk reports a nested + * construct before the one enclosing it and recovery can report the same text + * twice. The construct that begins earliest is the one everything after it is + * inside, so it is the one the safe reading ends at. + */ +function earliest(found: readonly UnfinishedConstruct[]): UnfinishedConstruct | undefined { + let chosen: UnfinishedConstruct | undefined; + for (const candidate of found) { + if (chosen === undefined || candidate.start < chosen.start) { + chosen = candidate; + } + } + return chosen; +} + +/** What the prefix says inside a region it has not finished delivering. */ +interface UnfinishedRegion { + readonly output: string; + /** Why this region is refused rather than awaited, when it is. */ + readonly violation?: string; +} + +/** + * The literal text inside an `` whose closing tag has not arrived. + * + * Only for a region at the prefix's own top level, which is the only place the + * earliest unfinished construct can be: everything before it the scan committed + * whole, so nothing recognized encloses it, and a construct that *did* enclose + * it would have begun earlier and been chosen instead. + * + * A region whose opening tag carried an attribute is refused here rather than + * awaited, because `` accepts none and no further text changes that — + * the same rule, from the same function, that an authored body is held to. + */ +function unfinishedRegion( + source: string, + pending: UnfinishedConstruct | undefined, +): UnfinishedRegion | undefined { + if (pending?.name !== "Output" || pending.contentStart === undefined) { + return undefined; + } + if (pending.attributed === true) { + return { output: "", violation: OUTPUT_PROPS }; + } + // The same walk again over the content delivered so far, so what counts as a + // nested tag, a fence or an inline code span is decided once. + const content = source.slice(pending.contentStart); + return { output: literal(scanSegments(settledTail(content))) }; +} + +/** + * The content of an unfinished region, up to where its delimiters stop being + * settled. + * + * A conservative cut rather than a reading: a `<` with no `>` after it is the + * first character of a delimiter whose rest has not arrived — this element's + * own closing tag, or a nested element — and the scanner's answer about it + * changes with the next character. `` arrive, and `` does, so projecting + * either would show a person a delimiter instead of their text and then take + * it away again. + * + * It only ever shortens what the scan already committed. Nothing here decides + * what a construct *is*; that stays the walk's, which has already said this + * region is still arriving. + */ +function settledTail(content: string): string { + const opened = content.lastIndexOf("<"); + if (opened === -1 || content.indexOf(">", opened) !== -1) { + return content; + } + return content.slice(0, opened); +} + +/** + * What `` carrying a prop says. + * + * Read from the canonical rule rather than written again here: the sentence an + * author sees for a region with props is the sentence a preview reports. + */ +const OUTPUT_PROPS: string = + outputPropsViolation({ + type: "component", + name: "Output", + props: { stated: true }, + expressions: {}, + children: [], + selfClosing: false, + }) ?? " accepts no props."; + +/** Every top-level `` region, in source order. */ +function regions(segments: readonly Segment[]): { element: ComponentElement }[] { + const found: { element: ComponentElement }[] = []; + for (const segment of segments) { + if (segment.type === "component" && segment.name === "Output") { + found.push({ element: segment }); + } + } + return found; +} + +/** + * The text of these segments whose meaning is already settled. + * + * A text segment that reads nothing is what it says. Everything else — an + * expression, an interpolation, a nested component, an executable block — means + * whatever evaluation decides, so it is left out and waits. A passive fence is + * ordinary text to the scanner and projects as the literal example it is, which + * is why a tag written inside one creates no region. + */ +function literal(segments: readonly Segment[]): string { + let text = ""; + for (const segment of segments) { + if (segment.type === "text" && !readsBinding(segment.content)) { + text += segment.content; + } + } + return text; +} diff --git a/packages/core/src/generated-xmd.ts b/packages/core/src/generated-xmd.ts index 59de297dc..3bcb3fab7 100644 --- a/packages/core/src/generated-xmd.ts +++ b/packages/core/src/generated-xmd.ts @@ -108,17 +108,26 @@ import { createDurableOperation } from "@executablemd/durable-streams"; import type { Json as DurableJson } from "@executablemd/durable-streams"; -import { scoped } from "effection"; -import type { Operation } from "effection"; +import { Err, Ok, scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { bodyHasOutput, outputPropsViolation } from "./body-structure.ts"; import { ExecutionDeclarationCatalog } from "./execution-declarations.ts"; import { Component } from "./component-api.ts"; -import { ErrorMode } from "./errors.ts"; +import { + documentationFailure, + durabilityFailure, + ErrorMode, + filesFatalFailure, + firstCause, +} from "./errors.ts"; +import { InvocationTeardownError } from "./invocation.ts"; +import { SecretDetectedError, SecretScannerError } from "./secrets/findings.ts"; import { CanonicalImports, retain } from "./components/component-resolution.ts"; import type { ComponentResolution, ImportedDefinition } from "./components/component-resolution.ts"; import { isComponentName } from "./components/registration.ts"; import { CORE_ORIGIN, CORE_REGISTRY } from "./components/registry.ts"; -import { createBlockCounter, expandSegmentsWithin } from "./expand.ts"; +import { createBlockCounter, expandBody, expandSegmentsWithin } from "./expand.ts"; import { extendPath } from "./expansion.ts"; import { prepareFetchRequest, requestRecord } from "./fetch-request.ts"; import { timeoutFetch } from "@executablemd/runtime"; @@ -126,6 +135,7 @@ import type { FetchRequest } from "./fetch-request.ts"; import { isJsonObject, parseJson } from "./json.ts"; import { markGeneratedRequestRefusal } from "./generated-request-refusal.ts"; import { GeneratedDataExpressions, validateDataExpression } from "./generated-expressions.ts"; +import { readsBinding } from "./generated-interpolation.ts"; import { capturedBinding } from "./invocation-rules.ts"; import { renderSegments } from "./render.ts"; import { scanSegments } from "./scanner.ts"; @@ -170,6 +180,53 @@ export class GeneratedXmdError extends Error { override name = "GeneratedXmdError"; } +/** + * What the generated source is being admitted *as*. + * + * A fragment and a root are written in the same language and admitted by the + * same walk, and they differ over one construct: a root selects what it renders + * with a top-level ``, and a fragment has no output region at all. That + * single difference is a ceiling rather than a detail — a run admitted as one + * and resumed as the other would expand source nobody admitted — so it travels + * with the policy and is compared like every other term. + * + * `` and `` are unavailable in both: neither has caller content + * to claim nor a value body to answer. + */ +type GeneratedContext = "fragment" | "root"; + +/** A root admission's own name for itself, absent from a fragment's record. */ +const ROOT_CONTEXT = "root"; + +/** What a generated root rendered, and whether it chose what to render. */ +export interface GeneratedXmdRootResult { + /** The root's canonical rendered text after evaluation completes. */ + readonly output: string; + /** Whether this root declares an explicit Output region. */ + readonly hasOutput: boolean; +} + +/** + * An ordinary failure of one generated root, and what it had rendered. + * + * `partial` is present only for a root that began evaluating: an admission + * refusal performed nothing, so there is nothing truthful to report about what + * it rendered. The rendering is what the root actually produced before the + * failure reached it — no further source is evaluated to complete a region or + * summarize one. + */ +export class GeneratedXmdRootError extends Error { + override name = "GeneratedXmdRootError"; + readonly partial?: GeneratedXmdRootResult; + + constructor(message: string, partial?: GeneratedXmdRootResult) { + super(message); + if (partial !== undefined) { + this.partial = partial; + } + } +} + /** The construct classes a fragment can be refused for. */ type Construct = | "block" @@ -856,6 +913,15 @@ interface RetainedInvocation { */ interface Policy { readonly allow: readonly GeneratedEffectClass[]; + /** + * Whether this source was admitted as a fragment or as a root. + * + * A ceiling, not a rendering preference: the two admit different constructs, + * so a retained admission resumes only in the context it was made in. Written + * to the record only for a root, which is what leaves every fragment record + * this build and the one before it wrote byte-identical and readable. + */ + readonly context: GeneratedContext; /** * The Workspace basis, or nothing for a host that evaluates against none. * @@ -1237,25 +1303,6 @@ function admitted(entries: readonly Entry[]): Map { return table; } -/** - * Whether this text would read anything. - * - * The two interpolation passes a text segment goes through are definitive - * on what a reference is, so this asks them rather than guessing: `\{` is - * protected exactly as expansion protects it, and what remains is matched by - * the same shapes `interpolate()` and `interpolateEvalBindings()` consume. - * Braces that neither pass would read — prose, a JSON sample, a CSS rule — are - * left alone. - */ -const ESCAPED_BRACE_PLACEHOLDER = "\uE000"; -const FRONTMATTER_REFERENCE = /\{(meta|props)\.[^}]+\}/; -const BINDING_REFERENCE = /\{[a-zA-Z_$][a-zA-Z0-9_$]*(?:\.[a-zA-Z_$][a-zA-Z0-9_$]*)*\}/; - -function reads(content: string): boolean { - const protectedEscapes = content.replaceAll("\\{", ESCAPED_BRACE_PLACEHOLDER); - return FRONTMATTER_REFERENCE.test(protectedEscapes) || BINDING_REFERENCE.test(protectedEscapes); -} - /** Two normalized requests describing the same read. */ function sameRequest(one: FetchRequest, other: FetchRequest): boolean { return JSON.stringify(requestRecord(one)) === JSON.stringify(requestRecord(other)); @@ -1297,6 +1344,7 @@ function currentPolicy( allow: readonly GeneratedEffectClass[], entries: readonly Entry[], ceilings: ReadonlyMap, + context: GeneratedContext, ): Policy { const requests: FetchRequest[] = []; const allowed: RetainedEntry[] = []; @@ -1315,6 +1363,7 @@ function currentPolicy( const workspace = workspaceBasis(request); return { allow: [...allow], + context, ...(workspace === undefined ? {} : { workspace }), allowed, requests, @@ -1375,6 +1424,12 @@ function policyRecord(policy: Policy): JsonObject { return { version: RECORD_VERSION, allow: [...policy.allow], + // Written only for a root. A fragment is what every record before this + // build described, so stating it would rewrite history that is already + // correct — and an older build reading a *root* record finds a member it + // does not know the rules of and refuses the continuation, which is the + // safe direction. + ...(policy.context === ROOT_CONTEXT ? { context: ROOT_CONTEXT } : {}), ...(policy.workspace === undefined ? {} : { @@ -1465,7 +1520,7 @@ function readPolicy(value: Json): Policy | undefined { if (value.version !== RECORD_VERSION) { return undefined; } - if (!exactly(value, ["version", "allow", "allowed", "requests"], ["workspace"])) { + if (!exactly(value, ["version", "allow", "allowed", "requests"], ["workspace", "context"])) { return undefined; } const workspace = Object.hasOwn(value, "workspace") @@ -1474,7 +1529,11 @@ function readPolicy(value: Json): Policy | undefined { if (workspace === MALFORMED) { return undefined; } - return readPolicyTerms(value, workspace, 2); + const context = readContext(value); + if (context === undefined) { + return undefined; + } + return readPolicyTerms(value, workspace, 2, context); } if (!exactly(value, ["allow", "roots", "selectedRoot", "allowed", "requests"])) { return undefined; @@ -1485,7 +1544,25 @@ function readPolicy(value: Json): Policy | undefined { if (workspace === MALFORMED || workspace === undefined) { return undefined; } - return readPolicyTerms(value, workspace, 1); + // Version 1 predates generated roots entirely, so it can only have been a + // fragment. The absence is the answer rather than a default to fall back on. + return readPolicyTerms(value, workspace, 1, "fragment"); +} + +/** + * Which context a version-2 record was admitted in. + * + * Absent is a fragment, because that is what every record written before roots + * existed described and what this build still writes for one. Present must be + * the one spelling a root record carries: a record naming some third context + * describes no shape this build has, and reading it as whichever it resembles + * would resume a root under a fragment's admission or the reverse. + */ +function readContext(value: JsonObject): GeneratedContext | undefined { + if (!Object.hasOwn(value, "context")) { + return "fragment"; + } + return value.context === ROOT_CONTEXT ? ROOT_CONTEXT : undefined; } /** The terms both versions share, read closed. */ @@ -1493,6 +1570,7 @@ function readPolicyTerms( value: JsonObject, workspace: { roots: readonly string[]; selectedRoot: string } | undefined, version: 1 | 2, + context: GeneratedContext, ): Policy | undefined { const classes = readClasses(value.allow); const identities = readAllowed(value.allowed, version); @@ -1502,6 +1580,7 @@ function readPolicyTerms( } return { allow: classes, + context, ...(workspace === undefined ? {} : { workspace }), allowed: identities, requests, @@ -1724,6 +1803,12 @@ function readNamed(value: Json, version: 1 | 2): RetainedInvocation[] | undefine * the *current* policy would wave through. */ function policyHolds(retained: Policy, current: Policy): boolean { + // Before anything else, because it decides which constructs the admission was + // about. A root resumed as a fragment would expand a top-level `` + // nothing admitted, and a fragment resumed as a root would be granted one. + if (retained.context !== current.context) { + return false; + } if (!workspaceHolds(retained.workspace, current.workspace)) { return false; } @@ -1802,7 +1887,26 @@ interface Preflight { } /** - * Walk the complete fragment, admitting it or refusing it whole. + * What the whole walk is decided against. + * + * One value rather than four threaded parameters, because every one of these is + * invariant for the walk: the admitted table, the normalized request ceilings, + * the context the source is being admitted in, and the list the walk records + * what it found into. Bundling them is what lets the *one* thing that differs + * between a fragment and a root — which constructs the language supplies + * context for — be read from the same place the rest of the decision is, + * instead of copying the policy walk once per context. + */ +interface Admission { + readonly table: ReadonlyMap; + readonly ceilings: ReadonlyMap; + readonly context: GeneratedContext; + /** What the source named, in the order it named it. Accumulated by the walk. */ + readonly named: Planned[]; +} + +/** + * Walk the complete source, admitting it or refusing it whole. * * Nothing here performs an effect. The requests are normalized — the candidate's * and the host's alike — because deciding whether two requests are the same one @@ -1814,6 +1918,7 @@ function* preflight( source: string, table: ReadonlyMap, ceilings: ReadonlyMap, + context: GeneratedContext, ): Operation { // Read once, here, so a contextual default that refuses fails as itself // rather than as a statement about the fragment. Every candidate request @@ -1830,7 +1935,7 @@ function* preflight( const segments = scanSegments(source, { generatedSource: id, baseOffset: 0, baseLine: 1 }); // A generated fragment starts with no bindings: it imports none from the // document that admitted it, and exports none back to it. - yield* walk(segments, table, ceilings, named, new Set()); + yield* walk({ table, ceilings, context, named }, segments, new Set(), TOP_LEVEL); return { segments, named }; } @@ -1865,17 +1970,16 @@ function* admitCandidateRequest(props: Record): Operation, - ceilings: ReadonlyMap, - named: Planned[], scope: Set, lexical: Lexical = ROOT_LEXICAL, ): Operation { + const { table, ceilings, named } = admission; for (const segment of segments) { switch (segment.type) { case "text": { - if (reads(segment.content)) { + if (readsBinding(segment.content)) { throw new Refusal("interpolation"); } break; @@ -1890,7 +1994,7 @@ function* walk( // table lookup would be refused as a component the host withheld, which // is a different and misleading thing to tell a candidate. if (RESERVED_STRUCTURAL.has(segment.name)) { - yield* structural(segment, table, ceilings, named, scope, lexical); + yield* structural(admission, segment, scope, lexical); break; } const entries = table.get(segment.name); @@ -1942,7 +2046,7 @@ function* walk( // body rather than a transparent region, so the lexical facts reset: // a `` written in a component's own body cannot break a loop // that encloses the invocation. - yield* walk(segment.children, table, ceilings, named, new Set(scope), ROOT_LEXICAL); + yield* walk(admission, segment.children, new Set(scope), ROOT_LEXICAL); // After the element, because a component's result does not exist until // it has run: `` beside `` is ordered, // and `` names nothing. @@ -1971,17 +2075,37 @@ function* walk( */ interface Lexical { readonly insideLoop: boolean; + /** + * Whether the next segment is written as a direct child of the source's own + * top level. + * + * What decides whether a root's `` is a declaration or a misplaced + * element, and the same lexical question `validateOutputPlacement()` answers + * for an authored body: only a direct top-level child declares a region. It + * is false everywhere inside anything — a construct's branches, a loop body, a + * component's children — because none of those is the top level, whatever runs + * there. + */ + readonly topLevel: boolean; } /** - * What encloses the fragment's own top level. + * What encloses a region written inside something. + * + * Neither a component body nor a value body: there is no caller content for + * `` to claim and no value body for `` to answer. It is not + * inside a loop, and it is not the top level, so a `` written here + * declares nothing even in a root. + */ +const ROOT_LEXICAL: Lexical = Object.freeze({ insideLoop: false, topLevel: false }); + +/** + * What encloses the source's own top level. * - * A generated fragment is not a component body and not a value body: there is - * no caller content for `` to claim, no output region for `` - * to select, and no value body for `` to answer. It is also not inside - * a loop. + * The one place a root may declare an output region. Everything else about it + * is what `ROOT_LEXICAL` says. */ -const ROOT_LEXICAL: Lexical = Object.freeze({ insideLoop: false }); +const TOP_LEVEL: Lexical = Object.freeze({ insideLoop: false, topLevel: true }); /** A typed empty table, so an absent one reads as no expressions rather than as `{}`. */ const NO_EXPRESSIONS: Record = {}; @@ -2013,14 +2137,20 @@ function stated(segment: ComponentElement, scope: ReadonlySet): void { const CONSUMED_BY_PARENT: ReadonlySet = new Set(["Else", "Case", "Answer", "Spawn"]); /** - * The structural names the generated root supplies no context for. - * - * Each of these is ordinary language wherever its context exists, and a - * fragment is simply not that place. Refused as a structural mistake rather - * than as a withheld permission, because nothing about the host's tables would - * make one of them work. + * The structural names generated source supplies no context for, in either + * context. + * + * Each of these is ordinary language wherever its context exists, and generated + * source is simply not that place: there is no caller to claim content from and + * no value body to answer. Refused as a structural mistake rather than as a + * withheld permission, because nothing about the host's tables would make one of + * them work. + * + * `` is not here. A fragment supplies no context for it either, but a + * root does — at its own top level and nowhere else — so which it is depends on + * the admission's context rather than on the name alone. */ -const NO_GENERATED_CONTEXT: ReadonlySet = new Set(["Content", "Output", "Return"]); +const NO_GENERATED_CONTEXT: ReadonlySet = new Set(["Content", "Return"]); /** * One structural construct, held to the ordinary source rules. @@ -2038,10 +2168,8 @@ const NO_GENERATED_CONTEXT: ReadonlySet = new Set(["Content", "Output", * the source and the admission of every possible path without fabricating one. */ function* structural( + admission: Admission, segment: ComponentElement, - table: ReadonlyMap, - ceilings: ReadonlyMap, - named: Planned[], scope: Set, lexical: Lexical, ): Operation { @@ -2051,6 +2179,29 @@ function* structural( if (CONSUMED_BY_PARENT.has(name) || NO_GENERATED_CONTEXT.has(name)) { throw new Refusal("structure"); } + // Whatever this construct encloses is not the top level, whichever construct + // it is. Settled once here so no branch below has to remember it. + const inner: Lexical = lexical.topLevel ? { ...lexical, topLevel: false } : lexical; + + if (name === "Output") { + // A root's own output declaration, and only there: a fragment renders what + // it renders, and an `` written anywhere below the top level + // declares nothing in an authored body either + // (`body-structure.ts`). The props rule is that body's as well, read from + // the same function rather than restated, because `` accepting a + // prop here and refusing one there would be two contracts. + if (admission.context !== ROOT_CONTEXT || !lexical.topLevel) { + throw new Refusal("structure"); + } + if (outputPropsViolation(segment) !== undefined) { + throw new Refusal("structure"); + } + // Transparent, because expansion expands the regions and the documentation + // around them against one live environment in source order — so a binding a + // region produces is visible after it, and preflight says the same. + yield* transparent(admission, segment.children, scope, inner); + return; + } // The construct's own props, over the bindings in effect where it was // written. @@ -2072,12 +2223,10 @@ function* structural( // incoming bindings. A prohibited component in the arm this run never // enters still refuses the whole fragment. yield* alternatives( + admission, [{ body: structure.whenTrue }, { body: structure.whenFalse }], - table, - ceilings, - named, scope, - lexical, + inner, ); return; } @@ -2091,12 +2240,10 @@ function* structural( // branch's own expression and is validated whether or not the comparison // would ever reach it. yield* alternatives( + admission, branches.map((branch) => ({ matcher: branch.element, body: branch.element.children })), - table, - ceilings, - named, scope, - lexical, + inner, ); return; } @@ -2105,8 +2252,8 @@ function* structural( refuseStructure(loopViolations(segment)); // Inside for the body alone: a `` after the loop is as stray as one // written where no loop ever was. - yield* transparent(segment.children, table, ceilings, named, scope, { - ...lexical, + yield* transparent(admission, segment.children, scope, { + ...inner, insideLoop: true, }); return; @@ -2123,14 +2270,14 @@ function* structural( // above already refused a `` or `` that would cross it. const incoming: ReadonlySet = new Set(scope); for (const spawn of structure.spawns) { - yield* walk(spawn.element.children, table, ceilings, named, new Set(incoming), ROOT_LEXICAL); + yield* walk(admission, spawn.element.children, new Set(incoming), ROOT_LEXICAL); } return; } if (name === "PrintErrors") { refuseStructure(printErrorsViolations(segment)); - yield* transparent(segment.children, table, ceilings, named, scope, lexical); + yield* transparent(admission, segment.children, scope, inner); return; } @@ -2145,20 +2292,20 @@ function* structural( if (child.type === "component" && child.name === "Answer") { refuseStructure(answerViolations(child)); stated(child, scope); - yield* transparent(child.children, table, ceilings, named, scope, lexical); + yield* transparent(admission, child.children, scope, inner); continue; } if (!isBlankText(child)) { body.push(child); } } - yield* transparent(body, table, ceilings, named, scope, lexical); + yield* transparent(admission, body, scope, inner); return; } if (name === "Let") { refuseStructure(letViolations(segment)); - yield* transparent(segment.children, table, ceilings, named, scope, lexical); + yield* transparent(admission, segment.children, scope, inner); // After its own body, like a component's `as`: a `` cannot name the // binding it is producing. const bound = letBindingName(segment); @@ -2182,7 +2329,7 @@ function* structural( // caller is the one capture it was asked for. const body = new Set(scope); body.add(item); - yield* walk(segment.children, table, ceilings, named, body, lexical); + yield* walk(admission, segment.children, body, inner); const capture = eachCaptureBinding(segment); if (capture !== undefined) { scope.add(capture); @@ -2220,10 +2367,8 @@ interface Alternative { * preflight. */ function* alternatives( + admission: Admission, branches: readonly Alternative[], - table: ReadonlyMap, - ceilings: ReadonlyMap, - named: Planned[], scope: Set, lexical: Lexical, ): Operation { @@ -2234,7 +2379,7 @@ function* alternatives( stated(branch.matcher, incoming); } const region = new Set(incoming); - yield* walk(branch.body, table, ceilings, named, region, lexical); + yield* walk(admission, branch.body, region, lexical); for (const bound of region) { if (!incoming.has(bound)) { outgoing.add(bound); @@ -2257,15 +2402,13 @@ function* alternatives( * would — when the value is needed. */ function* transparent( + admission: Admission, segments: readonly Segment[], - table: ReadonlyMap, - ceilings: ReadonlyMap, - named: Planned[], scope: Set, lexical: Lexical, ): Operation { const region = new Set(scope); - yield* walk(segments, table, ceilings, named, region, lexical); + yield* walk(admission, segments, region, lexical); for (const bound of region) { scope.add(bound); } @@ -2312,7 +2455,7 @@ function* admitSource( policy: Policy, ): Operation { try { - const { named } = yield* preflight(id, source, table, ceilings); + const { named } = yield* preflight(id, source, table, ceilings, policy.context); return parseJson({ version: RECORD_VERSION, decision: "admitted", @@ -2421,31 +2564,45 @@ function isConstruct(value: string): value is Construct { } /** - * Expand the admitted fragment through ordinary durable XMD effects. + * Expand the admitted source through ordinary durable XMD effects. * * The import provider installs at `min` on this scope alone, so it answers - * ahead of the execution's own terminal and is gone when the fragment is. It + * ahead of the execution's own terminal and is gone when the source is. It * reaches no component search path: what a name resolves to is the pinned * identity or nothing. * - * Errors here fail rather than print. A printed error is something an author - * reads and acts on; a generated fragment has no author, and a refusal that + * A fragment's errors fail rather than print. A printed error is something an + * author reads and acts on; generated source has no author, and a refusal that * rendered as text would leave every element after it still running — which is - * the partial effect the whole-fragment preflight exists to prevent. + * the partial effect the whole-source preflight exists to prevent. + * + * A root installs the mode every text root installs instead, because it *is* + * one: its regions and its documentation both fail the run on an undecided + * error, and the failure is what carries the rendering back rather than text in + * the middle of it. */ function expand( id: string, segments: Segment[], named: readonly Planned[], + context: GeneratedContext, componentRouting: ComponentRouting | undefined, syntax: SyntaxReference | undefined, + /** + * Where a root's rendering accumulates. + * + * Allocated by the caller so a failure partway still leaves it holding what + * the root produced before it — the same reason the document root allocates + * its own owner outside the expansion. + */ + owner: Segment[] | undefined, ): Operation { return scoped(function* () { - yield* ErrorMode.set("throw"); + yield* ErrorMode.set(context === ROOT_CONTEXT ? "output" : "throw"); // Every expression this expansion resolves — an ordinary prop and a // declared capture alike — reads the declarative grammar rather than the // trusted-document evaluator. Set on this scope, so it ends with the - // fragment and reaches nothing the document expands afterwards. + // source and reaches nothing the document expands afterwards. yield* GeneratedDataExpressions.set(true); const resolution = new GeneratedComponentResolution(named, componentRouting); yield* Component.around( @@ -2457,40 +2614,70 @@ function expand( }, { at: "min" }, ); + // No enclosing identity table: protected invocation domains travel only + // through the narrowed route. Import and form selection belong to this + // source, while the reference preserves the admitting site's documentation. + const environment = { + componentResolution: resolution, + forms: resolution.forms, + // Generated source writes the admitted composition table and nothing + // else, so it declares none. The empty catalog states that explicitly + // rather than inheriting the host's: a generated execution environment + // admits no installed structural syntax. + declarations: new ExecutionDeclarationCatalog([], []), + invokeGeneratedComponent: ( + fn: unknown, + invocation: ComponentInvocation, + body: Operation, + ) => resolution.invoke(fn, invocation, body), + ...(componentRouting === undefined ? {} : { componentRouting }), + ...(syntax === undefined ? {} : { syntax }), + }; + const path = extendPath("", { f: "gen", id }); + const counter = createBlockCounter(); + if (context === ROOT_CONTEXT && bodyHasOutput(segments)) { + // A root declaring top-level `` selects what renders, and + // selecting needs the whole body: `expandBody` executes the documentation + // around the regions and emits the regions, exactly as the document root + // does, so one rule decides what a root renders. + yield* expandBody( + segments, + [], + {}, + {}, + new Set(), + counter, + undefined, + undefined, + owner, + path, + undefined, + environment, + // No execution terminal: this source's imports are answered by the + // narrowed provider installed above, and it inherits nothing of the + // document execution that admitted it. + undefined, + // Generated source is the engine's own text, so it owns no value body + // and a written into it satisfies no declaration. + undefined, + ); + return renderSegments(owner ?? []); + } const expanded = yield* expandSegmentsWithin( segments, {}, {}, new Set(), - createBlockCounter(), - undefined, - extendPath("", { f: "gen", id }), + counter, + owner, + path, 0, undefined, - // No enclosing identity table: protected invocation domains travel only - // through the narrowed route. Import and form selection belong to this - // fragment, while the reference preserves the admitting site's documentation. - { - componentResolution: resolution, - forms: resolution.forms, - // A fragment writes the admitted composition table and nothing else, so - // it declares none. The empty catalog states that explicitly rather than - // inheriting the host's: a generated fragment's execution environment - // admits no installed structural syntax. - declarations: new ExecutionDeclarationCatalog([], []), - invokeGeneratedComponent: (fn, invocation, body) => resolution.invoke(fn, invocation, body), - ...(componentRouting === undefined ? {} : { componentRouting }), - ...(syntax === undefined ? {} : { syntax }), - }, - // No execution terminal: this fragment's imports are answered by the - // narrowed provider installed above, and it inherits nothing of the - // document execution that admitted it. + environment, undefined, - // A generated fragment is the engine's own text, so it owns no value body - // and a written into it satisfies no declaration. undefined, ); - return renderSegments(expanded); + return renderSegments(owner ?? expanded); }); } @@ -2512,16 +2699,137 @@ export function evaluateGeneratedXmd(request: GeneratedXmdRequest): Operation { + return evaluateGenerated(request, "fragment", componentRouting, syntax, undefined); +} + +/** + * Admit one generated root and perform what it asks for. + * + * The same admission, the same durable record and the same pinned resolution a + * fragment gets, in the one context that supplies ``. What it answers + * with is the root's canonical rendering and whether the root chose it, so a + * host can tell an empty region from no region at all without reading the + * source itself. + * + * An ordinary failure of the request — source core refused, or work the root's + * own elements failed at — comes back as `Err` carrying a + * {@link GeneratedXmdRootError}, because deciding what to do about generated + * source that did not work is the caller's. A failure of *this run* does not: + * a journal that no longer describes the run, a Files provider that is not + * there and a teardown that failed each keep the classification they already + * have, so a host cannot mistake one for a response worth correcting. + */ +export function* evaluateGeneratedXmdRoot( + request: GeneratedXmdRequest, +): Operation> { + // Allocated out here, so a failure partway leaves this frame holding exactly + // what the root rendered before it. The flag is read from the source core + // admitted rather than from the caller's copy, and it is the same reading + // `expand` selects regions with. + const owner: Segment[] = []; + let hasOutput = false; + try { + const output = yield* evaluateGenerated(request, ROOT_CONTEXT, undefined, undefined, { + owner, + declared: (value) => { + hasOutput = value; + }, + }); + return Ok({ output, hasOutput }); + } catch (error) { + if (!ordinaryGeneratedFailure(error)) { + throw error; + } + // Only for a root that began rendering. An admission refusal performed + // nothing, so reporting an empty rendering for it would be reporting + // something that never happened. + const partial: GeneratedXmdRootResult | undefined = + hasOutput || owner.length > 0 ? { output: renderSegments(owner), hasOutput } : undefined; + return Err(new GeneratedXmdRootError(safeDiagnostic(error), partial)); + } +} + +/** + * Whether this failure is the request's own rather than this run's. + * + * Almost everything that leaves the expansion of an admitted root is the + * request's: source core refused, an element that failed on its own terms, a + * host control that declined what the source asked for. Those are what a + * caller decides about, so they come back as a value. + * + * Four kinds are not, and each keeps the classification it already has. They + * are asked about by kind rather than by position, because every one of them + * can arrive wrapped around a failure of the root's own, and the wrapper's + * shape must not decide which is reported (`errors.ts`): + * + * - a **durability failure** — the journal no longer describes this run, so + * there is no root whose outcome this could be; + * - a **Files infrastructure failure** — a provider is absent or broke its own + * contract, which is not something generated source could have caused; + * - a **secret-policy failure** — the one decision that must not become a + * correctable answer, because correcting it is asking again for the + * publication it refused; and + * - a **teardown failure** — work the root owned did not finish unwinding, and + * a rendering reported beside that would describe a root still running. + */ +function ordinaryGeneratedFailure(error: unknown): boolean { + if (error instanceof InvocationTeardownError) { + return false; + } + if (durabilityFailure(error) !== undefined || filesFatalFailure(error) !== undefined) { + return false; + } + return firstCause(error, secretPolicyFailure) === undefined; +} + +/** The secret-policy refusal this failure carries, if any. */ +function secretPolicyFailure(candidate: unknown): Error | undefined { + return candidate instanceof SecretDetectedError || candidate instanceof SecretScannerError + ? candidate + : undefined; +} + +/** + * What a host is told about an ordinary failure. + * + * The canonical sentence and nothing around it. A stack names host paths, a + * cause graph carries whatever the elements were holding, and the generated + * source is the one thing a diagnostic about generated source must not echo. + * A document failure answers with the sentence the error mode already settled, + * which is what an operator reading a printed error would have read. + */ +function safeDiagnostic(error: unknown): string { + const documented = documentationFailure(error); + if (documented !== undefined) { + return documented.message; + } + return error instanceof Error ? error.message : String(error); +} + +/** Where a root's rendering goes, and what the admitted source said about it. */ +interface RootRendering { + readonly owner: Segment[]; + /** Called once, with what the admitted source declared. */ + readonly declared: (hasOutput: boolean) => void; +} + +function* evaluateGenerated( + request: GeneratedXmdRequest, + context: GeneratedContext, + componentRouting: ComponentRouting | undefined, + syntax: SyntaxReference | undefined, + rendering: RootRendering | undefined, ): Operation { const allow = selection(request.allow); const entries = selectedEntries(request, allow); const table = admitted(entries); const ceilings = yield* normalizedCeilings(entries); - const policy = currentPolicy(request, allow, entries, ceilings); + const policy = currentPolicy(request, allow, entries, ceilings, context); const stored = yield* persistAdmission( request.id, @@ -2565,6 +2873,18 @@ export function* evaluateProtectedGeneratedXmd( // The retained source is what expands, so a continuation runs exactly the // bytes this run admitted rather than a caller's copy of them. - const restored = yield* preflight(request.id, decided.source, table, ceilings); - return yield* expand(request.id, restored.segments, restored.named, componentRouting, syntax); + const restored = yield* preflight(request.id, decided.source, table, ceilings, context); + // Read from the admitted segments, which is the same reading `expand` selects + // regions with, and reported before the first effect so a failure partway + // still describes the root truthfully. + rendering?.declared(bodyHasOutput(restored.segments)); + return yield* expand( + request.id, + restored.segments, + restored.named, + context, + componentRouting, + syntax, + rendering?.owner, + ); } diff --git a/packages/core/src/scanner.ts b/packages/core/src/scanner.ts index 86bae0e9a..09d8e8f42 100644 --- a/packages/core/src/scanner.ts +++ b/packages/core/src/scanner.ts @@ -222,6 +222,18 @@ export function scanSegments( * two readings would disagree the first time one of them was wrong. */ elements?: SourceElement[], + /** + * Every construct this scan began and could not finish because the text ran + * out, collected as the scan gives each one up. + * + * The same walk again, and for the same reason. Reading text this scan left + * as prose and looking for an unclosed tag would be a second grammar, and it + * would have to re-decide every question this one already answered — whether + * a `>` was quoted, whether a backtick opened a code span, whether a `{` was + * an expression. Only this walk knows the difference between a tag it refused + * and a tag it ran out of text in the middle of. + */ + unfinished?: UnfinishedConstruct[], ): Segment[] { const index: PositionIndex = { origin, lineStarts: computeLineStarts(text) }; const segments: Segment[] = []; @@ -243,6 +255,9 @@ export function scanSegments( fenceMatch.fenceChar, fenceMatch.fenceLen, ); + if (!fenceEnd.closed) { + unfinished?.push({ start: pos }); + } const content = text.slice(fenceMatch.contentStart, fenceEnd.contentEnd); const fullFence = text.slice(pos, fenceEnd.fenceEnd); @@ -275,12 +290,15 @@ export function scanSegments( pos = codeEnd; continue; } + // A span that never closes is a span this text has not finished + // delivering, not prose that happens to contain a backtick. + unfinished?.push({ start: pos }); } // Check for component invocation: `<` followed by uppercase letter if (text[pos] === "<" && pos + 1 < text.length && /[A-Z]/.test(text[pos + 1]!)) { // Make sure we're not inside a fenced code block (handled above) - const component = parseComponentTag(text, pos, index, elements); + const component = parseComponentTag(text, pos, index, elements, unfinished); if (component) { // Flush text before component if (pos > textStart) { @@ -321,6 +339,35 @@ export interface SourceElement { readonly closing?: SourceRange; } +/** + * One construct this scan began and could not finish, because the text ended. + * + * Told apart from a construct the scan *refused*: `` is not a tag + * and no amount of further text makes it one, while `Hello` are the beginning of tags whose rest has not arrived. Only + * the walk can say which, because only the walk knows how far it got and why it + * stopped — so a reader that needs to know asks for this rather than looking at + * the prose the scan left behind. + * + * Offsets index the exact text that was scanned. + */ +export interface UnfinishedConstruct { + /** Where the construct begins. */ + readonly start: number; + /** The authored spelling, for a component tag whose name the scan read. */ + readonly name?: string; + /** + * Where this element's own content begins. + * + * Present only for an element whose opening tag completed and whose closing + * tag never arrived, which is the one case where the text already delivered + * contains content belonging to a known element. + */ + readonly contentStart?: number; + /** Whether that opening tag carried any attribute at all. */ + readonly attributed?: boolean; +} + /** The source spans of the top-level component invocations in `text`. */ export function scanComponentSpans(text: string): ComponentSpan[] { const spans: ComponentSpan[] = []; @@ -338,6 +385,8 @@ interface FenceOpen { interface FenceClose { contentEnd: number; fenceEnd: number; + /** Whether a closing fence was actually found, rather than the text ending. */ + closed: boolean; } /** @@ -432,7 +481,7 @@ function findFenceClose( } if (pos >= text.length || text[pos] === "\n") { const fenceEnd = pos < text.length ? pos + 1 : pos; - return { contentEnd: lineStart, fenceEnd }; + return { contentEnd: lineStart, fenceEnd, closed: true }; } } } @@ -441,12 +490,12 @@ function findFenceClose( const nextNewline = text.indexOf("\n", pos); if (nextNewline === -1) { // No closing fence found — content goes to end - return { contentEnd: text.length, fenceEnd: text.length }; + return { contentEnd: text.length, fenceEnd: text.length, closed: false }; } pos = nextNewline + 1; } - return { contentEnd: text.length, fenceEnd: text.length }; + return { contentEnd: text.length, fenceEnd: text.length, closed: false }; } interface ParsedComponent { @@ -464,6 +513,7 @@ function parseComponentTag( start: number, index: PositionIndex, elements?: SourceElement[], + unfinished?: UnfinishedConstruct[], ): ParsedComponent | null { let pos = start + 1; // Skip '<' @@ -480,6 +530,12 @@ function parseComponentTag( if (!name) { return null; } + // A name that reached the end of the text is a tag still arriving: there is + // no `/>` or `>` yet because there is no more text yet. + if (pos >= text.length) { + unfinished?.push({ start, name }); + return null; + } // Parse attributes const { @@ -487,11 +543,19 @@ function parseComponentTag( expressions, authoredExpressions, end: attrEnd, + exhausted, } = parseAttributes(text, pos, name); if (attrEnd === -1) { + if (exhausted) { + unfinished?.push({ start, name }); + } return null; } pos = attrEnd; + const attributed = + Object.keys(props).length > 0 || + Object.keys(expressions).length > 0 || + Object.keys(authoredExpressions).length > 0; // Skip whitespace pos = skipWhitespace(text, pos); @@ -499,7 +563,11 @@ function parseComponentTag( // Self-closing tag? if (pos < text.length && text[pos] === "/") { pos++; // Skip '/' - if (pos >= text.length || text[pos] !== ">") { + if (pos >= text.length) { + unfinished?.push({ start, name, attributed }); + return null; + } + if (text[pos] !== ">") { return null; } pos++; // Skip '>' @@ -521,7 +589,11 @@ function parseComponentTag( } // Opening tag: must end with '>' - if (pos >= text.length || text[pos] !== ">") { + if (pos >= text.length) { + unfinished?.push({ start, name, attributed }); + return null; + } + if (text[pos] !== ">") { return null; } pos++; // Skip '>' @@ -531,8 +603,13 @@ function parseComponentTag( // committed only with this parse: an incomplete passive tag has no children // the scanner recognized, so it must leave none behind. const nested: SourceElement[] | undefined = elements === undefined ? undefined : []; - const { children, end: childEnd } = parseChildren(text, pos, name, index, nested); + const { children, end: childEnd } = parseChildren(text, pos, name, index, nested, unfinished); if (childEnd === -1) { + // The opening tag completed and the closing one never arrived, so the text + // already delivered holds content belonging to a known element. Reported + // with where that content starts, because a reader projecting a region it + // has not finished receiving needs exactly that. + unfinished?.push({ start, name, attributed, contentStart: openingEnd }); return null; } @@ -569,6 +646,16 @@ interface ParsedAttributes { /** Authored text of the `{…}` props resolved into `props` above. */ authoredExpressions: Record; end: number; // position after last attribute, before /> or > + /** + * Whether `end: -1` means the text ran out rather than that an attribute was + * written wrongly. + * + * Both end the tag, and they are different facts about the same `-1`: a + * quote, a brace or a value that never arrived is a tag still being + * delivered, while a character that cannot begin an attribute name or a value + * is one no further text repairs. + */ + exhausted: boolean; } function parseAttributes( @@ -584,12 +671,12 @@ function parseAttributes( while (pos < text.length) { pos = skipWhitespace(text, pos); if (pos >= text.length) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } // End of attributes? if (text[pos] === "/" || text[pos] === ">") { - return { props, expressions, authoredExpressions, end: pos }; + return { props, expressions, authoredExpressions, end: pos, exhausted: false }; } // Spread props: {...expr} @@ -602,7 +689,7 @@ function parseAttributes( // Skip spread — consume the expression const exprEnd = findMatchingBrace(text, pos); if (exprEnd === -1) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } // We don't evaluate spread props — just skip them pos = exprEnd + 1; @@ -616,7 +703,7 @@ function parseAttributes( } const attrName = text.slice(attrNameStart, pos); if (!attrName) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: false }; } pos = skipWhitespace(text, pos); @@ -646,14 +733,14 @@ function parseAttributes( // Attribute value if (pos >= text.length) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } if (text[pos] === '"') { // String attribute: "value" const strEnd = findClosingQuote(text, pos + 1, '"'); if (strEnd === -1) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } props[attrName] = text.slice(pos + 1, strEnd); pos = strEnd + 1; @@ -661,7 +748,7 @@ function parseAttributes( // String attribute: 'value' const strEnd = findClosingQuote(text, pos + 1, "'"); if (strEnd === -1) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } props[attrName] = text.slice(pos + 1, strEnd); pos = strEnd + 1; @@ -669,7 +756,7 @@ function parseAttributes( // Expression attribute: {expr} const exprEnd = findMatchingBrace(text, pos); if (exprEnd === -1) { - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } const exprText = text.slice(pos + 1, exprEnd).trim(); if (bindsByReference(componentName, attrName)) { @@ -696,11 +783,13 @@ function parseAttributes( } pos = exprEnd + 1; } else { - return { props, expressions, authoredExpressions, end: -1 }; + // A value that begins with something no attribute value begins with. + // Further text does not make this a tag. + return { props, expressions, authoredExpressions, end: -1, exhausted: false }; } } - return { props, expressions, authoredExpressions, end: -1 }; + return { props, expressions, authoredExpressions, end: -1, exhausted: true }; } /** @@ -865,6 +954,7 @@ function parseChildren( tagName: string, index: PositionIndex, elements?: SourceElement[], + unfinished?: UnfinishedConstruct[], ): ParsedChildren { let pos = start; let textStart = pos; @@ -896,6 +986,9 @@ function parseChildren( fenceMatch.fenceChar, fenceMatch.fenceLen, ); + if (!fenceEnd.closed) { + unfinished?.push({ start: pos }); + } const content = text.slice(fenceMatch.contentStart, fenceEnd.contentEnd); const fullFence = text.slice(pos, fenceEnd.fenceEnd); const parsed = parseInfoString(fenceMatch.infoString); @@ -926,11 +1019,12 @@ function parseChildren( pos = codeEnd; continue; } + unfinished?.push({ start: pos }); } // Check for nested component if (text[pos] === "<" && pos + 1 < text.length && /[A-Z]/.test(text[pos + 1]!)) { - const nested = parseComponentTag(text, pos, index, elements); + const nested = parseComponentTag(text, pos, index, elements, unfinished); if (nested) { if (pos > textStart) { pushText(children, text.slice(textStart, pos)); diff --git a/packages/core/tests/agent-conversation.test.ts b/packages/core/tests/agent-conversation.test.ts new file mode 100644 index 000000000..c8d361c5a --- /dev/null +++ b/packages/core/tests/agent-conversation.test.ts @@ -0,0 +1,605 @@ +/** + * Tier CV — one retained agent conversation a host holds open. + * + * A document's `` and `` are written inside an expansion, with + * an element to name the placement and a journal already around them. A host + * discussing something with an agent has neither and still needs the genuine + * thing, so `useAgentConversation()` opens the conversation as an execution it + * keeps alive — and these rows are about what that costs and what it buys. + * + * The provider here is a stub installed through the conversation's own + * installations, which is where the accepted contract puts it: the assembly + * selects the provider and its creation policy before any of the conversation's + * code runs. It uses the delivered placement coordinator exactly as a real + * provider does, because configuration readback is one of the things under test + * and a stub that skipped the coordinator could not show it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent } from "@executablemd/durable-streams"; +import { Ok, scoped, spawn, suspend, withResolvers } from "effection"; +import type { Operation, Result, Stream } from "effection"; + +import { useAgentConversation } from "../host.ts"; +import type { AgentConversation, ExecutionInstallation } from "../host.ts"; +import { Agent } from "../src/agent/agent-api.ts"; +import type { + AgentOptions, + AgentPromptEvent, + PromptOptions, + Session, + SessionConfiguration, +} from "../src/agent/agent-api.ts"; +import { installAgentComponents } from "../src/agent/components.ts"; +import { sessionOf } from "../src/agent/session-use.ts"; +import type { AgentLaunchCoordinator } from "../src/agent/launch-coordinator.ts"; + +const AGENT = "stub-agent"; + +/** What the stub provider was asked to do, in order. */ +interface Traffic { + readonly prompts: { content: string; sessionKey: string; agent: string }[]; + readonly placements: string[]; + readonly applied: SessionConfiguration[]; + readonly optionProbes: (string | undefined)[]; +} + +interface StubOptions { + /** How the provider answers one turn. Default: completes, echoing the text. */ + readonly respond?: (content: string) => { + status?: "completed" | "failed" | "cancelled"; + deltas?: string[]; + }; + /** + * Held open before a turn's first event, for a case that interrupts one. + * + * Asked per turn, so a case can hold the turn it means to abandon and let + * the next one through — a hold that applied to every turn would stall the + * work the case is actually about. + */ + readonly hold?: (content: string) => Operation | undefined; + /** Called when a held turn has actually reached the provider. */ + readonly started?: (content: string) => void; + /** What the provider reports it verified, when it is not what was asked. */ + readonly verify?: (asked: SessionConfiguration) => SessionConfiguration; +} + +/** + * One conversation installation carrying a stub provider. + * + * `install()` runs before the execution's document, which is the position the + * accepted contract gives a host's own assembly: it selects the provider and + * its creation policy before any of the conversation's code exists. The + * provider itself is installed the ordinary way, inside the document, so it is + * handed the same launch coordinator a real one is — which is what lets this + * stub settle placements and report what it verified. + */ +function stub(traffic: Traffic, options: StubOptions = {}): ExecutionInstallation { + return { + *install(): Operation { + yield* installAgentComponents({ + rootProvider: { + factory: function* ( + providerOptions, + coordinator: AgentLaunchCoordinator, + ): Operation { + yield* Agent.around( + { + // deno-lint-ignore require-yield + *agent([name]) { + return name ?? providerOptions.defaultAgent; + }, + *session([routed]): Operation { + if (typeof routed === "string" || routed === undefined) { + return { sessionKey: `stub:${routed ?? "default"}`, cwd: "/stub" }; + } + const placement = coordinator.sessionPlacement(routed); + traffic.placements.push(placement.sessionIdentity ?? ""); + const session: Session = { + sessionKey: `stub:${routed.name ?? "default"}`, + cwd: "/stub", + }; + // Established, so what this turn asks is applied and read + // back — the only thing that makes a configuration a fact + // about the turn rather than a hope. + return yield* placement.complete(session, { + kind: "established", + // deno-lint-ignore require-yield + *configure(asked: SessionConfiguration): Operation { + traffic.applied.push(asked); + return options.verify ? options.verify(asked) : asked; + }, + }); + }, + // deno-lint-ignore require-yield + *options([agent]): Operation { + traffic.optionProbes.push(agent); + return { + agent: agent ?? AGENT, + model: { + selected: "m-1", + options: [{ id: "m-1", name: "One", description: null, group: null }], + }, + effort: null, + }; + }, + // deno-lint-ignore require-yield + *prompt([content, promptOptions]) { + return turn(traffic, content, promptOptions, options); + }, + }, + { at: "min" }, + ); + }, + options: { defaultAgent: AGENT, permissionMode: "deny-all" }, + }, + }); + }, + }; +} + +/** One provider turn, as the stub plays it out. */ +function turn( + traffic: Traffic, + content: string, + promptOptions: PromptOptions | undefined, + options: StubOptions, +): Stream { + return { + *[Symbol.iterator]() { + // The exact value this turn was routed, not a copy of it: for a + // configured turn that is the authentic use, and it is the only thing + // that can say what the conversation was put under. + const routed = promptOptions?.session; + const session: Session = + typeof routed === "object" && routed !== null + ? routed + : { sessionKey: "stub:default", cwd: "/stub" }; + traffic.prompts.push({ + content, + sessionKey: sessionOf(session)?.sessionKey ?? session.sessionKey, + agent: promptOptions?.agent ?? AGENT, + }); + const response = options.respond ? options.respond(content) : {}; + const deltas = response.deltas ?? [`[${content}]`]; + const events: AgentPromptEvent[] = [ + { type: "started", agent: promptOptions?.agent ?? AGENT, session }, + ...deltas.map((text): AgentPromptEvent => ({ type: "text_delta", text })), + { type: "terminal", status: response.status ?? "completed" }, + ]; + let index = 0; + let waited = false; + return { + *next() { + if (!waited) { + waited = true; + const holding = options.hold?.(content); + if (holding !== undefined) { + options.started?.(content); + yield* holding; + } + } + if (index < events.length) { + return { done: false, value: events[index++]! }; + } + return { done: true, value: deltas.join("") }; + }, + }; + }, + }; +} + +/** Every canonical prompt record this history holds. */ +function prompts(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "agent_prompt", + ); +} + +/** Whether this history records the conversation root's own terminal. */ +function terminated(events: readonly DurableEvent[]): boolean { + return events.some((event) => event.type === "close" && event.coroutineId === "root"); +} + +/** The durable name of each retained turn, in order. */ +function turnNames(events: readonly DurableEvent[]): string[] { + return prompts(events).map((event) => + event.type === "yield" ? event.description.name : "", + ); +} + +/** What one retained turn record holds, for a case that reads the journal. */ +function record(events: readonly DurableEvent[], name: string): Record { + const found = prompts(events).find( + (event) => event.type === "yield" && event.description.name === name, + ); + if (found?.type !== "yield" || found.result.status !== "ok") { + throw new Error(`the history holds no completed turn "${name}"`); + } + const value = found.result.value; + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error(`the turn "${name}" retained no record`); + } + return value as Record; +} + +/** Open one conversation and hand it to `body`, closing it afterwards. */ +function withConversation( + stream: InMemoryStream, + installation: ExecutionInstallation, + body: (conversation: AgentConversation) => Operation, + id = "chat-1", +): Operation> { + return scoped(function* () { + const opened = yield* useAgentConversation({ + history: stream, + id, + agent: AGENT, + installations: [installation], + }); + if (!opened.ok) { + return opened; + } + return Ok(yield* body(opened.value)); + }); +} + +function fresh(): Traffic { + return { prompts: [], placements: [], applied: [], optionProbes: [] }; +} + +describe("Tier CV — a conversation is an execution the host keeps open", () => { + it("CV1: two turns reach the same conversation and are retained in order", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation(stream, stub(traffic), function* (chat) { + return [yield* chat.prompt("first"), yield* chat.prompt("second")]; + }); + + if (!answered.ok) { + throw new Error(`the conversation did not open: ${answered.error.message}`); + } + const [one, two] = answered.value; + expect(one.ok && one.value).toBe("[first]"); + expect(two.ok && two.value).toBe("[second]"); + expect(traffic.prompts.map((call) => call.content)).toEqual(["first", "second"]); + // One conversation: both turns ran in the session the one placement named. + expect(new Set(traffic.prompts.map((call) => call.sessionKey)).size).toBe(1); + + const events = yield* stream.readAll(); + expect(prompts(events)).toHaveLength(2); + // The root was never completed, which is what makes this history a + // conversation to reopen rather than a finished run to replay. + expect(terminated(events)).toBe(false); + }); + + it("CV2: reopening continues the conversation and replays no provider work", function* () { + const stream = new InMemoryStream(); + const first = fresh(); + const opened = yield* withConversation(stream, stub(first), function* (chat) { + return yield* chat.prompt("first"); + }); + expect(opened.ok).toBe(true); + + const again = fresh(); + const continued = yield* withConversation(stream, stub(again), function* (chat) { + return yield* chat.prompt("second"); + }); + + if (!continued.ok) { + throw new Error(`reopening refused: ${continued.error.message}`); + } + expect(continued.value.ok && continued.value.value).toBe("[second]"); + // The retained turn replayed from its own record: the second process asked + // the provider for exactly the one turn the person asked for. + expect(again.prompts.map((call) => call.content)).toEqual(["second"]); + expect(prompts(yield* stream.readAll())).toHaveLength(2); + }); + + it("CV3: one conversation runs one turn at a time", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + const release = withResolvers(); + const started = withResolvers(); + + const outcome = yield* withConversation( + stream, + stub(traffic, { + hold: () => release.operation, + started: () => started.resolve(), + }), + function* (chat) { + const held = yield* spawn(() => chat.prompt("first")); + // The turn is running when the provider says so, not when the call was + // made: a spawned task starts a turn later than its spawner. + yield* started.operation; + const overlapping = yield* chat.prompt("second"); + release.resolve(); + return { overlapping, held: yield* held }; + }, + ); + + if (!outcome.ok) { + throw new Error(`the conversation did not open: ${outcome.error.message}`); + } + expect(outcome.value.overlapping.ok).toBe(false); + expect(!outcome.value.overlapping.ok && outcome.value.overlapping.error.message).toContain( + "one turn at a time", + ); + expect(outcome.value.held.ok).toBe(true); + // The refused call reached no provider at all. + expect(traffic.prompts.map((call) => call.content)).toEqual(["first"]); + }); + + it("CV4: a handle kept past the scope that opened it reaches nothing", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + const kept: { chat?: AgentConversation } = {}; + + const opened = yield* withConversation(stream, stub(traffic), function* (chat) { + kept.chat = chat; + return yield* chat.prompt("first"); + }); + expect(opened.ok).toBe(true); + + const refused = yield* kept.chat!.prompt("after"); + expect(refused.ok).toBe(false); + expect(!refused.ok && refused.error.message).toContain("has been closed"); + // And it reached no provider. + expect(traffic.prompts).toHaveLength(1); + }); + + it("CV5: a failed turn answers Err and stays retained", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation( + stream, + stub(traffic, { respond: () => ({ status: "failed" }) }), + function* (chat) { + return yield* chat.prompt("first"); + }, + ); + + if (!answered.ok) { + throw new Error(`the conversation did not open: ${answered.error.message}`); + } + expect(answered.value.ok).toBe(false); + // The turn is a fact of the history either way: a failure is an outcome, + // not an absence. + expect(prompts(yield* stream.readAll())).toHaveLength(1); + }); + + it("CV6: inspecting advertised choices sends no model turn", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation(stream, stub(traffic), function* (chat) { + return yield* chat.options(); + }); + + if (!answered.ok) { + throw new Error(`the conversation did not open: ${answered.error.message}`); + } + expect(answered.value.ok && answered.value.value.model?.selected).toBe("m-1"); + expect(traffic.optionProbes).toEqual([AGENT]); + expect(traffic.prompts).toHaveLength(0); + // Inspection journals nothing. + expect(prompts(yield* stream.readAll())).toHaveLength(0); + }); + + it("CV7: a turn's configuration is applied and read back for this conversation", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation(stream, stub(traffic), function* (chat) { + return yield* chat.prompt("first", { model: "m-1", effort: "high" }); + }); + + expect(answered.ok).toBe(true); + expect(traffic.applied).toEqual([{ model: "m-1", effort: "high" }]); + }); +}); + +describe("Tier CV — the history decides whether a conversation may open", () => { + it("CV8: a history that records its own terminal is a finished run, not a conversation", function* () { + const stream = new InMemoryStream([ + { type: "close", coroutineId: "root", result: { status: "ok", value: "" } }, + ]); + + const opened = yield* withConversation(stream, stub(fresh()), function* () { + return "reached"; + }); + + expect(opened.ok).toBe(false); + expect(!opened.ok && opened.error.message).toContain("finished run"); + }); +}); + +describe("Tier CV — nothing opens a provider before the first turn", () => { + it("CV9: opening and closing a conversation reaches no provider at all", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const opened = yield* withConversation(stream, stub(traffic), function* () { + return "opened"; + }); + + expect(opened.ok).toBe(true); + expect(traffic.prompts).toHaveLength(0); + expect(traffic.placements).toHaveLength(0); + expect(prompts(yield* stream.readAll())).toHaveLength(0); + }); +}); + +describe("Tier CV — a turn is its own sequence", () => { + it("CV10: two turns take distinct durable names, in order", function* () { + const stream = new InMemoryStream(); + const answered = yield* withConversation(stream, stub(fresh()), function* (chat) { + yield* chat.prompt("first"); + return yield* chat.prompt("second"); + }); + + expect(answered.ok).toBe(true); + expect(turnNames(yield* stream.readAll())).toEqual(["turn:1", "turn:2"]); + }); + + it("CV11: cancelling a turn starts fresh work rather than resuming it", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + const started = withResolvers(); + + const answered = yield* withConversation( + stream, + // Only the turn this case abandons is held; the one after it runs. + stub(traffic, { + hold: (content) => (content === "abandoned" ? suspend() : undefined), + started: () => started.resolve(), + }), + function* (chat) { + const held = yield* spawn(() => chat.prompt("abandoned")); + yield* started.operation; + // Cancellation is scope cancellation, which is what Stop is. + yield* held.halt(); + return yield* chat.prompt("asked again"); + }, + ); + + if (!answered.ok) { + throw new Error(`the conversation did not open: ${answered.error.message}`); + } + expect(answered.value.ok && answered.value.value).toBe("[asked again]"); + // The abandoned turn reached the provider and recorded nothing: its append + // never landed, so there is no outcome for anything to resume. + expect(traffic.prompts.map((call) => call.content)).toEqual(["abandoned", "asked again"]); + const events = yield* stream.readAll(); + expect(turnNames(events)).toEqual(["turn:2"]); + // And the second turn is a turn of its own rather than the first finished. + expect(record(events, "turn:2").text).toBe("[asked again]"); + }); +}); + +describe("Tier CV — a history belongs to one conversation", () => { + it("CV12: a history established for another conversation refuses", function* () { + const stream = new InMemoryStream(); + const opened = yield* withConversation( + stream, + stub(fresh()), + function* (chat) { + return yield* chat.prompt("first"); + }, + "chat-1", + ); + expect(opened.ok).toBe(true); + + const misrouted = yield* withConversation( + stream, + stub(fresh()), + function* () { + return "reached"; + }, + "chat-2", + ); + + expect(misrouted.ok).toBe(false); + expect(!misrouted.ok && misrouted.error.message).toContain("another conversation"); + }); + + it("CV13: an established conversation does not change agent in place", function* () { + const stream = new InMemoryStream(); + const opened = yield* withConversation(stream, stub(fresh()), function* (chat) { + return yield* chat.prompt("first"); + }); + expect(opened.ok).toBe(true); + + const switched = yield* scoped(function* () { + const result = yield* useAgentConversation({ + history: stream, + id: "chat-1", + agent: "another-agent", + installations: [stub(fresh())], + }); + return result.ok ? "opened" : result.error.message; + }); + + expect(switched).toContain("never changes agent in place"); + }); + + it("CV14: a retained turn with no outcome refuses before anything opens", function* () { + const traffic = fresh(); + // A history interrupted mid-turn: the record was offered and never settled. + const stream = new InMemoryStream([ + { + type: "yield", + coroutineId: "root", + description: { type: "agent_prompt", name: "turn:1", input: "interrupted" }, + result: { status: "err", error: { message: "interrupted" } }, + }, + ]); + + const opened = yield* withConversation(stream, stub(traffic), function* () { + return "reached"; + }); + + expect(opened.ok).toBe(false); + expect(!opened.ok && opened.error.message).toContain("recorded no outcome"); + // Nothing ran, and nothing was appended to the history it refused. + expect(traffic.prompts).toHaveLength(0); + expect(yield* stream.readAll()).toHaveLength(1); + }); +}); + +describe("Tier CV — what a turn ran under is the provider's own account", () => { + it("CV15: the retained record carries the configuration the provider verified", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation(stream, stub(traffic), function* (chat) { + return yield* chat.prompt("first", { model: "m-1", effort: "high" }); + }); + + expect(answered.ok).toBe(true); + expect(record(yield* stream.readAll(), "turn:1").configuration).toEqual({ + model: "m-1", + effort: "high", + }); + }); + + it("CV16: a provider that put the conversation under something else refuses the turn", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation( + stream, + stub(traffic, { verify: () => ({ model: "m-2", effort: "high" }) }), + function* (chat) { + return yield* chat.prompt("first", { model: "m-1", effort: "high" }); + }, + ); + + // The turn never started: a conversation running under settings nobody + // asked for is not this turn's conversation. + expect(traffic.prompts).toHaveLength(0); + if (answered.ok) { + expect(answered.value.ok).toBe(false); + } + }); + + it("CV17: the record names the conversation the provider said the turn ran in", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + + const answered = yield* withConversation(stream, stub(traffic), function* (chat) { + return yield* chat.prompt("first"); + }); + + expect(answered.ok).toBe(true); + const retained = record(yield* stream.readAll(), "turn:1"); + expect(retained.sessionKey).toBe(traffic.prompts[0]?.sessionKey); + expect(retained.agent).toBe(AGENT); + expect(retained.status).toBe("completed"); + }); +}); diff --git a/packages/core/tests/generated-xmd-preview.test.ts b/packages/core/tests/generated-xmd-preview.test.ts new file mode 100644 index 000000000..c7e19b88a --- /dev/null +++ b/packages/core/tests/generated-xmd-preview.test.ts @@ -0,0 +1,307 @@ +/** + * Tier GP — what an arriving generated root already says. + * + * A host streaming a reply shows the person what the reply says while the rest + * of it is still arriving. `previewGeneratedXmdRoot()` answers that and nothing + * else, and these cases hold it to four claims. + * + * **The transport's chunk boundaries do not matter.** Two identical accumulated + * prefixes project identically, however the bytes were split — inside a tag, + * inside an expression, inside a fence. The answer is a complete projection of + * what has arrived rather than a delta, so a caller replaces what it was + * showing. + * + * **Only text whose meaning is settled is shown.** Literal region text + * previews. An expression, an interpolation and a nested component wait for + * evaluation, because a preview that guessed would show a person something the + * finished root never says. A tag written inside a passive fence is the example + * it looks like and creates no region at all. + * + * **Incomplete and invalid are different answers.** Source still arriving is a + * success that says so; source the language does not allow where it is written + * is a refusal. Neither is the other, and `incomplete: false` says only that + * this prefix is whole. + * + * **Nothing runs.** No file is read, no provider reached, no component invoked, + * no expression evaluated. The function is not even an `Operation`, which is + * most of the proof; what the cases add is that a prefix naming work does not + * perform it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; + +import { previewGeneratedXmdRoot } from "../host.ts"; +import type { GeneratedXmdRootPreview } from "../host.ts"; + +/** The projection, or a failure naming what was refused instead. */ +function preview(source: string): GeneratedXmdRootPreview { + const result = previewGeneratedXmdRoot(source); + if (!result.ok) { + throw new Error(`the prefix was refused: ${result.error.message}`); + } + return result.value; +} + +/** Why the prefix was refused, or a failure naming what it projected instead. */ +function refused(source: string): string { + const result = previewGeneratedXmdRoot(source); + if (result.ok) { + throw new Error(`the prefix projected ${JSON.stringify(result.value)}`); + } + return result.error.message; +} + +/** Every prefix of `source`, one code unit at a time. */ +function prefixes(source: string): string[] { + const all: string[] = []; + for (let length = 0; length <= source.length; length += 1) { + all.push(source.slice(0, length)); + } + return all; +} + +describe("Tier GP — a whole prefix projects what it says", () => { + it("GP1: a complete region is the projection, and the prefix is whole", function* () { + expect(preview("\nHello there.\n\n")).toEqual({ + output: "\nHello there.\n", + incomplete: false, + }); + }); + + it("GP2: source outside a region, and the region's own delimiters, are left out", function* () { + const projected = preview("documentation\n\nselected\n\nmore\n"); + + expect(projected.output).toBe("selected"); + expect(projected.incomplete).toBe(false); + }); + + it("GP3: several regions project in source order as one text", function* () { + expect(preview("one\nbetween\ntwo").output).toBe("onetwo"); + }); + + it("GP4: a prefix with no region at all projects nothing, which is success", function* () { + expect(preview("Just prose, and a finished beside it.\n")).toEqual({ + output: "", + incomplete: false, + }); + }); + + it("GP5: an empty prefix projects nothing and is whole", function* () { + expect(preview("")).toEqual({ output: "", incomplete: false }); + }); +}); + +describe("Tier GP — an unfinished prefix says what it has", () => { + it("GP6: an Output whose closing tag has not arrived carries its literal text", function* () { + expect(preview("Hello")).toEqual({ output: "Hello", incomplete: true }); + }); + + it("GP7: the closing tag arriving leaves the same text and makes the prefix whole", function* () { + const arriving = preview("Hello"); + const arrived = preview("Hello"); + + expect(arrived.output).toBe(arriving.output); + expect(arriving.incomplete).toBe(true); + expect(arrived.incomplete).toBe(false); + }); + + it("GP8: a tag whose name is still arriving projects nothing and waits", function* () { + for (const partial of ["<", ""]) { + const projected = preview(partial); + expect(projected.output).toBe(""); + // `<` alone is prose until an uppercase letter follows it, which is the + // scanner's own rule rather than a second one. + expect(projected.incomplete).toBe(partial.length > 1); + } + }); + + it("GP9: an unfinished construct after a whole region keeps the earlier text", function* () { + const projected = preview('done\n` arrives this region is not known to be top level, and + // once it does the source is refused. Showing its text now would show the + // person something the finished root never says. + const projected = preview("\nok\n"); + + expect(projected.output).toBe(""); + expect(projected.incomplete).toBe(true); + }); + + it("GP11: an unfinished expression prop waits", function* () { + const projected = preview('done\ndone\n\n```md\nexample"); + + expect(projected.output).toBe("done"); + expect(projected.incomplete).toBe(true); + }); + + it("GP13: an unclosed inline code span waits", function* () { + const projected = preview("done\n\nand `half a span"); + + expect(projected.output).toBe("done"); + expect(projected.incomplete).toBe(true); + }); +}); + +describe("Tier GP — only settled meaning projects", () => { + it("GP14: literal text projects and interpolation waits", function* () { + expect(preview("Hello, {who}!").output).toBe(""); + expect(preview("Hello, world!").output).toBe("Hello, world!"); + }); + + it("GP15: an escaped brace is literal text, exactly as expansion protects it", function* () { + expect(preview("a \\{literal} brace").output).toBe("a \\{literal} brace"); + }); + + it("GP16: a brace neither interpolation pass would read is literal", function* () { + expect(preview("a { b } rule").output).toBe("a { b } rule"); + }); + + it("GP17: a nested component waits, and the literal text around it does not", function* () { + expect(preview("before after").output).toBe("before after"); + }); + + it("GP18: an executable block inside a region projects nothing of itself", function* () { + const projected = preview("text\n\n```bash exec\nprintf ran\n```\n"); + + expect(projected.output).toContain("text"); + expect(projected.output).not.toContain("printf"); + expect(projected.incomplete).toBe(false); + }); + + it("GP19: a tag inside a passive fence creates no region", function* () { + const source = "Here is an example:\n\n```md\nnot a region\n```\n"; + + expect(preview(source)).toEqual({ output: "", incomplete: false }); + }); + + it("GP20: a region holding a passive fence projects the example as the text it is", function* () { + const source = "\n```md\nan example\n```\n"; + const projected = preview(source); + + // One region, whose text happens to look like another. The fence is text to + // the scan, so the inner tag is the example it looks like. + expect(projected.output).toContain("an example"); + expect(projected.incomplete).toBe(false); + }); + + it("GP21: a passive fence that reads a binding waits, because expansion reads it", function* () { + // A fence is a text segment, and interpolation runs over text segments. + // Showing this brace would show a person something the finished root + // replaces. + const source = '\n```md\n\n\n```\n'; + + expect(preview(source).output).toBe(""); + }); +}); + +describe("Tier GP — invalid source is refused, and says nothing of the prefix", () => { + it("GP23: a region written below the top level is refused once it is whole", function* () { + expect(refused("\nok\n\n")).toContain( + "top-level child", + ); + }); + + it("GP24: a region nested inside another is refused", function* () { + expect(refused("outer inner")).toContain("top-level child"); + }); + + it("GP25: a is refused, because a root declares no returns", function* () { + expect(refused('ok\n\n')).toContain(""); + }); + + it("GP26: a region carrying a prop is refused, whole or still arriving", function* () { + expect(refused('ok')).toContain("no props"); + expect(refused('ok')).toContain("no props"); + }); + + it("GP27: an invalid region is refused rather than reported incomplete", function* () { + // The same text that refuses whole stays a refusal: this is not a prefix + // waiting for more input. + const whole = previewGeneratedXmdRoot("outer inner"); + expect(whole.ok).toBe(false); + }); +}); + +describe("Tier GP — the transport's boundaries do not decide the answer", () => { + const STREAMED = [ + "\nHello, world!\n\n", + '\n\ndone\n', + "\n```md\nexample\n```\n\n", + "documentation\n\none\n\ntwo\n", + ]; + + for (const [index, source] of STREAMED.entries()) { + it(`GP28: every prefix of source ${index + 1} projects the same answer twice`, function* () { + for (const prefix of prefixes(source)) { + const once = previewGeneratedXmdRoot(prefix); + const again = previewGeneratedXmdRoot(prefix); + expect(once.ok).toBe(again.ok); + if (once.ok && again.ok) { + expect(once.value).toEqual(again.value); + } + } + }); + + it(`GP29: source ${index + 1} ends whole, and every prefix projects a prefix of it`, function* () { + const complete = previewGeneratedXmdRoot(source); + if (!complete.ok) { + throw new Error(`the whole source was refused: ${complete.error.message}`); + } + expect(complete.value.incomplete).toBe(false); + // What a person is shown only ever grows towards what the source says. A + // projection that had to be taken away again would be a preview that + // guessed, which is the thing this projection does not do. + for (const prefix of prefixes(source)) { + const projected = previewGeneratedXmdRoot(prefix); + if (!projected.ok) { + throw new Error(`prefix ${JSON.stringify(prefix)}: ${projected.error.message}`); + } + expect(complete.value.output.startsWith(projected.value.output)).toBe(true); + } + }); + } + + it("GP30: the projection replaces rather than appends as a prefix grows", function* () { + const growing = ["He", "Hell", "Hello"]; + const projected = growing.map((prefix) => preview(prefix).output); + + expect(projected).toEqual(["He", "Hell", "Hello"]); + }); +}); + +describe("Tier GP — nothing runs", () => { + it("GP31: a prefix naming file, network and command work projects and performs none of it", function* () { + // There is no execution around this call: no durable stream, no Files + // provider, no Agent provider, no transport. A projection that read a file + // or reached a provider could not answer here at all, and one that invoked + // a component would need a resolution nothing has installed. + const source = + 'written\n\n' + + '\n\n' + + "```bash exec\nprintf ran\n```\n\n" + + "\nthe reply\n\n"; + + expect(preview(source)).toEqual({ output: "\nthe reply\n", incomplete: false }); + }); + + it("GP32: the same prefix projects the same answer with nothing installed, twice over", function* () { + // Synchronous and total: the answer is a value rather than an operation, so + // there is no suspension point at which anything could have happened. + const source = 'steady\nwritten\n'; + + expect(previewGeneratedXmdRoot(source)).toEqual(previewGeneratedXmdRoot(source)); + }); +}); diff --git a/packages/core/tests/generated-xmd-root.test.ts b/packages/core/tests/generated-xmd-root.test.ts new file mode 100644 index 000000000..a7317617d --- /dev/null +++ b/packages/core/tests/generated-xmd-root.test.ts @@ -0,0 +1,677 @@ +/** + * Tier GR — evaluating generated source as a root. + * + * A trusted host that means to show somebody what an Agent's reply *says* needs + * the one construct a fragment has no context for: a top-level ``, which + * selects what the reply renders and leaves everything around it as work. So + * `evaluateGeneratedXmdRoot()` admits the same source the fragment evaluator + * does, through the same walk, the same durable record and the same pinned + * resolution, and differs over exactly that. + * + * Three claims are under test here. + * + * **A root selects what it renders, and says whether it selected.** `hasOutput` + * is the root's own declaration rather than a measurement of the text, so an + * `` region that renders nothing is a different answer from a root that + * declared no region at all — which is what lets a host suppress the ordinary + * whole-body fallback without reading the source itself. + * + * **An ordinary failure is the request's, and keeps what the root rendered.** + * Source core refused, and work the root's own elements failed at, come back as + * `Err`. A root that began rendering carries the rendering it had produced; one + * refused at admission performed nothing and carries none, because an empty + * rendering for work that never started would not be true. + * + * **Context is a ceiling.** A root admission does not resume as a fragment and a + * fragment admission does not resume as a root, so neither can expand source the + * other was admitted for. Every fragment record written before roots existed + * stays readable and resumable as what it is. + */ + +import { beforeAll, describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { API } from "@executablemd/runtime"; +import type { FetchInit, RuntimeFetchResponse } from "@executablemd/runtime"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent } from "@executablemd/durable-streams"; + +import { pinnedJson } from "../src/generated-xmd.ts"; +import { retainedSource } from "../src/root-source.ts"; +import { useTempFileCompiler } from "../src/temp-file-compiler.ts"; +import { + evaluateGeneratedXmd, + evaluateGeneratedXmdRoot, + pinnedComponent, + pinnedFetch, + pinnedMutation, +} from "../host.ts"; +import { executeInstalled } from "../host.ts"; +import type { + DurablePreparation, + ExecutionInstallation, + GeneratedMutation, + GeneratedObservation, + GeneratedXmdRequest, + GeneratedXmdRootResult, + RetainedFragmentIdentity, +} from "../host.ts"; +import type { Json } from "../src/types.ts"; + +const ROOT_PATH = "repl/sidekick.md"; +const ROOT_SOURCE = "The host evaluated a generated root.\n"; +const URL_ONE = "https://api.example.test/one"; + +/** One request the host admits, written the way an element writes it. */ +const ADMITTED_REQUEST: Record = { url: URL_ONE }; + +function hostIdentity(origin: string, key: string, revision = "1"): RetainedFragmentIdentity { + return { kind: "component-answer", origin, key, revision }; +} + +const NO_PROPS = { type: "object", properties: {}, additionalProperties: false } as const; + +/** The one host observation these cases admit beside ``. */ +function countedProbe(performed: string[]): GeneratedObservation { + return pinnedComponent("Probe", hostIdentity("test://probe", "Probe"), { + kind: "function", + name: "Probe", + props: NO_PROPS, + // deno-lint-ignore require-yield + *fn(): Operation { + performed.push("probed"); + return "probed"; + }, + }); +} + +/** + * A component that fails the way a host control fails: on its own terms, after + * whatever ran before it already ran. + */ +function failing(performed: string[]): GeneratedObservation { + return pinnedComponent("Boom", hostIdentity("test://boom", "Boom"), { + kind: "function", + name: "Boom", + props: NO_PROPS, + // deno-lint-ignore require-yield + *fn(): Operation { + performed.push("boomed"); + throw new Error("the control refused this request"); + }, + }); +} + +/** A paired-only entry, so writing it self-closing is a form refusal. */ +function paired(): GeneratedMutation { + return pinnedMutation( + "Nest", + hostIdentity("test://nest", "Nest"), + { + kind: "function", + name: "Nest", + props: NO_PROPS, + // deno-lint-ignore require-yield + *fn(): Operation { + return "nested"; + }, + }, + "paired", + ); +} + +/** A self-closing-only entry, so giving it content is a content refusal. */ +function selfClosing(): GeneratedMutation { + return pinnedMutation( + "Flat", + hostIdentity("test://flat", "Flat"), + { + kind: "function", + name: "Flat", + props: NO_PROPS, + // deno-lint-ignore require-yield + *fn(): Operation { + return "flat"; + }, + }, + "self-closing", + ); +} + +interface Ceilings { + readonly observations?: readonly GeneratedObservation[]; + readonly mutations?: readonly GeneratedMutation[]; + readonly allow?: readonly ("read" | "write")[]; +} + +/** + * One generated request, with no Workspace basis. + * + * A Sidekick host evaluates against none — its admitted effects address the + * providers its own execution installed — so these cases state none either. + * + * A read table is always stated, because a selection that reaches an empty one + * is the host's own mistake and the evaluator says so before it has anything to + * say about the source. The probe is the table a case that only cares about + * structure gets. + */ +function request(source: string, ceilings: Ceilings = {}): GeneratedXmdRequest { + const stated = ceilings.observations ?? []; + return { + id: "turn-1", + source, + observations: stated.length > 0 ? stated : [countedProbe([])], + ...(ceilings.mutations === undefined ? {} : { mutations: ceilings.mutations }), + ...(ceilings.allow === undefined ? {} : { allow: ceilings.allow }), + }; +} + +/** What one substituted transport was asked to do. */ +interface Transport { + readonly performed: Array<{ url: string; init: FetchInit | undefined }>; +} + +function* useTransport(body: string): Operation { + const performed: Transport["performed"] = []; + yield* API.Fetch.around( + { + // deno-lint-ignore require-yield + *fetch([url, init]): Operation { + performed.push({ url, init }); + return { + status: 200, + headers: { get: () => null, entries: () => [] }, + // deno-lint-ignore require-yield + *text(): Operation { + return body; + }, + }; + }, + }, + { at: "min" }, + ); + return { performed }; +} + +/** + * Drive an evaluator from a `DurablePreparation`. + * + * A harness choice, not the production path: the CLI reaches the root evaluator + * from its own Sidekick owner. What a preparation gives these cases is a durable + * root the evaluator's own records belong to, which is the only thing a + * continuation needs. + */ +function driven(work: () => Operation): ExecutionInstallation { + return { prepare: work as DurablePreparation }; +} + +/** What one host-driven root evaluation produced. */ +interface Attempt { + /** What the evaluator answered, when the run reached it. */ + readonly answer?: Result; + /** Why the run itself failed, when it did. */ + readonly failure?: string; + readonly events: DurableEvent[]; +} + +function evaluateRoot( + candidate: GeneratedXmdRequest, + options: { stream?: InMemoryStream } = {}, +): Operation { + return scoped(function* () { + const stream = options.stream ?? new InMemoryStream(); + const captured: { answer?: Result } = {}; + const installation = driven(function* () { + captured.answer = yield* evaluateGeneratedXmdRoot(candidate); + }); + const execution = yield* executeInstalled( + { ...retainedSource(ROOT_PATH, ROOT_SOURCE), stream }, + [installation], + ); + const result = yield* execution; + const events = yield* stream.readAll(); + if (result.ok) { + return { ...(captured.answer === undefined ? {} : { answer: captured.answer }), events }; + } + return { failure: result.error.message, events }; + }); +} + +/** The same, through the fragment evaluator, so one context can follow another. */ +function evaluateFragment( + candidate: GeneratedXmdRequest, + options: { stream?: InMemoryStream } = {}, +): Operation<{ output?: string; failure?: string; events: DurableEvent[] }> { + return scoped(function* () { + const stream = options.stream ?? new InMemoryStream(); + const captured: { output?: string } = {}; + const installation = driven(function* () { + captured.output = yield* evaluateGeneratedXmd(candidate); + }); + const execution = yield* executeInstalled( + { ...retainedSource(ROOT_PATH, ROOT_SOURCE), stream }, + [installation], + ); + const result = yield* execution; + const events = yield* stream.readAll(); + if (result.ok) { + return { output: captured.output ?? "", events }; + } + return { failure: result.error.message, events }; + }); +} + +function admissions(events: DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); +} + +function observations(events: DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => + event.type === "yield" && event.description.type === "fetch" && event.result.status === "ok", + ); +} + +/** A history holding the admission and nothing after it. */ +function duringPreparation(events: DurableEvent[]): InMemoryStream { + const admitted = events.findIndex( + (event) => event.type === "yield" && event.description.type === "generated_xmd", + ); + return new InMemoryStream(events.slice(0, admitted + 1)); +} + +/** The policy one admission retained, for a case that reads the record. */ +function retainedPolicy(events: DurableEvent[]): Record { + const [admission] = admissions(events); + if (admission?.type !== "yield" || admission.result.status !== "ok") { + throw new Error("the run recorded no generated-XMD admission"); + } + const value = admission.result.value; + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error("the admission record is not an object"); + } + const { policy } = value; + if (typeof policy !== "object" || policy === null || Array.isArray(policy)) { + throw new Error("the admission record holds no policy"); + } + return policy; +} + +/** What the evaluator answered, or a failure naming why there is nothing to read. */ +function answered(attempt: Attempt): Result { + expect(attempt.failure).toBe(undefined); + if (attempt.answer === undefined) { + throw new Error("the run never reached the root evaluator"); + } + return attempt.answer; +} + +function rendered(attempt: Attempt): GeneratedXmdRootResult { + const answer = answered(attempt); + if (!answer.ok) { + throw new Error(`the root failed: ${answer.error.message}`); + } + return answer.value; +} + +function failed(attempt: Attempt): Error & { partial?: GeneratedXmdRootResult } { + const answer = answered(attempt); + if (answer.ok) { + throw new Error(`the root succeeded, rendering ${JSON.stringify(answer.value)}`); + } + return answer.error; +} + +describe("Tier GR — a root selects what it renders", () => { + beforeAll(() => useTempFileCompiler()); + + it("GR1: an explicit Output region is the whole rendering, and the root says it declared one", function* () { + const performed: string[] = []; + const attempt = yield* evaluateRoot( + request("documentation\n\n\nHello there.\n\n\nmore documentation\n", { + observations: [countedProbe(performed)], + }), + ); + + const result = rendered(attempt); + expect(result.hasOutput).toBe(true); + expect(result.output).toContain("Hello there."); + expect(result.output).not.toContain("documentation"); + }); + + it("GR2: an Output region that renders nothing still declares one", function* () { + const attempt = yield* evaluateRoot(request("\n")); + + expect(rendered(attempt)).toEqual({ output: "", hasOutput: true }); + }); + + it("GR3: a root with no Output renders its whole body and declares none", function* () { + const performed: string[] = []; + const attempt = yield* evaluateRoot( + request("Hello there.\n\n\n", { observations: [countedProbe(performed)] }), + ); + + const result = rendered(attempt); + expect(result.hasOutput).toBe(false); + expect(result.output).toContain("Hello there."); + expect(result.output).toContain("probed"); + expect(performed).toEqual(["probed"]); + }); + + it("GR4: documentation beside a region runs for its effects and renders nothing", function* () { + const performed: string[] = []; + const attempt = yield* evaluateRoot( + request("\n\n\nselected\n\n", { + observations: [countedProbe(performed)], + }), + ); + + const result = rendered(attempt); + expect(result.output).toContain("selected"); + expect(result.output).not.toContain("probed"); + // It ran — rendering nothing is a decision about what the root shows, not + // about what its documentation does. + expect(performed).toEqual(["probed"]); + }); + + it("GR5: several regions render in source order as one rendering", function* () { + const performed: string[] = []; + const attempt = yield* evaluateRoot( + request("\nfirst\n\n\n\n\n\nsecond\n\n", { + observations: [countedProbe(performed)], + }), + ); + + const result = rendered(attempt); + expect(result.hasOutput).toBe(true); + expect(result.output.indexOf("first")).toBeGreaterThanOrEqual(0); + expect(result.output.indexOf("second")).toBeGreaterThan(result.output.indexOf("first")); + expect(result.output).not.toContain("probed"); + expect(performed).toEqual(["probed"]); + }); +}); + +describe("Tier GR — an ordinary failure keeps what the root rendered", () => { + beforeAll(() => useTempFileCompiler()); + + it("GR6: a runtime failure after a committed effect answers Err with the rendering so far", function* () { + const performed: string[] = []; + const transport = yield* useTransport("fetched"); + const attempt = yield* evaluateRoot( + request(`\nbefore\n\n\n\n\n\nafter\n\n`, { + observations: [pinnedFetch([ADMITTED_REQUEST]), failing(performed)], + }), + ); + + const error = failed(attempt); + // The control's own sentence, and nothing around it. + expect(error.message).toContain("the control refused this request"); + expect(error.message).not.toContain("generated-xmd-root.test.ts"); + // The effect before it happened and is retained: a failure is not a + // rollback. + expect(transport.performed).toHaveLength(1); + expect(observations(attempt.events)).toHaveLength(1); + // And the rendering the root had produced when it failed, carrying the same + // declaration the success would have carried. + expect(error.partial?.hasOutput).toBe(true); + expect(error.partial?.output).toContain("before"); + expect(error.partial?.output).not.toContain("after"); + expect(performed).toEqual(["boomed"]); + }); + + it("GR7: a root with no Output that fails partway keeps what it rendered", function* () { + const performed: string[] = []; + const attempt = yield* evaluateRoot( + request("before\n\n\n\nafter\n", { observations: [failing(performed)] }), + ); + + const error = failed(attempt); + expect(error.partial?.hasOutput).toBe(false); + expect(error.partial?.output).toContain("before"); + expect(error.partial?.output).not.toContain("after"); + }); + + it("GR8: an admission refusal carries no rendering, because nothing was rendered", function* () { + const attempt = yield* evaluateRoot(request("\n")); + + const error = failed(attempt); + expect(error.message).toContain("did not admit"); + expect(error.partial).toBe(undefined); + expect(observations(attempt.events)).toHaveLength(0); + }); +}); + +describe("Tier GR — the root context admits one construct more, and no others", () => { + beforeAll(() => useTempFileCompiler()); + + const REFUSED: Array<[string, string, string, Ceilings]> = [ + ["a name this host did not admit", "\n", "did not admit", {}], + [ + "an executable code block", + "\nok\n\n\n```bash exec\nprintf ran\n```\n", + "executable code block", + {}, + ], + [ + "a computed expression prop", + "\n\n\n", + "declarative data", + { observations: [countedProbe([])] }, + ], + [ + "interpolation of a binding that does not exist", + "\nHello, {who}!\n\n", + "reads a binding", + {}, + ], + ["", "\n\n\n", "structural construct", {}], + ["", '\nok\n\n\n\n', "structural construct", {}], + [ + "an written below the top level", + "\n\nok\n\n\n", + "structural construct", + {}, + ], + [ + "an carrying a prop", + '\nok\n\n', + "structural construct", + {}, + ], + [ + "a paired-only identity written self-closing", + "\n\n\n", + "self-closing a component", + { allow: ["write"], mutations: [paired()] }, + ], + [ + "a self-closing-only identity given content", + "\nx\n\n", + "only in its self-closing form", + { allow: ["write"], mutations: [selfClosing()] }, + ], + [ + "an unadmitted name in a branch this run would not take", + "\n\nok\n\n\n\n\n\n", + "did not admit", + {}, + ], + ]; + + for (const [what, source, reason, ceilings] of REFUSED) { + it(`GR9: ${what} refuses the root before any effect`, function* () { + const performed: string[] = []; + const transport = yield* useTransport("fetched"); + const attempt = yield* evaluateRoot( + request(source, { + ...ceilings, + observations: [ + pinnedFetch([ADMITTED_REQUEST]), + failing(performed), + ...(ceilings.observations ?? []), + ], + }), + ); + + const error = failed(attempt); + expect(error.message).toContain(reason); + expect(error.partial).toBe(undefined); + expect(transport.performed).toHaveLength(0); + expect(observations(attempt.events)).toHaveLength(0); + expect(performed).toEqual([]); + // The source is untrusted text, and a refusal of it does not republish it. + expect(error.message).not.toContain("Unknown"); + expect(error.message).not.toContain("printf"); + }); + } + + it("GR10: a fragment still has no context for a top-level Output", function* () { + const attempt = yield* evaluateFragment(request("\nok\n\n")); + + expect(attempt.failure).toContain("structural construct"); + }); +}); + +describe("Tier GR — a retained admission resumes only in its own context", () => { + beforeAll(() => useTempFileCompiler()); + + /** Source whose one effect is retained by its own durable record. */ + const WITH_EFFECT = `\n\n\n`; + const FRAGMENT_EFFECT = `\n`; + + it("GR11: a root admission records its context, and a fragment admission records none", function* () { + const asRoot = yield* evaluateRoot(request("\nok\n\n")); + expect(rendered(asRoot).hasOutput).toBe(true); + expect(retainedPolicy(asRoot.events).context).toBe("root"); + + const asFragment = yield* evaluateFragment(request("ok\n")); + expect(asFragment.failure).toBe(undefined); + // The shape every fragment record before roots existed was written in. A + // member added here would make this build's records unreadable to the one + // that wrote them. + expect(Object.hasOwn(retainedPolicy(asFragment.events), "context")).toBe(false); + }); + + it("GR12: the same context resumes and performs the effect it had not reached", function* () { + const first = yield* scoped(function* () { + yield* useTransport("fetched"); + return yield* evaluateRoot( + request(WITH_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + ); + }); + expect(rendered(first).hasOutput).toBe(true); + expect(observations(first.events)).toHaveLength(1); + + const again = yield* scoped(function* () { + const transport = yield* useTransport("fetched again"); + const attempt = yield* evaluateRoot( + request(WITH_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + { stream: duringPreparation(first.events) }, + ); + return { attempt, performed: transport.performed.length }; + }); + + // The admission restored, and the effect it had not reached performed once. + expect(rendered(again.attempt).hasOutput).toBe(true); + expect(again.performed).toBe(1); + expect(observations(again.attempt.events)).toHaveLength(1); + }); + + it("GR13: a root admission resumed as a fragment refuses and performs nothing", function* () { + const first = yield* scoped(function* () { + yield* useTransport("fetched"); + return yield* evaluateRoot( + request(WITH_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + ); + }); + expect(rendered(first).hasOutput).toBe(true); + + const again = yield* scoped(function* () { + const transport = yield* useTransport("fetched again"); + const attempt = yield* evaluateFragment( + request(WITH_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + { stream: duringPreparation(first.events) }, + ); + return { attempt, performed: transport.performed.length }; + }); + + expect(again.attempt.failure).toContain("admitted under"); + expect(again.performed).toBe(0); + }); + + it("GR14: a fragment admission resumed as a root refuses and performs nothing", function* () { + const first = yield* scoped(function* () { + yield* useTransport("fetched"); + return yield* evaluateFragment( + request(FRAGMENT_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + ); + }); + expect(first.failure).toBe(undefined); + + const again = yield* scoped(function* () { + const transport = yield* useTransport("fetched again"); + const attempt = yield* evaluateRoot( + request(FRAGMENT_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + { stream: duringPreparation(first.events) }, + ); + return { attempt, performed: transport.performed.length }; + }); + + const error = failed(again.attempt); + expect(error.message).toContain("admitted under"); + // A ceiling refusal is this run's history rather than the request's own + // text, so there is no rendering to report and nothing ran. + expect(error.partial).toBe(undefined); + expect(again.performed).toBe(0); + }); + + it("GR15: a released fragment admission still resumes as the fragment it was", function* () { + const first = yield* scoped(function* () { + yield* useTransport("fetched"); + return yield* evaluateFragment( + request(FRAGMENT_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + ); + }); + expect(first.failure).toBe(undefined); + + const again = yield* scoped(function* () { + const transport = yield* useTransport("fetched again"); + const attempt = yield* evaluateFragment( + request(FRAGMENT_EFFECT, { observations: [pinnedFetch([ADMITTED_REQUEST])] }), + { stream: duringPreparation(first.events) }, + ); + return { attempt, performed: transport.performed.length }; + }); + + expect(again.attempt.failure).toBe(undefined); + expect(again.performed).toBe(1); + }); +}); + +/** Core's own composition table, which every admission retains. */ +const COMPOSITION_NAME = pinnedJson().name; + +describe("Tier GR — the trusted composition table reaches a root too", () => { + beforeAll(() => useTempFileCompiler()); + + it("GR16: a root writes the composition table without the host naming it", function* () { + // The host's read table admits the probe and says nothing about ``. + const attempt = yield* evaluateRoot(request('\n\n\n')); + + expect(rendered(attempt).output).toContain('"Ada"'); + expect(COMPOSITION_NAME).toBe("Json"); + }); + + it("GR17: a root still may not read a binding through text interpolation", function* () { + const attempt = yield* evaluateRoot( + request('\n\n\n{who}\n\n'), + ); + + // The ceiling a fragment is held to, unchanged: the root context supplies + // `` and widens nothing else. + expect(failed(attempt).message).toContain("reads a binding"); + }); +}); diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 84275cf1f..685fb822c 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -400,6 +400,84 @@ access modes here; until then native launch cannot claim or widen any additional root. Filesystem accessibility and rendered agent instructions remain separate: the cwd is not injected as text, and rendered instructions grant no directory. +## A conversation a trusted host holds open + +`` and `` are written inside an expansion, with an element to +name the placement and a journal already around them. A host that discusses +something with an agent has neither: a person is typing into a chat, the +history outlives the process, and there is no document. `useAgentConversation()` +on `@executablemd/core/host` is that conversation, and it is the genuine one — +canonical Session placement, journaled turns, verified configuration, and an +identity the provider reattaches to. + +```ts +const opened = yield* useAgentConversation({ + history, // the DurableStream holding this conversation's canonical history + id, // the stable host-owned identity that history belongs to + agent, // the resolved agent name, already selected by the host + installations, // the trusted assembly that selects the provider and its policy +}); +``` + +The handle is provided by a resource owned by the caller's scope and offers two +members: `prompt(input, configuration?)`, which sends one journaled turn and +answers with its completed reply, and `options(request?)`, which inspects the +selected agent's advertised choices. Stop is scope cancellation rather than a +third member. + +**It is an element inside its own execution.** A provider and its placement +coordinator are installed inside `Execution.document`, and a durable preparation +precedes both, so a conversation opened there could reach neither. The +conversation is a host-declared identity component in a synthetic root core +owns: canonical resolution selects it, hands its implementation the execution's +claimant, and the session is placed under the engine's own identity for that +invocation. The `id` names the history and nothing else — it carries no +claimant, establishes no native identity and overrides none. + +**Its root is never completed.** A durable root that records its terminal +replays instead of reopening, and reopening a conversation is the ordinary case. +The element therefore serves turns and does not return; the halt that releases +the resource appends nothing, because a root records a terminal only on its own +success or failure. What the history keeps is a partial continuation. + +**Each turn is its own canonical sequence.** A turn is one journaled Prompt +under its own durable name, offered in the conversation's sequence exactly as +`` offers one in a document's, through the same `runPrompt` the +component uses rather than a copy of it. It runs in a task of its own, so +cancelling the call halts the turn: its append never lands, the position stays +free, and the next turn is a new turn rather than that one resumed. Reopening +re-offers the retained turns in their own order, which replays each from its +record and reaches no provider. + +A turn whose outcome the history does not hold refuses the conversation before +anything opens. Resuming it would re-send a prompt the provider may already have +accepted, and writing a terminal for it would make an unfinished turn look +complete; reconciling it belongs to the host, with the provider's own account of +that turn. Because the refusal happens before the execution exists, it appends +nothing to the history it refused. + +The history is bound to its conversation by a durable record written on the +first opening and replayed afterwards. A history established for another `id` +refuses rather than reconnecting the wrong chat, and one established with +another agent refuses rather than changing agent in place — an explicit agent +change is a new conversation with a new `id`. + +**What a turn ran under is the provider's own account.** A placement is made per +turn, because a placement settles what it asks before it is routed and is good +for one use: the first is fresh and the provider creates the conversation, and a +later one is established, so what the turn asks is applied and read back through +§Session configuration's own machinery. A provider that reports it put the +conversation under something else refuses that turn rather than having those +settings recorded. `options()` sends no model prompt and journals nothing, and +is called on explicit interaction rather than to restore a pane. + +One turn runs at a time; an overlapping call refuses rather than queuing, and a +handle kept past the scope that opened it refuses because the conversation it +addressed has been cancelled. A refusal before a turn starts — an unavailable +agent, an identity the provider will not reattach, settings it would not +honour — is that call's answer and leaves the conversation usable. Only a +durability failure ends it. + ## Components `installAgentComponents()` registers six components for the installing scope diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index b74096ca9..15a5ad256 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -1913,7 +1913,7 @@ run but are absent from the diagnostic trace. | `src/fetch-request.ts` | `prepareFetchRequest()`, `FetchRequest`, `FetchRequestError` — what `` admits before transport (§6.18) | | `src/fetch-response.ts` | `detachHeaders()`, `detachStatus()`, `FetchResponseRecord` — the response detached from the provider's (§6.18) | | `src/fetch-journal.ts` | `persistFetch()` — the `fetch` durable effect (§6.18, §10.1) | -| `src/generated-xmd.ts` | `evaluateGeneratedXmd()`, `pinnedFetch()`, `pinnedFileRead()`, `pinnedFileWrite()`, `pinnedFileDelete()`, `pinnedComponent()`, `pinnedMutation()`, `GeneratedEffectClass`, `GeneratedComponentForm`, `GeneratedMutation`, `GeneratedObservation`, `GeneratedRequest`, `GeneratedXmdError` — admitting Agent-generated source through the trusted-host seam under the always-on trusted composition table and a caller-selected subset of the closed effect classes `read` and `write`, resolving each name and authored form to one exact pinned identity — core Json, the self-closing `` read, the paired `` write and the self-closing `` among them — and preserving every component's ordinary binding and output behavior. `evaluateGeneratedXmd()` returns `Operation` containing the fragment's rendered text. Its admission and every generated effect use occurrence-owned identities in the existing ordinary durable sequence, with no `ephemeral()` bridge, staging or result envelope | +| `src/generated-xmd.ts` | `evaluateGeneratedXmd()`, `pinnedFetch()`, `pinnedFileRead()`, `pinnedFileWrite()`, `pinnedFileDelete()`, `pinnedComponent()`, `pinnedMutation()`, `GeneratedEffectClass`, `GeneratedComponentForm`, `GeneratedMutation`, `GeneratedObservation`, `GeneratedRequest`, `GeneratedXmdError` — admitting Agent-generated source through the trusted-host seam under the always-on trusted composition table and a caller-selected subset of the closed effect classes `read` and `write`, resolving each name and authored form to one exact pinned identity — core Json, the self-closing `` read, the paired `` write and the self-closing `` among them — and preserving every component's ordinary binding and output behavior. `evaluateGeneratedXmd()` returns `Operation` containing the fragment's rendered text. `evaluateGeneratedXmdRoot()` admits the same source as a **root** — the one context that supplies a top-level ``, with `` and `` unavailable in both — and returns `Operation>` stating the canonical rendering and whether the root declared a region at all. Which context an admission was made in is a term of its retained policy and is compared whole, so a root admission never resumes as a fragment or the reverse; a fragment record states no context, which is the shape every record written before roots existed carries. An ordinary failure answers `Err(GeneratedXmdRootError)` carrying the rendering the root had produced, while durability, Files-infrastructure, secret-policy and teardown failures keep their own classification. Its admission and every generated effect use occurrence-owned identities in the existing ordinary durable sequence, with no `ephemeral()` bridge, staging or result envelope | | `src/glob-source.ts` | `GLOB_PROPS`, `GLOB_RETURNS`, `globPatterns()`, `globFailure()` — the props, return contract, source rules and sanitized failure sentence one search is held to, shared by the ordinary `` component and the Glob capability an evaluation profile admits, so one dialect answers both (§6.14) | | `src/components/component-resolution.ts` | `CanonicalImports`, `ComponentResolution` — the witness a closed execution issues for the definition it produced and verifies where it is invoked | | `src/invocation.ts` | `withInvocation()`, `Invocation`, `InvocationTeardownError` — the component invocation boundary (§4.4) | @@ -1953,6 +1953,9 @@ run but are absent from the diagnostic trace. | `packages/cli/src/deno-workflow.ts` | the Deno run store and Workspace attachment; Node and Bun install the refusing host instead | | `packages/workflow/src/deno/workspace/files.ts` | the transaction-bound `API.Files` provider — one durable Workspace effect per document read, write, directory ensure and search | | `packages/workflow/src/deno/workspace/host.ts` | `withWorkflowWorkspace()` — the run's effect coordinator, logical cwd `/`, and Files provider installed together inside one execution | +| `src/generated-xmd-preview.ts` | `previewGeneratedXmdRoot()`, `GeneratedXmdRootPreview`, `GeneratedXmdPreviewError` — projecting the literal top-level `` text a response prefix already says. It takes the whole accumulated prefix and returns `Result` directly rather than an `Operation`, because it performs no suspended work: no file is read, no name resolved, no component invoked, no expression evaluated, no admission granted and no history written. A prefix still arriving is a success stating `incomplete`, and source the language does not allow where it is written is an `Err` — told apart by the scan's own report of the constructs it began and could not finish, through the same walk that decides what a fence, a quoted `>`, an inline code span and a tag-like expression are. Only settled meaning projects: an expression, an interpolation and a nested component wait, and a passive fence is the literal example it looks like | +| `src/generated-interpolation.ts` | `readsBinding()` — the one rule deciding whether a piece of text would read a binding, shared by admission, which refuses such text, and by the prefix preview, which leaves it out. Text one of them called literal and the other called a read would be two readings of one rule | +| `src/agent/conversation.ts` | `useAgentConversation()`, `AgentConversation`, `AgentConversationRequest`, `AgentConversationError` — one retained agent conversation a trusted host holds open over the history stream it supplies, under the installations it captured through `executeInstalled()`. A host-declared identity component in a synthetic root core owns, because a provider and its placement coordinator are installed inside `Execution.document` and a durable preparation precedes both; the session is therefore placed under the engine's own claimant-issued identity and the host's conversation id names the history alone. The element serves turns and does not return, so the root records no terminal and reopening resumes a partial continuation. Each turn is one journaled Prompt under its own durable name, run in a task of its own, reaching the same `runPrompt()` the component does rather than a copy | | `packages/workflow/src/generated-observations.ts` | `evaluateGeneratedFragment()` — the workflow policy adapter over `evaluateGeneratedXmd()`, returning its rendered text directly as `Operation`; `GeneratedObservation` remains the read-table entry type, not an output envelope | | `packages/workflow/src/journal.ts` | the `workflow_run` record, canonical-record recognition, and the refusals that name differing fields without their values | | `packages/workflow/src/run.ts` | `retainedWorkflowInstallation()` and `getWorkflowRun()` — associating one execution with a run already created, as the `ExecutionInstallation` a trusted host passes to `executeInstalled()`, contributing a mandatory run-identity admission and the `prepare` hook that reopens the run inside the durable root. The association resolves no revision and imports no Git feature, and a run retains the bytes it executes, so nothing afterwards reaches a repository for its Markdown | From 2765a76be0916e1d7ae0201345eb7fe7b861ad2e Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Sat, 10 Oct 2026 11:08:22 -0400 Subject: [PATCH 2/4] =?UTF-8?q?=F0=9F=90=9B=20Join=20a=20cancelled=20turn,?= =?UTF-8?q?=20and=20hold=20it=20to=20the=20conversation=20it=20established?= =?UTF-8?q?=20(#883)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two Planner findings against `d45bbfc6`. **P1 — cancelling a prompt returned before the provider had stopped.** The caller's cleanup resolved the cancellation signal and returned; the worker then halted the turn, but that worker belongs to the long-lived conversation rather than to the cancelled caller. So a halt that had returned proved nothing about whether provider work was still running — and a CLI cycle cannot build on that. The caller now waits for the turn to finish unwinding before its own halt completes. The worker reports that by resolving a signal from an `ensure` registered *before* the turn is spawned, which is what makes it the last thing to unwind: cleanups registered later run first, so the turn and every cleanup the provider registered inside it have finished by the time the caller is released. The conversation stays unavailable to another turn through that cleanup, because it is not free until the previous turn's work is over. A failure while the turn unwinds stays an authoritative teardown failure, and the conversation closing ends the wait too — then there is nothing left to do the joining. **P2 — nothing tested the native conversation an established history names.** The previous evidence compared the logical host id and the agent, which are different contracts. A conversation's established native identity is now read from the canonical turn records themselves: a completed Prompt retains the exact conversation the provider said that turn ran in, so the journal already holds the establishment. That is what lets a turn accepted before anything acknowledged a mapping be reconciled to *that* identity rather than leaving the conversation looking unestablished and open to a different one — and it needs no mapping record of its own, so there is no second account to disagree with the first. A provider that resolves a different native conversation is refused before the turn starts, where refusing costs nothing. One that names a different conversation only after the turn has run is refused too, and the established identity stands rather than being replaced. A history whose own turns name two different conversations cannot be opened at all: there is no rule for choosing between them that is not a guess. CV18–CV19 are the held-teardown regressions, CV20–CV24 the identity ones. Each was checked against its own negative control: removing the caller's join fails CV18 alone, removing the pre-turn check fails CV21 alone, removing the late-assertion check fails CV23 alone, and removing the two-identity refusal fails CV24 alone. --- packages/core/src/agent/conversation.ts | 265 ++++++++++++---- .../core/tests/agent-conversation.test.ts | 282 +++++++++++++++++- 2 files changed, 485 insertions(+), 62 deletions(-) diff --git a/packages/core/src/agent/conversation.ts b/packages/core/src/agent/conversation.ts index 1841d21ad..ce26f1917 100644 --- a/packages/core/src/agent/conversation.ts +++ b/packages/core/src/agent/conversation.ts @@ -98,8 +98,14 @@ import type { import { AgentInternal } from "./internal.ts"; import { runPrompt } from "./function-components.ts"; import type { Carried } from "./function-components.ts"; -import { AGENT_PROMPT, persistPrompt, promptFailureFromRecord } from "./journal.ts"; +import { + AGENT_PROMPT, + parsePromptRecord, + persistPrompt, + promptFailureFromRecord, +} from "./journal.ts"; import { sessionPlacement } from "./session-request.ts"; +import { sessionOf } from "./session-use.ts"; import type { ComponentInvocation, IdentityClaimant } from "../invocation-identity.ts"; import type { Json } from "../types.ts"; @@ -163,6 +169,7 @@ interface PromptJob { readonly configuration?: SessionConfiguration; readonly answer: Resolvers>; readonly cancelled: Resolvers; + readonly joined: Resolvers; } interface OptionsJob { @@ -170,6 +177,7 @@ interface OptionsJob { readonly request?: AgentOptionsRequest; readonly answer: Resolvers>; readonly cancelled: Resolvers; + readonly joined: Resolvers; } interface Resolvers { @@ -182,8 +190,28 @@ interface Resolvers { interface Live { /** False once the resource that owns this conversation has been released. */ open: boolean; - /** True while one turn or inspection is in flight. */ + /** + * True while one turn or inspection is in flight. + * + * It stays true through a cancelled call's own cleanup, because the + * conversation is not free until that turn's work has finished unwinding — a + * second turn admitted in between would start while the first was still + * inside the provider. + */ busy: boolean; + /** + * Resolved when the conversation itself is released. + * + * What stops a caller waiting on a turn that nothing is left to run: the + * element is gone, so nobody will report that turn joined, and the wait has + * to end on this instead. + */ + readonly closed: Resolvers; +} + +/** The native conversation identity one turn saw the provider name. */ +interface Observed { + native?: string; } /** What a history already holds about the conversation it belongs to. */ @@ -192,6 +220,18 @@ interface Retained { readonly turns: readonly string[]; /** The order the next turn takes, which is past every retained one. */ readonly next: number; + /** + * The native conversation identity this history has already established, if + * any. + * + * Read from the canonical turn records themselves rather than from a mapping + * record of its own. A completed Prompt retains the exact conversation the + * provider said that turn ran in, so the journal already holds the + * establishment — which is what lets a turn accepted before anything + * acknowledged a mapping be reconciled to *that* identity, instead of leaving + * the conversation looking unestablished and open to a different one. + */ + readonly native: string | undefined; } /** @@ -210,12 +250,15 @@ export function useAgentConversation( return resource(function* (provide) { const jobs = createQueue(); const ready = withResolvers>(); - const live: Live = { open: true, busy: false }; + const live: Live = { open: true, busy: false, closed: withResolvers() }; // Before the execution exists, so a caller halted while this resource is - // being acquired closes the handle rather than leaving one that looks open. + // being acquired closes the handle rather than leaving one that looks open + // — and so a caller waiting for its turn to join stops waiting when there + // is no longer anything running it. yield* ensure(() => { live.open = false; + live.closed.resolve(); }); // Read before anything opens, because what it finds decides whether this @@ -276,31 +319,44 @@ function handle( } const answer = withResolvers>(); const cancelled = withResolvers(); + const joined = withResolvers(); // What makes this cancellation rather than an ordinary return: a call // that was answered has nothing to cancel, and signalling one anyway // would make every finished turn look abandoned. let answered = false; - try { - // Registered before the send, so a caller halted between the two - // leaves no turn running that nothing is waiting for. - yield* ensure(() => { + let sent = false; + // Registered before the send, so a caller halted between the two + // leaves no turn running that nothing is waiting for. + yield* ensure(function* () { + try { + if (!sent) { + return; + } if (!answered) { cancelled.resolve(); } - }); - jobs.add({ - kind: "prompt", - input, - ...(configuration === undefined ? {} : { configuration }), - answer, - cancelled, - }); - const settled = yield* answer.operation; - answered = true; - return settled; - } finally { - live.busy = false; - } + // Asking is not stopping. This call does not finish until the turn + // it started has finished unwinding — the provider's own cleanup + // included — so a caller whose halt has returned knows that work is + // over. The conversation closing ends the wait too, because then + // there is nothing left to do the joining. + yield* race([joined.operation, live.closed.operation]); + } finally { + live.busy = false; + } + }); + jobs.add({ + kind: "prompt", + input, + ...(configuration === undefined ? {} : { configuration }), + answer, + cancelled, + joined, + }); + sent = true; + const settled = yield* answer.operation; + answered = true; + return settled; }); }, options(inspection?: AgentOptionsRequest): Operation> { @@ -311,25 +367,33 @@ function handle( } const answer = withResolvers>(); const cancelled = withResolvers(); + const joined = withResolvers(); let answered = false; - try { - yield* ensure(() => { + let sent = false; + yield* ensure(function* () { + try { + if (!sent) { + return; + } if (!answered) { cancelled.resolve(); } - }); - jobs.add({ - kind: "options", - ...(inspection === undefined ? {} : { request: inspection }), - answer, - cancelled, - }); - const settled = yield* answer.operation; - answered = true; - return settled; - } finally { - live.busy = false; - } + yield* race([joined.operation, live.closed.operation]); + } finally { + live.busy = false; + } + }); + jobs.add({ + kind: "options", + ...(inspection === undefined ? {} : { request: inspection }), + answer, + cancelled, + joined, + }); + sent = true; + const settled = yield* answer.operation; + answered = true; + return settled; }); }, }; @@ -430,6 +494,9 @@ function* held( yield* replayTurn(name); } let order = retained.next; + // What the history already established, carried forward so a turn in this + // process is held to it as firmly as one after a reopening is. + let established = retained.native; announce(Ok(undefined)); while (true) { const next = yield* jobs.next(); @@ -441,7 +508,12 @@ function* held( yield* inspect(request, job); continue; } - yield* deliver(request, identity, job, order); + const observed: Observed = {}; + yield* deliver(request, identity, established, observed, job, order); + // Established by the first turn the provider reported one for, and never + // replaced afterwards: a turn naming a different one was refused before it + // could record anything. + established ??= observed.native; order += 1; } // Nothing closes this queue, and an element that returned would let the root @@ -513,6 +585,8 @@ function* replayTurn(name: string): Operation { function deliver( request: AgentConversationRequest, identity: string, + established: string | undefined, + observed: Observed, job: PromptJob, order: number, ): Operation { @@ -520,7 +594,17 @@ function deliver( // returns: a turn still running would hold its durable operation open, and // the next turn would be a second one open on the same sequence. return scoped(function* () { - const turn = yield* spawn(() => conversationTurn(request, identity, job, order)); + // Registered *before* the turn is spawned, which is what makes it the last + // thing to unwind: cleanups registered later run first, so the turn — and + // every cleanup the provider registered inside it — has finished by the + // time this reports the turn joined. A caller waiting on this therefore + // waits for the work, not for the request to stop being interesting. + yield* ensure(() => { + job.joined.resolve(); + }); + const turn = yield* spawn(() => + conversationTurn(request, identity, established, observed, job, order), + ); const outcome = yield* race([ (function* (): Operation<"finished"> { try { @@ -545,7 +629,9 @@ function deliver( // Giving up on the answer is not cancelling the turn: the task is its own, // and halting it is what leaves its append unmade — a fact rather than a // suffix for the next turn to resume. Observed, because a halt that nobody - // waited for is a turn still unwinding while the next one starts. + // waited for is a turn still unwinding while the next one starts, and a + // failure while it unwinds is an authoritative teardown failure rather + // than this turn's answer. yield* turn.halt(); }); } @@ -563,6 +649,8 @@ function deliver( function conversationTurn( request: AgentConversationRequest, identity: string, + established: string | undefined, + observed: Observed, job: PromptJob, order: number, ): Operation { @@ -576,6 +664,20 @@ function conversationTurn( } finally { issuance.close(); } + // Before the turn starts, because this is the one point where refusing + // costs nothing: the provider has named the conversation it resolved, and + // one that is not the established conversation is not this conversation. + // Replacing the identity is never the answer — the history already says + // which conversation this is, and a turn sent elsewhere would reach one + // this chat has never been in. + const resolved = sessionOf(session)?.agentSessionId; + if (established !== undefined && resolved !== undefined && resolved !== established) { + throw new AgentConversationError( + "this provider resolved a different native conversation from the one this history " + + "established, so a turn sent to it would reach a conversation this chat has never " + + "been in. An established conversation is not reattached to another identity.", + ); + } // Resolved before the turn starts, so an unavailable agent fails as // itself rather than being journaled as a failed turn. const agent = yield* Agent.operations.agent(request.agent); @@ -588,6 +690,25 @@ function conversationTurn( () => carried.association, () => carried.begun, ); + // A provider may name the conversation only once the turn has started, + // which is the ordinary first-turn case. Read from the record, because + // that is the account the journal keeps and the one a later opening + // reconciles against. + if (record.agentSessionId !== undefined) { + if (established !== undefined && record.agentSessionId !== established) { + job.answer.resolve( + Err( + new AgentConversationError( + "this turn ran in a different native conversation from the one this history " + + "established. The established identity stands; this turn's outcome is not " + + "this conversation's to continue from.", + ), + ), + ); + return; + } + observed.native = record.agentSessionId; + } const failure = promptFailureFromRecord(record); job.answer.resolve(failure === undefined ? Ok(record.text) : Err(failure)); } catch (error) { @@ -607,24 +728,29 @@ function conversationTurn( } /** Inspect the selected agent's advertised choices. Journals nothing. */ -function* inspect(request: AgentConversationRequest, job: OptionsJob): Operation { - const answered = yield* race([ - (function* (): Operation<{ readonly done: Result }> { - try { - return { done: Ok(yield* Agent.operations.options(request.agent, job.request)) }; - } catch (error) { - return { done: Err(error instanceof Error ? error : new Error(String(error))) }; - } - })(), - (function* (): Operation<{ readonly cancelled: true }> { - yield* job.cancelled.operation; - return { cancelled: true }; - })(), - ]); - if ("cancelled" in answered) { - return; - } - job.answer.resolve(answered.done); +function inspect(request: AgentConversationRequest, job: OptionsJob): Operation { + return scoped(function* () { + yield* ensure(() => { + job.joined.resolve(); + }); + const answered = yield* race([ + (function* (): Operation<{ readonly done: Result }> { + try { + return { done: Ok(yield* Agent.operations.options(request.agent, job.request)) }; + } catch (error) { + return { done: Err(error instanceof Error ? error : new Error(String(error))) }; + } + })(), + (function* (): Operation<{ readonly cancelled: true }> { + yield* job.cancelled.operation; + return { cancelled: true }; + })(), + ]); + if ("cancelled" in answered) { + return; + } + job.answer.resolve(answered.done); + }); } /** @@ -646,6 +772,7 @@ function* openable(request: AgentConversationRequest): Operation(); let highest = 0; for (const event of events) { if (event.type !== "yield" || event.description.type !== AGENT_PROMPT) { @@ -663,8 +790,28 @@ function* openable(request: AgentConversationRequest): Operation 1) { + // Two different identities among one conversation's own turns. There is no + // rule for choosing between them that is not a guess, and guessing would + // attach the next turn to whichever happened to be read last. + return Err( + new AgentConversationError( + "this conversation's retained turns name more than one native conversation, so which " + + "one it is cannot be established from its own history. Reconcile it against the " + + "provider's own account before sending another turn.", + ), + ); } - return Ok(Object.freeze({ turns: Object.freeze(turns), next: highest + 1 })); + const [native] = asserted; + return Ok(Object.freeze({ turns: Object.freeze(turns), next: highest + 1, native })); } /** Whether these events hold the conversation root's own terminal. */ diff --git a/packages/core/tests/agent-conversation.test.ts b/packages/core/tests/agent-conversation.test.ts index c8d361c5a..7b1adda62 100644 --- a/packages/core/tests/agent-conversation.test.ts +++ b/packages/core/tests/agent-conversation.test.ts @@ -19,7 +19,7 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { Ok, scoped, spawn, suspend, withResolvers } from "effection"; +import { ensure, Ok, race, scoped, sleep, spawn, suspend, withResolvers } from "effection"; import type { Operation, Result, Stream } from "effection"; import { useAgentConversation } from "../host.ts"; @@ -64,6 +64,18 @@ interface StubOptions { readonly started?: (content: string) => void; /** What the provider reports it verified, when it is not what was asked. */ readonly verify?: (asked: SessionConfiguration) => SessionConfiguration; + /** + * The native conversation identity the provider asserts, per turn. + * + * A real provider asserts it on the Session it issued, and reports that same + * value on the turn's `started` event — which is how the journal comes to + * hold the provider's own account of which conversation a turn ran in. + * Asked per turn so a case can have the provider resolve a *different* + * conversation on reopening. + */ + readonly asserts?: (content: string) => string | undefined; + /** Asserted when the session resolves rather than when the turn starts. */ + readonly assertsAtPlacement?: string; } /** @@ -100,6 +112,9 @@ function stub(traffic: Traffic, options: StubOptions = {}): ExecutionInstallatio const session: Session = { sessionKey: `stub:${routed.name ?? "default"}`, cwd: "/stub", + ...(options.assertsAtPlacement === undefined + ? {} + : { agentSessionId: options.assertsAtPlacement }), }; // Established, so what this turn asks is applied and read // back — the only thing that makes a configuration a fact @@ -153,13 +168,18 @@ function turn( // configured turn that is the authentic use, and it is the only thing // that can say what the conversation was put under. const routed = promptOptions?.session; - const session: Session = + const placed: Session = typeof routed === "object" && routed !== null ? routed : { sessionKey: "stub:default", cwd: "/stub" }; + // Asserting the native identity the way a provider does: on the value it + // reports as the conversation this turn ran in. + const asserted = options.asserts?.(content); + const session: Session = + asserted === undefined ? placed : { ...placed, agentSessionId: asserted }; traffic.prompts.push({ content, - sessionKey: sessionOf(session)?.sessionKey ?? session.sessionKey, + sessionKey: sessionOf(placed)?.sessionKey ?? placed.sessionKey, agent: promptOptions?.agent ?? AGENT, }); const response = options.respond ? options.respond(content) : {}; @@ -603,3 +623,259 @@ describe("Tier CV — what a turn ran under is the provider's own account", () = expect(retained.status).toBe("completed"); }); }); + +describe("Tier CV — cancelling a turn joins the work it started", () => { + it("CV18: the caller's halt waits for the provider's own cleanup", function* () { + const started = withResolvers(); + const cleanupStarted = withResolvers(); + const cleanupRelease = withResolvers(); + + const answered = yield* withConversation( + new InMemoryStream(), + stub(fresh(), { + // A turn whose teardown is held open, which is the only shape that can + // tell "the caller stopped waiting" apart from "the work stopped". + hold: () => + (function* (): Operation { + yield* ensure(function* () { + cleanupStarted.resolve(); + yield* cleanupRelease.operation; + }); + started.resolve(); + yield* suspend(); + })(), + }), + function* (chat) { + // Registered before the held work exists, so the gate opens however + // this case leaves and nothing is left blocked on it. + yield* ensure(() => { + cleanupRelease.resolve(); + }); + const caller = yield* spawn(() => chat.prompt("held")); + yield* started.operation; + const halted = yield* spawn(function* (): Operation { + yield* caller.halt(); + return "halt returned"; + }); + yield* cleanupStarted.operation; + const observation = yield* race([ + (function* (): Operation { + return yield* halted; + })(), + (function* (): Operation { + yield* sleep(25); + return "halt waits"; + })(), + ]); + // Released after the observation, so the case joins its own work + // rather than leaving a held cleanup behind. + cleanupRelease.resolve(); + yield* halted; + return observation; + }, + ); + + if (!answered.ok) { + throw answered.error; + } + // Asking is not stopping: the halt is still unwinding the provider's + // cleanup, so a caller that has returned could not claim the work stopped. + expect(answered.value).toBe("halt waits"); + }); + + it("CV19: fresh work starts after a cancelled turn whose teardown was held", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + const started = withResolvers(); + const cleanupRelease = withResolvers(); + + const answered = yield* withConversation( + stream, + stub(traffic, { + hold: (content) => + content !== "abandoned" + ? undefined + : (function* (): Operation { + yield* ensure(function* () { + yield* cleanupRelease.operation; + }); + started.resolve(); + yield* suspend(); + })(), + }), + function* (chat) { + const caller = yield* spawn(() => chat.prompt("abandoned")); + yield* started.operation; + const halted = yield* spawn(function* (): Operation { + yield* caller.halt(); + }); + cleanupRelease.resolve(); + yield* halted; + // The conversation is free again only because the cancelled turn's + // work finished unwinding first. + return yield* chat.prompt("asked again"); + }, + ); + + if (!answered.ok) { + throw answered.error; + } + expect(answered.value.ok && answered.value.value).toBe("[asked again]"); + expect(traffic.prompts.map((call) => call.content)).toEqual(["abandoned", "asked again"]); + // The abandoned turn recorded nothing, so nothing resumes it. + expect(turnNames(yield* stream.readAll())).toEqual(["turn:2"]); + }); +}); + +describe("Tier CV — the native conversation an established history names", () => { + it("CV20: the provider's assertion is retained as the turn's own account", function* () { + const stream = new InMemoryStream(); + + const answered = yield* withConversation( + stream, + stub(fresh(), { asserts: () => "native-1" }), + function* (chat) { + return yield* chat.prompt("first"); + }, + ); + + expect(answered.ok).toBe(true); + expect(record(yield* stream.readAll(), "turn:1").agentSessionId).toBe("native-1"); + }); + + it("CV21: a provider resolving a different native conversation refuses the turn", function* () { + const stream = new InMemoryStream(); + const first = fresh(); + const opened = yield* withConversation( + stream, + stub(first, { asserts: () => "native-1" }), + function* (chat) { + return yield* chat.prompt("first"); + }, + ); + expect(opened.ok).toBe(true); + + // Reopened against a provider that resolves some other conversation. + const again = fresh(); + const continued = yield* withConversation( + stream, + stub(again, { assertsAtPlacement: "native-other", asserts: () => "native-other" }), + function* (chat) { + return yield* chat.prompt("second"); + }, + ); + + if (!continued.ok) { + throw new Error(`reopening refused outright: ${continued.error.message}`); + } + expect(continued.value.ok).toBe(false); + expect(!continued.value.ok && continued.value.error.message).toContain( + "not reattached to another identity", + ); + // Refused before the turn started: no prompt was sent anywhere, and the + // established identity stands in the history unchanged. + expect(again.prompts).toHaveLength(0); + expect(turnNames(yield* stream.readAll())).toEqual(["turn:1"]); + expect(record(yield* stream.readAll(), "turn:1").agentSessionId).toBe("native-1"); + }); + + it("CV22: a turn accepted before anything acknowledged a mapping reconciles to that identity", function* () { + const stream = new InMemoryStream(); + const first = fresh(); + // The whole history is one accepted turn and nothing after it: there is no + // mapping record, only the provider's own account of the turn. + const opened = yield* withConversation( + stream, + stub(first, { asserts: () => "native-1" }), + function* (chat) { + return yield* chat.prompt("first"); + }, + ); + expect(opened.ok).toBe(true); + const events = yield* stream.readAll(); + expect(turnNames(events)).toEqual(["turn:1"]); + + // Reopened against a provider that resolves that same conversation. It is + // reconciled to that identity rather than treated as unestablished, and no + // replacement conversation is created. + const again = fresh(); + const continued = yield* withConversation( + stream, + stub(again, { assertsAtPlacement: "native-1", asserts: () => "native-1" }), + function* (chat) { + return yield* chat.prompt("second"); + }, + ); + + if (!continued.ok) { + throw new Error(`reopening refused: ${continued.error.message}`); + } + expect(continued.value.ok && continued.value.value).toBe("[second]"); + // The accepted turn was not resent, and the second turn joined the same + // native conversation. + expect(again.prompts.map((call) => call.content)).toEqual(["second"]); + expect(record(yield* stream.readAll(), "turn:2").agentSessionId).toBe("native-1"); + }); + + it("CV23: a turn that ran in another native conversation does not replace the established one", function* () { + const stream = new InMemoryStream(); + const first = fresh(); + const opened = yield* withConversation( + stream, + stub(first, { asserts: () => "native-1" }), + function* (chat) { + return yield* chat.prompt("first"); + }, + ); + expect(opened.ok).toBe(true); + + // The placement resolves the established conversation, and the provider + // then reports the turn ran somewhere else — an assertion that arrives too + // late to be refused before the turn. + const again = fresh(); + const continued = yield* withConversation( + stream, + stub(again, { assertsAtPlacement: "native-1", asserts: () => "native-late" }), + function* (chat) { + return yield* chat.prompt("second"); + }, + ); + + if (!continued.ok) { + throw new Error(`reopening refused outright: ${continued.error.message}`); + } + expect(continued.value.ok).toBe(false); + expect(!continued.value.ok && continued.value.error.message).toContain( + "established identity stands", + ); + }); + + it("CV24: a history naming two native conversations cannot be opened", function* () { + const stream = new InMemoryStream(); + const traffic = fresh(); + let turn = 0; + const opened = yield* withConversation( + stream, + // Two turns, two different native conversations: nothing in the history + // says which one this chat is. + stub(traffic, { + asserts: () => { + turn += 1; + return `native-${turn}`; + }, + }), + function* (chat) { + yield* chat.prompt("first"); + return yield* chat.prompt("second"); + }, + ); + expect(opened.ok).toBe(true); + + const reopened = yield* withConversation(stream, stub(fresh()), function* () { + return "reached"; + }); + + expect(reopened.ok).toBe(false); + expect(!reopened.ok && reopened.error.message).toContain("more than one native conversation"); + }); +}); From 05d03afb0ff1356707f66f2f1b741a5d8ec6ba95 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Sat, 10 Oct 2026 15:09:32 -0400 Subject: [PATCH 3/4] =?UTF-8?q?=E2=9C=85=20Rejoin=20the=20conversation=20a?= =?UTF-8?q?=20provider=20asserted=20before=20its=20mapping=20(#883)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The provider's commit order is the contract: the adapter accepts the session, ACPX's own record is promoted to assert an identity, the host's mapping is acknowledged second, and the session becomes established last. The window between the first two is the one a chat has to recover from — the agent is in a conversation nobody wrote down — and no stub provider can show it, because a stub has no acknowledgement to interrupt. Tier HC drives the real provider for that: `installAgentComponents` with `createAcpxProvider` as the root provider, exactly as `installAgentProviderStack` assembles it, with ACPX's runtime replaced by the scriptable fake this package already drives it with and the host's `established` hook as the one seam a case refuses. `provider.test.ts` SM9 says that window leaves one canonical assertion; these rows ask what the conversation can still do after it. HC1 is the window itself. The interrupted turn never started, so it retained no identity, and what the next turn joins is decided by the provider's own record: reopening reconciles to that same assertion, commits it, creates nothing in its place and does not resend the interrupted prompt. HC2 is the same interruption once a completed turn has retained an identity, where the journal is the account that survives — the refused turn is its own recorded fact and the chat is still the conversation its first turn established. HC3 is what that identity is for: a provider answering with another conversation has that turn's outcome refused, and a history whose own turns name two of them cannot be opened. CV22 claimed the first of these and did not test it; it is renamed to what it proves — reopening joins the identity its retained turns name, without resending them — and points at HC1 for the window before a turn has retained one. The architecture section and §A conversation a trusted host holds open now state the established-identity rule and the cancellation join the revision introduced. Each row was checked against its own control: leaving the acknowledgement alone fails HC1 alone, and then HC2 alone; letting the provider answer with the same conversation fails HC3 alone. Against the parent conversation source HC3 fails and HC1/HC2 pass, which is the expected shape — their reconciliation is the provider's work, and HC3 is the row the Core change owns. --- architecture.md | 14 + packages/acp/tests/host-conversation.test.ts | 314 ++++++++++++++++++ .../core/tests/agent-conversation.test.ts | 15 +- specs/acp-client-spec.md | 17 +- 4 files changed, 356 insertions(+), 4 deletions(-) create mode 100644 packages/acp/tests/host-conversation.test.ts diff --git a/architecture.md b/architecture.md index b748131ae..b76a9f26e 100644 --- a/architecture.md +++ b/architecture.md @@ -2588,6 +2588,20 @@ so one established for another id or another agent refuses rather than reconnecting the wrong chat, and an established conversation never changes agent in place. +Which *native* conversation a chat is also comes from those records and from +nothing beside them. A completed turn retains the exact conversation the +provider said it ran in, so the journal already holds the establishment — which +is what lets a turn the provider accepted before anything acknowledged a mapping +be reconciled to that same conversation instead of leaving the chat looking +unestablished and open to a different one. A provider resolving a different +conversation refuses the next turn before it is sent; one naming a different +conversation only after the turn ran has that outcome refused with the +established identity standing; and a history whose own turns name two of them +cannot be opened, because choosing between them would be a guess. Cancelling a +turn joins it: the call does not finish until the turn and the provider's own +cleanup have finished unwinding, and the conversation admits no second turn +until then — so a host whose cancellation returned knows that work is over. + What a turn ran under is the provider's own account. A placement is made per turn, because a placement settles what it asks before it is routed and is good for exactly one use: the first is fresh and the provider creates the diff --git a/packages/acp/tests/host-conversation.test.ts b/packages/acp/tests/host-conversation.test.ts new file mode 100644 index 000000000..83df04150 --- /dev/null +++ b/packages/acp/tests/host-conversation.test.ts @@ -0,0 +1,314 @@ +/** + * Tier HC — one host conversation over the canonical ACP provider. + * + * `useAgentConversation()` is proven against a stub provider in + * `packages/core/tests/agent-conversation.test.ts`, where a stub is the right + * instrument: those rows are about what the conversation does with what a + * provider says. One thing a stub cannot show is the *order* the canonical + * provider commits in — the adapter accepts the session, the provider's own + * record is promoted to assert an identity, the host's mapping is acknowledged + * second, and the session becomes established last. The window between the + * first two is the one a chat actually has to recover from: the agent is in a + * conversation nobody wrote down. + * + * So these rows install the real provider, with ACPX's runtime replaced by the + * scriptable fake this package already drives it with. The installation is the + * one `installAgentProviderStack` makes — `installAgentComponents` with + * `createAcpxProvider` as the root provider — so what the conversation reaches + * is the product's provider rather than a second account of it, and the host's + * mapping acknowledgement is the one seam a case interrupts. + * + * The companion row at the provider's own level is `provider.test.ts` SM9, + * which shows that window leaves exactly one canonical assertion. These rows + * ask the question a chat asks: after it, what can the conversation still do? + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent } from "@executablemd/durable-streams"; +import { Ok, scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { installAgentComponents } from "@executablemd/core"; +import { useAgentConversation } from "@executablemd/core/host"; +import type { AgentConversation, ExecutionInstallation } from "@executablemd/core/host"; + +import { createAcpxProvider } from "../src/provider.ts"; +import type { AcpxSessionIdentity, AcpxSessionPolicy } from "../src/provider.ts"; +import { createFakeRuntime, makeRegistry, makeStore, useFlatWorld } from "./helpers.ts"; +import type { FakeRuntimeHarness } from "./helpers.ts"; + +const AGENT = "scribe"; +const WORKSPACE = "/workspace"; + +/** The runtime's own directory: provider-owned, and not a checkout. */ +const HOST_DIR = "/runs/sessions/host"; + +/** Where this host puts the one session a chat conversation runs in. */ +const SESSION_DIR = "/runs/sessions/cwd/8f2a"; + +/** The key this host places that session under — a chat, not a directory. */ +const CHAT_SESSION_KEY = "xmd:chat:v1:acpx:scribe-cmd:default"; + +/** The conversation the fake adapter opens for that key, and asserts. */ +const NATIVE = `agent-session:${CHAT_SESSION_KEY}`; + +/** + * One host's agent assembly, with its mapping acknowledgement under a switch. + * + * `place` answers `pending` every time, which is the truthful answer from a + * host whose mapping was never acknowledged: it knows where the session lives + * and has retained nothing about what stands there. `established` is the + * acknowledgement itself, so refusing it once is exactly the interruption these + * rows are about — and the provider's own assertion is already written by then. + * + * The runtime and the store are the host's, held outside the installation, so + * the same provider state survives a conversation being closed and reopened. + */ +interface Host { + /** The installation a conversation is opened with. */ + readonly installation: ExecutionInstallation; + /** The ACPX runtime behind the provider, for what it was asked to do. */ + readonly harness: FakeRuntimeHarness; + /** The ACPX store behind the provider, which outlives each conversation. */ + readonly store: ReturnType; + /** Every identity this host was asked to retain, in order. */ + readonly retained: AcpxSessionIdentity[]; + /** Refuse the next acknowledgement, as a host interrupted before its commit does. */ + refuseNextRetention(): void; +} + +function host(): Host { + const harness = createFakeRuntime(); + const store = makeStore(); + const retained: AcpxSessionIdentity[] = []; + let refusing = false; + const sessions: AcpxSessionPolicy = { + // deno-lint-ignore require-yield + *place() { + return { sessionKey: CHAT_SESSION_KEY, cwd: SESSION_DIR, state: "pending" }; + }, + // deno-lint-ignore require-yield + *established(_placement, identity) { + retained.push(identity); + if (refusing) { + refusing = false; + throw new Error("the host could not retain this session"); + } + }, + }; + return { + harness, + store, + retained, + refuseNextRetention() { + refusing = true; + }, + installation: { + *install(): Operation { + yield* useFlatWorld(WORKSPACE); + yield* installAgentComponents({ + defaultAgent: AGENT, + permissionMode: "deny-all", + rootProvider: { + factory: createAcpxProvider({ + createRuntime: harness.create, + sessionStore: store, + agentRegistry: makeRegistry({ scribe: "scribe-cmd" }), + // deno-lint-ignore require-yield + agentCwd: function* () { + return HOST_DIR; + }, + mcpServers: [], + newSessionOptions: { allowedTools: [], systemPrompt: "none" }, + permissions: "strict", + sessions, + }), + options: { defaultAgent: AGENT, permissionMode: "deny-all" }, + }, + }); + }, + }, + }; +} + +/** Open one conversation over this history, hand it to `body`, and close it. */ +function withConversation( + stream: InMemoryStream, + installation: ExecutionInstallation, + body: (conversation: AgentConversation) => Operation, +): Operation> { + return scoped(function* () { + const opened = yield* useAgentConversation({ + history: stream, + id: "chat-1", + agent: AGENT, + installations: [installation], + }); + if (!opened.ok) { + return opened; + } + return Ok(yield* body(opened.value)); + }); +} + +/** Every canonical prompt record this history holds, in order. */ +function turns( + events: readonly DurableEvent[], +): { name: string; record: Record }[] { + return events.flatMap((event) => { + if (event.type !== "yield" || event.description.type !== "agent_prompt") { + return []; + } + if (event.result.status !== "ok") { + throw new Error(`the retained turn "${event.description.name}" recorded no outcome`); + } + return [ + { name: event.description.name, record: event.result.value as Record }, + ]; + }); +} + +/** The text of every prompt the provider was actually asked to run, in order. */ +function asked(target: Host): string[] { + return target.harness.turns.map((turn) => turn.input.text); +} + +/** One turn's answer, or the failure that stands in a refusal's place. */ +function answer(settled: Result>): string { + if (!settled.ok) { + throw new Error(`the conversation could not be opened: ${settled.error.message}`); + } + return settled.value.ok ? settled.value.value : settled.value.error.message; +} + +describe("Tier HC — a host conversation across the provider's own commit window", () => { + /** + * The window SM9 describes, met by a conversation rather than by a document. + * + * Nothing here is reconciled from the journal, and that is the point: the + * interrupted turn never started, so it retained no identity to reconcile + * against. What the next turn joins is decided by the provider's own record, + * which is the only account of that conversation anyone kept. + */ + it("HC1: a session asserted before its mapping was acknowledged is joined, not replaced", function* () { + const target = host(); + const stream = new InMemoryStream(); + + target.refuseNextRetention(); + const interrupted = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("first"); + }); + + expect(answer(interrupted)).toBe("the host could not retain this session"); + // The pre-commit window, from the chat's side: the provider's own record + // asserts a conversation and is no longer awaiting materialization, while + // the host retained nothing about it. + expect(target.store.records.get(CHAT_SESSION_KEY)?.agentSessionId).toBe(NATIVE); + expect(target.store.records.get(CHAT_SESSION_KEY)?.sessionMaterialization).toBe(undefined); + expect(target.retained.map((identity) => identity.agentSessionId)).toEqual([NATIVE]); + // The turn is retained as the fact it is: it ran in no conversation this + // history can name, so it establishes nothing either way. + expect(turns(yield* stream.readAll()).map((turn) => turn.name)).toEqual(["turn:1"]); + expect(turns(yield* stream.readAll())[0]!.record.agentSessionId).toBe(undefined); + + // The first owner is gone. Explicit new work over the same history. + const continued = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("again"); + }); + + expect(answer(continued)).toBe("hello world"); + // Reconciled to that same assertion and committed, with nothing created in + // its place — and the interrupted turn was not sent a second time. + expect(target.retained.map((identity) => identity.agentSessionId)).toEqual([NATIVE, NATIVE]); + expect([...target.store.records.keys()]).toEqual([CHAT_SESSION_KEY]); + expect(asked(target)).toEqual(["first", "again"]); + const retained = turns(yield* stream.readAll()); + expect(retained.map((turn) => turn.name)).toEqual(["turn:1", "turn:2"]); + expect(retained[1]!.record.agentSessionId).toBe(NATIVE); + }); + + /** + * The same interruption, once the history has an identity of its own. + * + * Here the journal is the account that survives, because a completed turn + * retained the conversation the provider named. The acknowledgement that + * failed changes none of that: the refused turn is its own recorded fact, and + * the conversation is still the one its first turn established. + */ + it("HC2: an acknowledgement refused on an established chat leaves its identity standing", function* () { + const target = host(); + const stream = new InMemoryStream(); + + const opened = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("first"); + }); + expect(answer(opened)).toBe("hello world"); + + target.refuseNextRetention(); + const refused = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("second"); + }); + + expect(answer(refused)).toBe("the host could not retain this session"); + // Refused before the turn reached the backend, so the prompt was never run + // — and the established conversation is still the one the history names. + expect(asked(target)).toEqual(["first"]); + const interrupted = turns(yield* stream.readAll()); + expect(interrupted.map((turn) => turn.name)).toEqual(["turn:1", "turn:2"]); + expect(interrupted[0]!.record.agentSessionId).toBe(NATIVE); + expect(interrupted[1]!.record.status).toBe("failed"); + + const continued = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("third"); + }); + + expect(answer(continued)).toBe("hello world"); + expect(asked(target)).toEqual(["first", "third"]); + expect([...target.store.records.keys()]).toEqual([CHAT_SESSION_KEY]); + const reconciled = turns(yield* stream.readAll()); + expect(reconciled.map((turn) => turn.name)).toEqual(["turn:1", "turn:2", "turn:3"]); + expect(reconciled[2]!.record.agentSessionId).toBe(NATIVE); + }); + + /** + * And what the established identity is *for*. + * + * A provider that answers with another conversation after this history has + * established one is not reconciliation, however plausible the answer looks: + * the turn ran in a history this chat has never been in. The outcome is + * refused to the caller and the established identity stands — which is the + * same rule CV23 states against a stub, asked here of the real seam. + */ + it("HC3: a turn the provider ran in another native conversation is refused", function* () { + const target = host(); + const stream = new InMemoryStream(); + + const opened = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("first"); + }); + expect(answer(opened)).toBe("hello world"); + + // The adapter answers with a different conversation than the one this + // chat's own turn named. + target.harness.assertIdentity = "agent-session:somewhere-else"; + const diverged = yield* withConversation(stream, target.installation, function* (chat) { + return yield* chat.prompt("second"); + }); + + expect(answer(diverged)).toContain("established identity stands"); + // Recorded as what it was — a turn that ran somewhere else — so the host + // reconciles it rather than the next turn inheriting whichever identity + // was read last. + const retained = turns(yield* stream.readAll()); + expect(retained[0]!.record.agentSessionId).toBe(NATIVE); + expect(retained[1]!.record.agentSessionId).toBe("agent-session:somewhere-else"); + + const reopened = yield* withConversation(stream, target.installation, function* () { + return "reached"; + }); + + expect(reopened.ok).toBe(false); + expect(!reopened.ok && reopened.error.message).toContain("more than one native conversation"); + }); +}); diff --git a/packages/core/tests/agent-conversation.test.ts b/packages/core/tests/agent-conversation.test.ts index 7b1adda62..15a69f8e1 100644 --- a/packages/core/tests/agent-conversation.test.ts +++ b/packages/core/tests/agent-conversation.test.ts @@ -779,11 +779,20 @@ describe("Tier CV — the native conversation an established history names", () expect(record(yield* stream.readAll(), "turn:1").agentSessionId).toBe("native-1"); }); - it("CV22: a turn accepted before anything acknowledged a mapping reconciles to that identity", function* () { + /** + * Reopening against the identity the history itself names. + * + * What the next turn is held to comes from the retained turn records and from + * nothing else — this conversation keeps no mapping record of its own, so a + * record is the only account of which conversation it is. The window *before* + * a turn has retained one is the canonical provider's to reconcile, and it is + * proven there: `packages/acp/tests/host-conversation.test.ts` HC1. + */ + it("CV22: reopening joins the identity its retained turns name, without resending them", function* () { const stream = new InMemoryStream(); const first = fresh(); - // The whole history is one accepted turn and nothing after it: there is no - // mapping record, only the provider's own account of the turn. + // One completed turn and nothing after it: the provider's own account of + // that turn is all this history holds about which conversation it is. const opened = yield* withConversation( stream, stub(first, { asserts: () => "native-1" }), diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 685fb822c..85d944639 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -471,9 +471,24 @@ conversation under something else refuses that turn rather than having those settings recorded. `options()` sends no model prompt and journals nothing, and is called on explicit interaction rather than to restore a pane. +**The native conversation an established history names.** A completed turn +retains the exact conversation the provider said it ran in, so the retained +records are what establish a chat's native identity; there is no mapping record +beside them. A provider that resolves a different conversation refuses the next +turn before it is sent, a provider that names a different one only after the +turn ran has that turn's outcome refused with the established identity standing, +and a history whose own turns name two conversations cannot be opened at all. +The window before any turn has retained one belongs to the provider: ACPX's own +record asserts an identity before the host's mapping is acknowledged, and the +next attachment reconciles and commits that same assertion rather than creating +a replacement (§ACPX provider). + One turn runs at a time; an overlapping call refuses rather than queuing, and a handle kept past the scope that opened it refuses because the conversation it -addressed has been cancelled. A refusal before a turn starts — an unavailable +addressed has been cancelled. Cancelling a call joins the turn it started: the +call finishes once that turn and the provider's cleanup inside it have finished +unwinding, and the conversation stays unavailable to another turn until then, so +a completed cancellation is proof that the provider's work has stopped. A refusal before a turn starts — an unavailable agent, an identity the provider will not reattach, settings it would not honour — is that call's answer and leaves the conversation usable. Only a durability failure ends it. From 79d404d04d02c3840892473b6f1b2cf540f3edd1 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Sat, 10 Oct 2026 18:09:08 -0400 Subject: [PATCH 4/4] =?UTF-8?q?=F0=9F=90=9B=20Give=20a=20generated=20root?= =?UTF-8?q?=20an=20environment=20of=20its=20own=20(#883)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A root could not write an entry draft, which is the one thing the following CLI consumer needs it for. The accepted program captures entry source with `` and hands the captured string to a paired control, and admission refused it twice over: `{code}` is a read, and so is the `{names}` inside the captured passive fence. I had written that refusal in as GR17 on the premise that a root inherits nothing and therefore needs the fragment's rule. The premise was wrong in both directions, which is why the fix is two things rather than one. A root *did* inherit the admitting document's bindings — a root evaluated inside a host component rendered `{secret}` as that document's value — so the refusal was load-bearing for disclosure and relaxing it alone would have let a reply read a host binding by naming it, and let a captured draft carry one into an entry. But a root driven from a preparation had no environment to bind into either, so `` had nothing to write to. So a root is now given an environment of its own — empty bindings, empty meta, empty props — installed on the expansion scope, and only then does its interpolation become admissible. The only names it resolves are the ones it bound itself; every other brace renders exactly as written, which is what lets an entry's own braces and nested fences reach the control as the text they are. A fragment is deliberately untouched. It is admitted *into* a document and expands against that document's environment, which is why its rule is that it may not read a binding through interpolation at all. Released fragment behaviour and records are unchanged: 28 tests, 289 steps, as before. GR17 is inverted to what the contract actually says — a root reads a binding it bound itself. GR18–GR21 are the accepted flow: a literal fill reaching the paired receiver exactly once, a captured passive fence, the accepted Each/greeting source with its entry-only references intact and none of it run, and a nested passive fence. GR22 is what the environment is for: a root does not read the bindings of the document that evaluated it, in rendered chat or in a filled draft. The GR9 row asserting that a non-existent binding refuses a root is gone; it is literal text now. Each half is necessary and was checked by removing it: restoring the blanket refusal fails GR17 and GR19–GR22, and removing the environment fails the same rows — the first because admission refuses the capture, the second because `` has nowhere to bind. The reviewer's archived passive-draft probe passes unchanged, as does the cancellation probe. --- architecture.md | 28 ++- packages/core/src/generated-interpolation.ts | 18 +- packages/core/src/generated-xmd.ts | 44 ++++- .../core/tests/generated-xmd-root.test.ts | 165 ++++++++++++++++-- specs/executable-mdx-spec.md | 2 +- 5 files changed, 231 insertions(+), 26 deletions(-) diff --git a/architecture.md b/architecture.md index b76a9f26e..b2a8022e7 100644 --- a/architecture.md +++ b/architecture.md @@ -2480,11 +2480,29 @@ value body to answer — and an `` written below the top level declares nothing in a root either, which is the same lexical rule an authored body is held to. -Nothing else differs. The same whole-source preflight, the same `generated_xmd` -durable admission, the same pinned identities and the same refusal classes, so -there is one walk rather than one per context: the context is a term of the -admission the walk is decided against, beside the table and the request -ceilings. +One other thing differs, and it is the reason the root context exists at all. +A fragment is admitted *into* a document and expands against that document's +binding environment, so a brace it writes could name a binding the document +holds — which is why a fragment may not read a binding through interpolation, +and why that refusal is about disclosure rather than about syntax. A root is a +whole source of its own and is given an environment of its own: empty bindings, +empty meta, empty props. The only names it resolves are the ones it bound +itself, and every other brace renders exactly as written. + +That is what makes a root able to write an entry draft. The accepted way a +reply prepares one is to capture entry source with `` and +hand the captured string to a paired control, and entry source legitimately +carries braces and fences of its own — `` names +nothing the reply has, and must arrive at the control as the text it is. A +blanket refusal would have refused the reply its own capture; an inherited +environment would have let the draft carry the admitting document's bindings +out. Both are refused by the same arrangement rather than by two rules. + +Otherwise nothing differs. The same whole-source preflight, the same +`generated_xmd` durable admission, the same pinned identities and the same +refusal classes, so there is one walk rather than one per context: the context +is a term of the admission the walk is decided against, beside the table and +the request ceilings. **Which context an admission was made in is one of its ceilings.** A retained root admission does not resume as a fragment and a fragment admission does not diff --git a/packages/core/src/generated-interpolation.ts b/packages/core/src/generated-interpolation.ts index 12a00ba09..9020bfc0b 100644 --- a/packages/core/src/generated-interpolation.ts +++ b/packages/core/src/generated-interpolation.ts @@ -8,11 +8,19 @@ * Braces that neither pass would read — prose, a JSON sample, a CSS rule — are * left alone. * - * Its own module because two decisions depend on it and they must agree. - * Admission refuses generated source that reads a binding through - * interpolation; a streaming preview leaves such text out rather than showing a - * person a brace that the finished root will replace. Text one of them called - * literal and the other called a read would be two readings of one rule. + * Its own module because two decisions depend on it and they must agree on + * *what a read is*. A fragment admission refuses source that reads a binding + * through interpolation, because a fragment expands against the environment of + * the document that admitted it. A streaming preview leaves such text out + * rather than showing a person a brace the finished source will replace. Text + * one of them called literal and the other called a read would be two readings + * of one rule. + * + * What the two do with the answer differs, and that is not a disagreement. A + * root admission does not consult this at all: a root has an environment of its + * own, so interpolation there reads only what the root bound and anything else + * stays as written. The preview still treats such text as a read, because + * whether it will change when evaluated is exactly its question. */ const ESCAPED_BRACE_PLACEHOLDER = ""; diff --git a/packages/core/src/generated-xmd.ts b/packages/core/src/generated-xmd.ts index 3bcb3fab7..a78caae13 100644 --- a/packages/core/src/generated-xmd.ts +++ b/packages/core/src/generated-xmd.ts @@ -17,9 +17,11 @@ * does. * * What is refused is a construct *class*: executable code blocks, expression - * props, interpolation that reads a binding, a result binding, a component the - * host did not admit, and a request that is malformed or outside the admitted - * set. The record and the diagnostic name the class and nothing else. Generated + * props, a result binding, a component the host did not admit, and a request + * that is malformed or outside the admitted set. A *fragment* additionally + * refuses interpolation that reads a binding, because it expands against the + * environment of the document that admitted it; a root has an environment of + * its own and reads only what it bound. The record and the diagnostic name the class and nothing else. Generated * source is exactly the text a refusal must not echo — which is why a candidate * request is normalized behind a converting boundary rather than allowed to * report itself, and why a refusal is *returned* by the admission rather than @@ -136,6 +138,8 @@ import { isJsonObject, parseJson } from "./json.ts"; import { markGeneratedRequestRefusal } from "./generated-request-refusal.ts"; import { GeneratedDataExpressions, validateDataExpression } from "./generated-expressions.ts"; import { readsBinding } from "./generated-interpolation.ts"; +import { propsEnvironment } from "./eval-env.ts"; +import { liveEnvironment } from "./live-env.ts"; import { capturedBinding } from "./invocation-rules.ts"; import { renderSegments } from "./render.ts"; import { scanSegments } from "./scanner.ts"; @@ -1979,7 +1983,20 @@ function* walk( for (const segment of segments) { switch (segment.type) { case "text": { - if (readsBinding(segment.content)) { + // A fragment is admitted *into* a document and expands against that + // document's environment, so a brace it writes could name a binding + // the document holds — which is why a fragment may not read one at + // all, and why that refusal is about disclosure rather than about + // syntax. + // + // A root inherits no environment. It begins with empty bindings, empty + // meta and empty props, so the only names it can resolve are the ones + // it bound itself and text naming anything else stays exactly as + // written. Refusing interpolation here would refuse the response its + // own `` capture — and with it the one way a response has of + // writing an entry draft, whose source is entry text that legitimately + // carries braces of its own. + if (admission.context !== ROOT_CONTEXT && readsBinding(segment.content)) { throw new Refusal("interpolation"); } break; @@ -2599,6 +2616,25 @@ function expand( ): Operation { return scoped(function* () { yield* ErrorMode.set(context === ROOT_CONTEXT ? "output" : "throw"); + if (context === ROOT_CONTEXT) { + // A root begins with an empty binding environment, and this is what makes + // that true rather than assumed. Interpolation reads whatever environment + // it finds through the component Api, so a root evaluated inside a host + // component's invocation would otherwise resolve *that* document's + // bindings: a response could read one by naming it, and a captured entry + // draft could carry one out. With an environment of its own the only + // names a root resolves are the ones it bound itself, and every other + // brace renders exactly as written — which is also what lets a captured + // passive fence keep the entry's own braces. + // + // A fragment is deliberately left as it was. It is admitted *into* a + // document and expands against that document's environment, which is why + // its own rule is that it may not read a binding through interpolation at + // all; changing what it sees would change released fragment behavior. + const isolated = propsEnvironment({}); + liveEnvironment(isolated); + yield* Component.around({ env: () => isolated }, { at: "min" }); + } // Every expression this expansion resolves — an ordinary prop and a // declared capture alike — reads the declarative grammar rather than the // trusted-document evaluator. Set on this scope, so it ends with the diff --git a/packages/core/tests/generated-xmd-root.test.ts b/packages/core/tests/generated-xmd-root.test.ts index a7317617d..d645ef330 100644 --- a/packages/core/tests/generated-xmd-root.test.ts +++ b/packages/core/tests/generated-xmd-root.test.ts @@ -37,6 +37,7 @@ import type { FetchInit, RuntimeFetchResponse } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; +import { content } from "../src/component-api.ts"; import { pinnedJson } from "../src/generated-xmd.ts"; import { retainedSource } from "../src/root-source.ts"; import { useTempFileCompiler } from "../src/temp-file-compiler.ts"; @@ -461,12 +462,6 @@ describe("Tier GR — the root context admits one construct more, and no others" "declarative data", { observations: [countedProbe([])] }, ], - [ - "interpolation of a binding that does not exist", - "\nHello, {who}!\n\n", - "reads a binding", - {}, - ], ["", "\n\n\n", "structural construct", {}], ["", '\nok\n\n\n\n', "structural construct", {}], [ @@ -665,13 +660,161 @@ describe("Tier GR — the trusted composition table reaches a root too", () => { expect(COMPOSITION_NAME).toBe("Json"); }); - it("GR17: a root still may not read a binding through text interpolation", function* () { + it("GR17: a root reads a binding it bound itself", function* () { const attempt = yield* evaluateRoot( - request('\n\n\n{who}\n\n'), + request('\n\n\nHello, {who}!\n\n'), + ); + + // A fragment may not read a binding through interpolation, because it + // expands against the environment of the document that admitted it. A root + // has an environment of its own, so the only thing it can reach by naming + // is what it bound — and that is the whole of why this differs. + // Quoted because `` binds the JSON value, which is what the same + // source renders to in an ordinary document. + expect(rendered(attempt).output).toContain('Hello, "Ada"!'); + }); +}); + +/** + * Tier GR — writing the entry draft from captured passive source. + * + * This is the one program the following CLI consumer is built around, and it is + * the reason a root's interpolation rule cannot be the fragment's. A response + * captures entry source with ``, which renders its body and + * selects the code node out of it, then hands that literal string to the + * paired control that fills the shared draft. + * + * Everything load-bearing about it is a *literal*: the captured text is entry + * source, not response syntax, so the entry's own braces and fences have to + * arrive at the receiver exactly as the Agent wrote them, nothing inside the + * fence may be executed, and the receiver must be handed that string once. + */ +describe("Tier GR — a root fills an entry draft with literal captured source", () => { + beforeAll(() => useTempFileCompiler()); + + /** The paired receiver, recording exactly what each invocation was handed. */ + function receiver(fills: string[]): GeneratedMutation { + return pinnedMutation( + "REPL.EntryInput", + hostIdentity("test://sidekick", "EntryInput"), + { + kind: "function", + name: "REPL.EntryInput", + props: NO_PROPS, + *fn(): Operation { + fills.push(String(yield* content())); + return "Entry input updated."; + }, + }, + "paired", + ); + } + + function fill(source: string): Operation<{ attempt: Attempt; fills: string[] }> { + return scoped(function* () { + const fills: string[] = []; + const attempt = yield* evaluateRoot( + request(source, { allow: ["read", "write"], mutations: [receiver(fills)] }), + ); + return { attempt, fills }; + }); + } + + it("GR18: a literal body reaches the receiver exactly once", function* () { + const { attempt, fills } = yield* fill("Hello\n"); + + expect(answered(attempt).ok).toBe(true); + expect(fills).toEqual(["Hello"]); + }); + + it("GR19: a captured passive fence fills the draft with its own text", function* () { + const { attempt, fills } = yield* fill( + '\n```xmd\nHello\n```\n\n' + + "{code}\n", + ); + + expect(answered(attempt).ok).toBe(true); + expect(fills).toEqual(["Hello"]); + }); + + it("GR20: the entry's own braces survive the capture, and none of it runs", function* () { + const entry = '\nHello, {name}!\n'; + const { attempt, fills } = yield* fill( + '\n```xmd\n' + + entry + + "\n```\n\n{code}\n", + ); + + expect(answered(attempt).ok).toBe(true); + // `names` and `name` belong to the entry this draft will run as. They are + // not response bindings and do not need to be: a root resolves only what it + // bound, so an entry-only reference arrives as the text it is. An `` + // the response had actually expanded would have rendered its body instead + // of handing the element over. + expect(fills).toEqual([entry]); + }); + + /** + * What the root's own environment is *for*. + * + * A root that resolved the admitting document's bindings would let a response + * read one by naming it, and would let a captured draft carry one out — so + * the rule that a root may interpolate at all depends on this being true. + * Driven through a host component rather than a preparation, because a + * preparation has no document bindings to be isolated from. + */ + it("GR22: a root does not read the bindings of the document that evaluated it", function* () { + const fills: string[] = []; + let answer: Result | undefined; + const execution = yield* executeInstalled( + { + ...retainedSource("repl/host.md", '\n\n'), + stream: new InMemoryStream(), + }, + [ + { + components: [ + { + name: "SidekickHost", + origin: "test://sidekick-host", + props: NO_PROPS, + factory: () => + function* (): Operation { + answer = yield* evaluateGeneratedXmdRoot( + request( + "[{secret}]\n" + + "[{secret}]\n", + { allow: ["read", "write"], mutations: [receiver(fills)] }, + ), + ); + return ""; + }, + }, + ], + }, + ], + ); + const result = yield* execution; + + expect(result.ok).toBe(true); + expect(answer?.ok).toBe(true); + // Literal in both places: what renders as chat, and what a draft would + // carry into an entry. + expect(answer?.ok === true && answer.value.output).toContain("[{secret}]"); + expect(fills).toEqual(["[{secret}]"]); + }); + + it("GR21: a nested passive fence survives the capture", function* () { + // The outer fence has to be longer than the one it contains, which is the + // ordinary Markdown rule and the one the accepted instructions state. + const entry = "Run this:\n\n```sh\nxmd run greeting.md\n```"; + const { attempt, fills } = yield* fill( + '\n````xmd\n' + + entry + + "\n````\n\n{code}\n", ); - // The ceiling a fragment is held to, unchanged: the root context supplies - // `` and widens nothing else. - expect(failed(attempt).message).toContain("reads a binding"); + expect(answered(attempt).ok).toBe(true); + expect(fills).toEqual([entry]); }); }); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 15a5ad256..172c4304a 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -1913,7 +1913,7 @@ run but are absent from the diagnostic trace. | `src/fetch-request.ts` | `prepareFetchRequest()`, `FetchRequest`, `FetchRequestError` — what `` admits before transport (§6.18) | | `src/fetch-response.ts` | `detachHeaders()`, `detachStatus()`, `FetchResponseRecord` — the response detached from the provider's (§6.18) | | `src/fetch-journal.ts` | `persistFetch()` — the `fetch` durable effect (§6.18, §10.1) | -| `src/generated-xmd.ts` | `evaluateGeneratedXmd()`, `pinnedFetch()`, `pinnedFileRead()`, `pinnedFileWrite()`, `pinnedFileDelete()`, `pinnedComponent()`, `pinnedMutation()`, `GeneratedEffectClass`, `GeneratedComponentForm`, `GeneratedMutation`, `GeneratedObservation`, `GeneratedRequest`, `GeneratedXmdError` — admitting Agent-generated source through the trusted-host seam under the always-on trusted composition table and a caller-selected subset of the closed effect classes `read` and `write`, resolving each name and authored form to one exact pinned identity — core Json, the self-closing `` read, the paired `` write and the self-closing `` among them — and preserving every component's ordinary binding and output behavior. `evaluateGeneratedXmd()` returns `Operation` containing the fragment's rendered text. `evaluateGeneratedXmdRoot()` admits the same source as a **root** — the one context that supplies a top-level ``, with `` and `` unavailable in both — and returns `Operation>` stating the canonical rendering and whether the root declared a region at all. Which context an admission was made in is a term of its retained policy and is compared whole, so a root admission never resumes as a fragment or the reverse; a fragment record states no context, which is the shape every record written before roots existed carries. An ordinary failure answers `Err(GeneratedXmdRootError)` carrying the rendering the root had produced, while durability, Files-infrastructure, secret-policy and teardown failures keep their own classification. Its admission and every generated effect use occurrence-owned identities in the existing ordinary durable sequence, with no `ephemeral()` bridge, staging or result envelope | +| `src/generated-xmd.ts` | `evaluateGeneratedXmd()`, `pinnedFetch()`, `pinnedFileRead()`, `pinnedFileWrite()`, `pinnedFileDelete()`, `pinnedComponent()`, `pinnedMutation()`, `GeneratedEffectClass`, `GeneratedComponentForm`, `GeneratedMutation`, `GeneratedObservation`, `GeneratedRequest`, `GeneratedXmdError` — admitting Agent-generated source through the trusted-host seam under the always-on trusted composition table and a caller-selected subset of the closed effect classes `read` and `write`, resolving each name and authored form to one exact pinned identity — core Json, the self-closing `` read, the paired `` write and the self-closing `` among them — and preserving every component's ordinary binding and output behavior. `evaluateGeneratedXmd()` returns `Operation` containing the fragment's rendered text. `evaluateGeneratedXmdRoot()` admits the same source as a **root** — the context that supplies a top-level ``, with `` and `` unavailable in both — and returns `Operation>` stating the canonical rendering and whether the root declared a region at all. A root is also given its own binding environment — empty bindings, empty meta, empty props — so interpolation there resolves only what the root itself bound and every other brace renders as written; a fragment expands against the environment of the document that admitted it and therefore keeps its refusal of interpolation that reads a binding. That is what lets a root hand literal captured source, braces and nested passive fences included, to a pinned paired control. Which context an admission was made in is a term of its retained policy and is compared whole, so a root admission never resumes as a fragment or the reverse; a fragment record states no context, which is the shape every record written before roots existed carries. An ordinary failure answers `Err(GeneratedXmdRootError)` carrying the rendering the root had produced, while durability, Files-infrastructure, secret-policy and teardown failures keep their own classification. Its admission and every generated effect use occurrence-owned identities in the existing ordinary durable sequence, with no `ephemeral()` bridge, staging or result envelope | | `src/glob-source.ts` | `GLOB_PROPS`, `GLOB_RETURNS`, `globPatterns()`, `globFailure()` — the props, return contract, source rules and sanitized failure sentence one search is held to, shared by the ordinary `` component and the Glob capability an evaluation profile admits, so one dialect answers both (§6.14) | | `src/components/component-resolution.ts` | `CanonicalImports`, `ComponentResolution` — the witness a closed execution issues for the definition it produced and verifies where it is invoked | | `src/invocation.ts` | `withInvocation()`, `Invocation`, `InvocationTeardownError` — the component invocation boundary (§4.4) |