diff --git a/.gitignore b/.gitignore index c0c861c5c..df86be375 100644 --- a/.gitignore +++ b/.gitignore @@ -27,3 +27,10 @@ packages/web/generated/ # that walked one would report on a file nobody wrote. site/*.timestamp-*.mjs .bench/ + +# Session scratch: a test log and the break-control record are evidence, and +# evidence belongs under .reviewer/ rather than in the product commit. +.cliall.log +.cli.log +.controls.json +.controls.py diff --git a/architecture.md b/architecture.md index e1f2eea95..a490c995a 100644 --- a/architecture.md +++ b/architecture.md @@ -2821,6 +2821,53 @@ actions is closed so every one is journalable. ``` +### 1a. One element's expansion can be surrounded + +`Component.expand` offers an observer the whole of one executable element's +expansion: before resolution and props validation, through the accepted body +and its owned cleanup, to the acceptance of what it produced. Core issues a +request and the public chain composes around it; a handler delegates that exact +request once with `next(request)`. + +The authority is the shape bound execution already uses. The request is branded +with a private field, so a copy carrying the same members is not it; the claim +is spent once, so a repeated delegation runs nothing; and the settlement is +readable whether or not the chain returned normally, so a handler that catches +what canonical expansion raised has not rescued it. A handler that returns +without delegating refuses the work, and what a handler returns is ignored. + +Observation reaches no outcome. Phases carry `Result` and nothing else — +what the element produced, what it bound and which error identity core is +holding stay private — and a failure is reported as a detached, frozen `Error` +preserving the selected name, message and explanatory causes. Completion is +published only after the whole dispatch, middleware cleanup included, has +unwound and canonical acceptance is reconciled. + +Each subscription is registered with the element's latest phase and then told +each change in order, ending after exactly one terminal observation. A reader +that is slow, absent or cancelled delays nothing and cancels nothing: execution +never waits for an observer, and a consumer that must see the terminal phase +belongs to an owner outliving the dispatch rather than to the handler's frame. + +Every path that expands something an author wrote crosses this once: the +component paths, and the structural constructs ``, ``, ``, +`` and ``, plus the selected ``, the `` that runs and +each `` an `` starts. An unselected branch expands nothing and is +told nothing. + +### 1b. Where a text's elements are, without running it + +`inspectSource(text, kind)` answers where each recognized element's delimiters +are and what it is called. It is the scanner's own reading, collected by the +same walk that decides what a fence, an inline code span, a quoted `>` and a +tag-like expression are, so there is no second grammar to disagree with the +first — and a nested collector is committed only with the parse that found it. + +A document's body boundary is read lexically, matching the installed +extractor's envelope rule without calling its value parser, so inspecting a +document interprets neither its header nor its body. Definition parsing asserts +the two boundaries agree. + ### 2. Decide once An error no middleware handles is decided exactly once, where it is raised, @@ -5336,6 +5383,8 @@ Status is measured against main. | `` region `output` mode | an undecided error fails the document execution | built on main | | `` rendering selection | chooses which regions of a body render, and buffers a root that declares one; it decides nothing about failure | built on main | | `Expansion` / `getExpansion()` | describes the current logical element expansion | built on main | +| `Component.expand` | surrounds one element's complete expansion — before resolution, through the accepted body and its owned cleanup, to the acceptance of what it produced. Core issues a branded request the public chain composes around; a handler delegates it once, may refuse the work by throwing or by not delegating, and decides nothing about the outcome. Phases carry `Result` and a detached, frozen report, so no canonical identity, binding or resource crosses. Completion follows the whole dispatch, middleware cleanup included. Each subscription is registered with the latest phase and then told each change in order, ending after one terminal observation; a slow, absent or cancelled reader delays and cancels nothing | built on this stack | +| `inspectSource()` | answers where a text's executable elements are written and what they are called, without resolving, compiling, evaluating or running any of it. The scanner's own reading, so fences, inline code, quoted delimiters and tag-like expressions are decided once; a document's body boundary is read lexically, without the frontmatter value parser, and definition parsing asserts the two agree | built on this stack | | document targets | catalogs a root document's addressable static headings, resolves one selector to one exact target, and projects the document to it before expansion | built on the #412 stack | | document-aware `xmd run … --help` | describes what one document declares and every target it addresses, each as a full document reference with the description its section states, by inspection alone | built on the #463 stack | | standard-input root documents | `xmd run -` and `xmd run -- -` read the whole root document from standard input, once, to end of file, and run it through the ordinary run profile. Fixed grammar selects it — the explicit `run` command form plus a document argument that is exactly `-`, read from the parser's own unconsumed remainder so a `-` another option took as its value is not one — and every other spelling keeps the meaning it had: the shorthand `xmd -` executes the file named `-`, `xmd run -#Section` executes that file's `Section`, another command's `-` is that command's, and `--eval -` keeps its refusal. `-` is the one filename the option grammar leaves unwritable, so the reference grammar reaches it and nothing else beginning with `-` is read as a document. The parsed path and every recovered reference stay separate facts until the grammar is settled, so a command line naming two roots refuses in either order, before the read and before either candidate is inspected. The reader is a value each runtime-named entrypoint supplies and the shared CLI never reaches a stdin global; what comes back is `retainedSource("", source)`, adding no root-source variant, constructor, digest member or public API. The complete input is acquired before inspection, provider setup, the secret-detection announcement, journal creation, root admission and execution, inside the run's existing deadline; a failed read is one fixed sentence carrying no host error, input or path, and cancellation tears the reader down without becoming one | built on this stack | @@ -5696,6 +5745,9 @@ DurableEvents -> frozen ReplModel -> resolved immutable view | `description.ts` | opaque immutable descriptions and the closed action boundary | | `reconcile.ts` | reconciling descriptions into one mounted Freedom tree | | `handoff.ts` | the acknowledged commit boundary | +| `source-reading.ts` | one entry as output and then source, with what each element is doing | +| `presentation-text.ts` · `presentation-style.ts` | what the characters of a reading are, and the colours it is drawn in | +| `fitting.ts` | engine-measured widths, and the rows a reading is cut into | | `layout.ts` | deterministic placement at four sizes | | `renderer.ts` | drawing the mounted tree, and the frame map | | `frame.ts` | the one acknowledged frame stream | diff --git a/packages/cli/src/repl/agent.ts b/packages/cli/src/repl/agent.ts index 71153a8e2..964ed8524 100644 --- a/packages/cli/src/repl/agent.ts +++ b/packages/cli/src/repl/agent.ts @@ -72,6 +72,7 @@ import type { AgentPromptPublisher, ExecutionInstallation, } from "@executablemd/core/host"; +import type { ReplWaits } from "./lifecycle.ts"; import { DurableContext } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; @@ -426,7 +427,18 @@ function remove(attachment: Attachment, request: LiveRequest): void { * so no one is woken. No teardown path appends a record, writes an audit or * closes a coroutine. */ -export function useReplAgent(mode: PermissionMode): Operation { +export function useReplAgent( + mode: PermissionMode, + /** + * Where a pending request says the element asking for it is waiting, or none. + * + * Optional because the authority is complete without it: counting a wait is + * something a reading wants, not something deciding a permission needs. A + * kernel given none counts nothing, which is what an execution with no + * session reading behaves like. + */ + waits?: ReplWaits, +): Operation { return resource(function* (provide) { const changes = createSignal(); /** @@ -642,6 +654,15 @@ export function useReplAgent(mode: PermissionMode): Operation { // Decided before anything is published: an unowned request publishes no // reading and no key, and never becomes a denial. const held = owner(attachment, yield* currentCoroutine()); + // The element this request is being asked on behalf of is waiting from + // here until somebody answers. Released in `ensure`, so abandoning the + // request at teardown releases the wait without inventing a decision — + // and so a request that is never answered leaves no element reading as + // waiting after its scope has gone. + const release = waits === undefined ? undefined : yield* waits.hold("permission"); + if (release !== undefined) { + yield* ensure(release); + } return yield* action(function (resolve) { let settled = false; const live: LiveRequest = { diff --git a/packages/cli/src/repl/application.ts b/packages/cli/src/repl/application.ts index c9f01b454..99ae90ba6 100644 --- a/packages/cli/src/repl/application.ts +++ b/packages/cli/src/repl/application.ts @@ -82,6 +82,10 @@ import type { ReplAdmission, ReplWindow } from "./layout-admission.ts"; import { ORDINARY, REPL_PALETTE, styleOf } from "./presentation-style.ts"; import type { ReplPresentationRole, ReplRowStyle } from "./presentation-style.ts"; import { jsonRuns, sourceRuns } from "./presentation-text.ts"; +import { entryReading, NO_READING } from "./source-reading.ts"; +import type { ReplReading } from "./source-reading.ts"; +import { readingRuns } from "./fitting.ts"; +import type { ReplFittedLine, ReplPreparedReading, ReplPreparedRow } from "./fitting.ts"; import type { ReplTerminalSize } from "./terminal.ts"; import { decodeLocation, @@ -127,6 +131,8 @@ import { SELECT_ROW, } from "./components/rows.ts"; import type { ReplAction } from "./components/actions.ts"; +import { NO_LIFECYCLE } from "./lifecycle.ts"; +import type { ReplLifecycleReading } from "./lifecycle.ts"; export type { ReplAction }; @@ -156,6 +162,15 @@ export interface ReplLive { * live reading describes work this process is doing, and none of those is. */ readonly agent: ReplAgentReading; + /** + * What this process is doing element by element, for the entry that is open. + * + * Empty wherever a live reading has nothing to say: a replay, a document + * with no element in flight, and every frozen prefix. What is here is what + * this process watched happen, which is why retained syntax and a recorded + * admission add nothing to it. + */ + readonly lifecycle: ReplLifecycleReading; } /** @@ -766,6 +781,10 @@ export function viewFor( // a fact about the position being inspected. running: false, agent: NO_AGENT, + // And the elements this process is expanding are the head's, not + // this position's: a reading frozen before them describes a file + // that had not reached them yet. + lifecycle: NO_LIFECYCLE, }; return Ok( Object.freeze({ @@ -809,6 +828,7 @@ export function refusedView( pausable: false, running: false, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }), location: encodeLocation(state.route), refusal: reason, @@ -1082,6 +1102,28 @@ export function reduceRepl( refusal: undefined, }); } + case "scroll-reading": { + // The same rule again, over the window the frame measured for the reading + // that is showing. Which reading that is decides the offset this moves, + // so selecting an earlier entry and scrolling it leaves the live entry's + // own position where the reader left it. + const window = admission.windows.get(READING_WINDOW); + if (window === undefined) { + return settled({ ...state, refusal: undefined }); + } + const key = entryReadingKeyOf(state); + return settled({ + ...state, + viewports: Object.freeze({ + ...state.viewports, + readings: Object.freeze({ + ...state.viewports.readings, + [key]: scrolled(window, action.delta), + }), + }), + refusal: undefined, + }); + } case "select-permission": { // The key is taken first, because a `+permission` candidate resolves only // while the request it names is pending — so the state that navigates has @@ -2339,7 +2381,7 @@ function described(view: ReplView, context: ReplPresentationContext): DescribedS items.push( row( "sessions:earlier", - " [^ earlier]", + ` ${EARLIER}`, { select: "scroll-sessions", delta: -1 }, styleOf("action"), { @@ -2351,7 +2393,7 @@ function described(view: ReplView, context: ReplPresentationContext): DescribedS items.push( row( "sessions:later", - " [v later]", + ` ${LATER}`, { select: "scroll-sessions", delta: 1 }, styleOf("action"), { @@ -2381,7 +2423,7 @@ function described(view: ReplView, context: ReplPresentationContext): DescribedS items.push( row( "entries:earlier", - " [^ earlier]", + ` ${EARLIER}`, { select: "scroll-entries", delta: -1 }, styleOf("action"), { here: view.focused }, @@ -2393,7 +2435,7 @@ function described(view: ReplView, context: ReplPresentationContext): DescribedS items.push( row( "entries:later", - " [v later]", + ` ${LATER}`, { select: "scroll-entries", delta: 1 }, styleOf("action"), { here: view.focused }, @@ -2421,33 +2463,48 @@ function described(view: ReplView, context: ReplPresentationContext): DescribedS // the same reading. With nothing selected the whole execution is the locus, // which is what a one-entry execution has always shown. const inspectable = showEntry; - // Not until a width has been measured. A transcript row is bounded to the - // region it lands in, and an unbounded one makes that region wider than its - // share — which would corrupt the very measurement that is about to answer - // how wide it is. + // Not until a width has been measured. A reading row is bounded to the region + // it lands in, and an unbounded one makes that region wider than its share — + // which would corrupt the very measurement that is about to answer how wide + // it is. const surface = context.widths?.surface; - for (const [index, transcript] of inspectable && surface !== undefined - ? transcriptOf(model, selection).entries() - : []) { - // One cell is one row, so a recorded row that holds several lines of output - // becomes several cells. A cell given more than one line would show only the - // first, which is the whole of what a reader would then believe was there. - for (const [offset, part] of transcriptLines(transcript, surface ?? 0).entries()) { - items.push(line(`line:${index}:${offset}`, part.text, part.style)); - } - } - // The live overlay, explicitly below the recorded rows and explicitly labelled. - // Once the durable close exists its recorded output is in the transcript and - // this is empty, so the two never both claim to be the output. - // - // It belongs to the entry producing it, which is the last one this prefix - // admitted — nothing earlier can still be running. A reader who has selected - // an earlier entry is reading a settled transcript, and text from a run that - // is not the one they are looking at would be attributed to it. - if (inspectable && live.output.length > 0 && livesHere(model, selection)) { - for (const [offset, text] of live.output.split("\n").entries()) { - items.push(line(`line:live:${offset}`, `… ${text}`, styleOf("output"))); + // The entry's own reading: what it produced, then the source that produced + // it, as the rows the frame fitted to this pane. The controls that move it sit + // outside the moving content, which is why they are described before the rows + // rather than among them. + const prepared = inspectable && surface !== undefined ? context.reading : undefined; + // Only where there is a reading to move. An execution that has admitted + // nothing has no output and no source, and two controls over an empty pane + // would be Tab stops that scroll nothing — read from the model rather than + // from the fitted rows, because the pass that reserves these controls is the + // pass that has not fitted any. + if (inspectable && model.entries.length > 0) { + // Both controls, always, exactly as the Sessions reading does it. They + // consume their capacity *before* the rows do: the pass that measures how + // many rows the window holds has to be the pass that already has them in + // it, or the window is measured two rows too tall and the control it then + // grows is drawn over the last row of the reading. Measured: it was. + items.push( + row( + "reading:earlier", + ` ${EARLIER}`, + { select: "scroll-reading", delta: -1 }, + styleOf("action"), + { here: view.focused }, + ), + ); + if (prepared !== undefined) { + items.push(...shown(readingRows(prepared), context, READING_WINDOW)); } + items.push( + row( + "reading:later", + ` ${LATER}`, + { select: "scroll-reading", delta: 1 }, + styleOf("action"), + { here: view.focused }, + ), + ); } // The inspection controls, and only where there is an inspection region to put @@ -2694,6 +2751,45 @@ function shown( return content.slice(held.from, held.from + held.count); } +/** + * One prepared reading as the rows a description places. + * + * Each row's runs are the composed ones — rail, indent, text, and the badge + * against the pane's right inner edge — so what a description says and what the + * frame measured are one list rather than two that have to agree. + */ +function readingRows(prepared: ReplPreparedReading): readonly Described[] { + return prepared.rows.map((one) => { + const runs = readingRuns(prepared, one); + return line(`reading:${one.key}`, runText(runs), one.style, runs); + }); +} + +/** + * The reading one view holds, before any width has fitted it. + * + * The selected entry's own, and nothing when a prefix admitted none. Live text + * belongs to the entry producing it — the last one this prefix admitted, since + * entries are serial — so a reader looking at an earlier entry is given that + * entry's retained reading rather than text from the run that is still going. + */ +export function readingOf(view: ReplView): ReplReading { + const key = view.selection.entry?.key; + const entry = + key === undefined + ? view.model.entries[view.model.entries.length - 1] + : view.model.entries.find((one) => one.key === key); + if (entry === undefined) { + return NO_READING; + } + return entryReading({ + entry, + lifecycle: view.live.lifecycle, + live: livesHere(view.model, view.selection) ? view.live.output : "", + inspected: view.state.route.at !== undefined, + }); +} + /** * The transcript rows the selected locus holds. * @@ -3074,7 +3170,7 @@ function drawerFor( exit: Described, drawable: boolean, ): { readonly drawer: Described; readonly rows: readonly Described[] } | undefined { - const held = drawerContent(view, context.widths?.drawer ?? 0); + const held = drawerContent(view, context.widths?.drawer ?? 0, context.preview); if (held === undefined) { return undefined; } @@ -3084,7 +3180,7 @@ function drawerFor( rows.push( row( "drawer:scroll:up", - padControl("[^ earlier]", width), + padControl(EARLIER, width), { select: "scroll", delta: -1 }, styleOf("action"), { here: view.focused }, @@ -3096,7 +3192,7 @@ function drawerFor( rows.push( row( "drawer:scroll:down", - padControl("[v later]", width), + padControl(LATER, width), { select: "scroll", delta: 1 }, styleOf("action"), { here: view.focused }, @@ -3165,7 +3261,35 @@ interface DrawerContent { * many rows there are to window. A second counter is how a clamp comes to * disagree with the list it is clamping. */ -function drawerContent(view: ReplView, width: number): DrawerContent | undefined { +/** + * The rows a question's message is read in. + * + * The frame's fitted rows where it prepared them, and the logical lines where + * it has not — the pass that measures the drawer's width cannot already have + * fitted text to it. An unfitted pass describes no window content anyway, so + * the fallback is what a measurement sees and never what a reader does. + */ +function messageRows( + question: ReplQuestion, + preview: readonly ReplFittedLine[] | undefined, +): readonly { readonly text: string; readonly runs: readonly ReplTokenRun[] }[] { + if (preview !== undefined) { + return preview.map((one) => ({ + text: " ".repeat(one.indent) + one.text, + runs: + one.indent === 0 + ? one.runs + : tokenRuns([{ text: " ".repeat(one.indent), token: "punctuation" }, ...one.runs]), + })); + } + return question.message.split("\n").map((text) => ({ text, runs: sourceRuns(text) })); +} + +function drawerContent( + view: ReplView, + width: number, + preview?: readonly ReplFittedLine[] | undefined, +): DrawerContent | undefined { const open = view.selection.drawers[view.selection.drawers.length - 1]; if (open === undefined) { return undefined; @@ -3287,14 +3411,14 @@ function drawerContent(view: ReplView, width: number): DrawerContent | undefined // [submit] — is built in order and then windowed. A drawer too short to hold // all of it scrolls, rather than describing rows that have no frame to be // placed in. - for (const [offset, text] of question.message.split("\n").entries()) { + for (const [offset, part] of messageRows(question, preview).entries()) { content.push({ described: drawerLine( `drawer:message:${offset}`, - text, + part.text, width, styleOf("source"), - sourceRuns(text), + part.runs, ), }); } @@ -3552,30 +3676,6 @@ function lasting(kind: ReplLiveChoice["kind"]): string { return kind === "allow_always" || kind === "reject_always" ? " for this Agent session" : ""; } -/** - * How many rows the drawer's ordered content holds in total. - * - * The complete message, the form's description, every field with its - * annotation, enum summary, offered values and editable line, every validation - * message, and `[submit]`. Counted the same way the rows are built, because the - * reducer clamps a scroll against this before any of them exist. - */ -function drawerContentRows(question: ReplQuestion | undefined, form: ReplFormState): number { - if (question === undefined) { - return 0; - } - let rows = question.message.split("\n").length; - rows += question.form.description === undefined ? 0 : 1; - for (const one of question.form.fields) { - rows += 2; - rows += one.description === undefined ? 0 : 1; - rows += one.choices === undefined ? 0 : 1 + one.choices.length; - } - rows += form.messages.length; - // [submit] scrolls with the form it submits. - return rows + 1; -} - /** The first line of a message, for a control that is one row tall. */ /** * One drawer's title: what kind of reading it is, and which one. @@ -3917,12 +4017,27 @@ function failedLine(message: string, width: number): string { return `${line.slice(0, Math.max(FAILED_PREFIX.length, width - 1))}…`; } +/** + * What the two window controls are called, wherever a reading has a window. + * + * One spelling, so Sessions, the catalog, a drawer and the entry reading + * cannot drift apart — and arrows rather than the ASCII they replaced, which + * is what the accepted presentation states. The semantic actions behind them + * are unchanged, and no keyboard shortcut is introduced: these are controls a + * person activates the way they always did. + */ +export const EARLIER = "[\u2191 earlier]"; +/** The other one. */ +export const LATER = "[\u2193 later]"; + /** The Sessions reading's window. */ export const SESSIONS_WINDOW = "sessions"; /** The Entries catalog's window. */ export const ENTRIES_WINDOW = "entries"; /** The open drawer's content window, whichever reading it is showing. */ export const DRAWER_WINDOW = "drawer"; +/** The entry reading's window: output and source together, in the transcript. */ +export const READING_WINDOW = "reading"; /** The action row's own structural id, which admission measures against. */ export const ACTION_ROW = "box:footer:actions"; @@ -3992,6 +4107,25 @@ export interface ReplPresentationContext { * turning this off must not make a control behind the modal reachable. */ readonly capture: "capture" | "passthrough"; + /** + * The entry reading this frame prepared, or none before it has prepared one. + * + * One immutable list of fitted rows, built once per frame from engine answers + * and shared by everything downstream: the row count, the window's capacity, + * the descriptions and the boxes all read it, so a prepared row and a drawn + * row cannot disagree about what was prepared. The measuring pass has none, + * which is why it describes none. + */ + readonly reading: ReplPreparedReading | undefined; + /** + * The open question's message, fitted to the drawer's own width. + * + * Prepared beside the reading and for the same reason: a drawer row is one + * row, so a long line of somebody's preview can only be shown whole if it + * was cut into rows before the window counted them. Scrolling reveals + * prepared rows; it cannot recover the rest of an unprepared one. + */ + readonly preview: readonly ReplFittedLine[] | undefined; } /** One reading's descriptions, paired with where the engine places them. */ @@ -4009,6 +4143,26 @@ export interface ReplPresentation { * at. The live question and a pending permission have their own offsets already, * so they have no key here. */ +/** + * The key the entry reading's own offset is stored under. + * + * The execution, the prefix being read and which entry's reading it is — so + * selecting an earlier entry finds that entry's own position again rather than + * wherever the last one was left, and the same entry read at two history + * positions is two readings of two different files. + * + * Process-local, like every other window: nothing durable holds one, so a + * location opened in another process starts each reading at its first row. + */ +export function entryReadingKeyOf(state: ReplState): string { + // `scopes[0]` is the selected entry's key, which is the route's own answer to + // which entry this is — read from the route rather than resolved against a + // model, because a reducer answering a keystroke has no business reprojecting + // a prefix to learn which window it is moving. + const entry = state.route.scopes[0] ?? "execution"; + return `reading:${state.route.execution}:${state.route.at ?? "head"}:${entry}`; +} + export function readingKeyOf(state: ReplState, open: ReplDrawerRef): string | undefined { // The prefix being read is part of the identity: the same binding at two // history positions is two readings of two different files. @@ -4063,6 +4217,8 @@ type ReplSlot = | "sessions" | "entries-fixed" | "entries" + | "reading-fixed" + | "reading" | "transcript" | "inspection" | "drawer-above" @@ -4119,6 +4275,14 @@ function slotOf(key: string): ReplSlot | undefined { if (key.startsWith("entries:") || key.startsWith("entry:") || key.startsWith("scope:")) { return "entries"; } + // The reading's own window controls stay put around it, so a measured + // viewport is the moving part alone. + if (key === "reading:earlier" || key === "reading:later") { + return "reading-fixed"; + } + if (key.startsWith("reading:")) { + return "reading"; + } if (key.startsWith("line:")) { return "transcript"; } @@ -4429,10 +4593,38 @@ export function presentationFor( // every region would be a wide screen with the columns removed: the reader // would scroll past three lists to reach the one they asked for, and every // row of the other two would still be a target. + // + // The Entries surface is the catalog *and* the reading of whichever entry + // it has selected — the same two things the sidebar and the transcript are + // at a wider size, stacked because there is one column to put them in. A + // narrow frame that showed the catalog alone would route a reader to the + // surface their entry is on and then show them nothing of it. + // + // The catalog takes its own height while it fits and its share when it does + // not, exactly as it does in the sidebar; the reading takes what is left. const routed = view.state.route.surface === "sessions" ? listColumn("content", "sessions-fixed", "sessions", SESSIONS_WINDOW) - : listColumn("content", "entries-fixed", "entries", ENTRIES_WINDOW); + : [ + box({ + id: "box:content:entries", + region: "content", + props: stackProps(context.entriesRows === undefined ? sharedColumnRows(size) : "fit"), + children: listColumn( + "content", + "entries-fixed", + "entries", + ENTRIES_WINDOW, + context.entriesRows, + ), + }), + box({ + id: "box:content:reading", + region: "content", + props: stackProps("grow"), + children: listColumn("content", "reading-fixed", "reading", READING_WINDOW), + }), + ]; regions.push({ region: "content", id: "box:content" }); columns.push( box({ @@ -4501,14 +4693,7 @@ export function presentationFor( edgedColumn("transcript", undefined, REPL_PALETTE.centreSurface, [ ...paneTitle("transcript:heading", "transcript"), ...located, - box({ - id: "box:transcript:viewport", - region: "transcript", - props: viewportProps(), - children: of("transcript").map((candidate) => - rowBox(candidate, "transcript", context.widths?.surface), - ), - }), + ...listColumn("transcript", "reading-fixed", "reading", READING_WINDOW), ]), edgedColumn("inspection", inspectionWidth(size), REPL_PALETTE.bindingsSurface, [ ...paneTitle("inspection:heading", "inspection"), @@ -4654,9 +4839,13 @@ export function admissionFor(input: { readonly view: ReplView; readonly manifest: ReplLayoutManifest; readonly widths: ReplMeasuredWidths; + /** The reading this frame fitted, which is what its window admits rows of. */ + readonly reading?: ReplPreparedReading | undefined; + /** The open question's message, fitted, which its own window admits rows of. */ + readonly preview?: readonly ReplFittedLine[] | undefined; readonly boundsOf: (id: string) => ReplBounds | undefined; }): ReplAdmission { - const { view, manifest, widths, boundsOf } = input; + const { view, manifest, widths, reading, preview, boundsOf } = input; const windows = new Map(); for (const slot of manifest.viewports) { const capacity = capacityOf(boundsOf(slot.id)); @@ -4664,7 +4853,7 @@ export function admissionFor(input: { slot.window, admitRows({ offset: offsetFor(view, slot.window, capacity), - total: totalOf(view, widths, slot.window), + total: totalOf(view, widths, reading, preview, slot.window), capacity, }), ); @@ -4725,6 +4914,9 @@ function offsetOf(view: ReplView, window: string): number { if (window === SESSIONS_WINDOW) { return view.state.viewports.sessions; } + if (window === READING_WINDOW) { + return view.state.viewports.readings[entryReadingKeyOf(view.state)] ?? 0; + } if (window === ENTRIES_WINDOW) { return view.state.viewports.entries; } @@ -4738,14 +4930,26 @@ function offsetOf(view: ReplView, window: string): number { * from: a frozen prefix shows nothing of the present, so counting against the * live head would clamp a historical reading against rows it does not have. */ -function totalOf(view: ReplView, widths: ReplMeasuredWidths, window: string): number { +function totalOf( + view: ReplView, + widths: ReplMeasuredWidths, + reading: ReplPreparedReading | undefined, + preview: readonly ReplFittedLine[] | undefined, + window: string, +): number { if (window === SESSIONS_WINDOW) { return sessionContentRows(view.state, view.model, view.live); } if (window === ENTRIES_WINDOW) { return entriesRowCount(view.model); } - return drawerContent(view, widths?.drawer ?? 0)?.content.length ?? 0; + // The rows the frame actually fitted, not a count of logical lines: a line + // that wrapped is several rows, and a window admitting the smaller number + // would stop short of text it is showing the top of. + if (window === READING_WINDOW) { + return reading?.rows.length ?? 0; + } + return drawerContent(view, widths?.drawer ?? 0, preview)?.content.length ?? 0; } export { HISTORY_ROWS }; diff --git a/packages/cli/src/repl/components/actions.ts b/packages/cli/src/repl/components/actions.ts index c7c9d1e5c..fd85bf332 100644 --- a/packages/cli/src/repl/components/actions.ts +++ b/packages/cli/src/repl/components/actions.ts @@ -71,6 +71,15 @@ export type ReplAction = * whichever the reducer guessed rather than the list the control belongs to. */ | { readonly kind: "scroll-entries"; readonly delta: number } + /** + * Move the entry reading by whole rows. + * + * Its own member for the same reason the other two are: the transcript's + * reading, the catalog and a drawer's question can all be open at once, and + * one action meaning whichever the reducer guessed would move a region the + * reader was not pointing at. + */ + | { readonly kind: "scroll-reading"; readonly delta: number } /** * Show only the conversation this provider session key names. * diff --git a/packages/cli/src/repl/components/rows.ts b/packages/cli/src/repl/components/rows.ts index 42a4e0df5..249b81ac9 100644 --- a/packages/cli/src/repl/components/rows.ts +++ b/packages/cli/src/repl/components/rows.ts @@ -235,6 +235,10 @@ function activation(input: ReplViewData): ReplAction | undefined { const delta = named["delta"]; return typeof delta === "number" ? { kind: "scroll-entries", delta } : undefined; } + if (select === "scroll-reading") { + const delta = named["delta"]; + return typeof delta === "number" ? { kind: "scroll-reading", delta } : undefined; + } if (select === "session") { const session = named["session"]; return typeof session === "string" && session.length > 0 diff --git a/packages/cli/src/repl/elicitation.ts b/packages/cli/src/repl/elicitation.ts index e7f42cceb..8177adaaf 100644 --- a/packages/cli/src/repl/elicitation.ts +++ b/packages/cli/src/repl/elicitation.ts @@ -42,6 +42,7 @@ */ import { action, createSignal } from "effection"; +import type { ReplWaits } from "./lifecycle.ts"; import type { Operation, Stream } from "effection"; import { Elicitation, ElicitationProviderError, prepareElicitation } from "@executablemd/core"; import { validateParsed } from "@executablemd/core"; @@ -433,7 +434,15 @@ export function readQuestionForm(schema: Json): ReplQuestionForm { * outermost, so an outer provider would answer ahead of this one — the opposite * of what a provider is. */ -export function* useReplElicitation(): Operation { +export function* useReplElicitation( + /** + * Where a question says the element asking it is waiting, or none. + * + * Optional because the provider is complete without it: counting a wait is + * something a reading wants, not something answering a question needs. + */ + waits?: ReplWaits, +): Operation { const changes = createSignal(); let pending: ReplQuestion | undefined; let asked = 0; @@ -453,44 +462,51 @@ export function* useReplElicitation(): Operation { // thing that will judge the answer on its way out. const prepared = yield* prepareElicitation(request.schema, "Elicit"); asked++; - return yield* action(function (resolve) { - let settled = false; - const question: ReplQuestion = { - message: request.message, - // No schema member. The parsed form is everything a reader needs, - // and retaining the caller's object would hand a live reference to - // whatever built it out to the application. - form, - submit(values: Readonly>): ReplFormOutcome { - if (settled) { - return { kind: "invalid", issues: [] }; - } - // Every property the form actually holds, copied into a plain - // object — the empty string included. An optional field someone - // deliberately cleared is present and empty, which a schema may - // well accept; dropping it would answer a different question from - // the one on the screen. A field nobody has touched is absent. - const assembled: Record = {}; - for (const field of form.fields) { - const value = values[field.name]; - if (value !== undefined) { - assembled[field.name] = value; + // This element is waiting on an answer, from here until the question + // stops existing — however it stops: answered, halted or failed above. + const release = yield* waits?.hold("question") ?? noWait(); + try { + return yield* action(function (resolve) { + let settled = false; + const question: ReplQuestion = { + message: request.message, + // No schema member. The parsed form is everything a reader needs, + // and retaining the caller's object would hand a live reference to + // whatever built it out to the application. + form, + submit(values: Readonly>): ReplFormOutcome { + if (settled) { + return { kind: "invalid", issues: [] }; + } + // Every property the form actually holds, copied into a plain + // object — the empty string included. An optional field someone + // deliberately cleared is present and empty, which a schema may + // well accept; dropping it would answer a different question from + // the one on the screen. A field nobody has touched is absent. + const assembled: Record = {}; + for (const field of form.fields) { + const value = values[field.name]; + if (value !== undefined) { + assembled[field.name] = value; + } } - } - const issues = validateParsed(prepared.validate, assembled); - if (issues.length > 0) { - return { kind: "invalid", issues: Object.freeze([...issues]) }; - } - settled = true; - resolve(assembled); - return { kind: "answered", answer: Object.freeze({ ...assembled }) }; - }, - }; - publish(question); - // Whatever ends this — an answer, a halt, a failure upstream — the - // request stops being pending exactly when it stops existing. - return () => publish(undefined); - }); + const issues = validateParsed(prepared.validate, assembled); + if (issues.length > 0) { + return { kind: "invalid", issues: Object.freeze([...issues]) }; + } + settled = true; + resolve(assembled); + return { kind: "answered", answer: Object.freeze({ ...assembled }) }; + }, + }; + publish(question); + // Whatever ends this — an answer, a halt, a failure upstream — the + // request stops being pending exactly when it stops existing. + return () => publish(undefined); + }); + } finally { + release(); + } }, }, { at: "min" }, @@ -506,3 +522,9 @@ export function* useReplElicitation(): Operation { }, }; } + +/** No reading is counting waits, so nothing is held and nothing is released. */ +// deno-lint-ignore require-yield +function* noWait(): Operation<() => void> { + return () => {}; +} diff --git a/packages/cli/src/repl/expansion.ts b/packages/cli/src/repl/expansion.ts index 1309c67f6..5725f7f30 100644 --- a/packages/cli/src/repl/expansion.ts +++ b/packages/cli/src/repl/expansion.ts @@ -62,6 +62,8 @@ export type ExpansionState = "playing" | "pausing" | "paused"; * the inventory test says so rather than leaving an expansion path nothing * controls. Pause is sound only while every path crosses something here. */ +import type { ReplWaits } from "./lifecycle.ts"; + export interface BoundaryPartition { /** Operations that bracket one expansion walk. */ readonly walks: readonly string[]; @@ -79,10 +81,18 @@ export interface BoundaryPartition { * `hasBinding` answer a question about an invocation the walk has already * reached; holding one would stop the same walk the gate before it already * stops, one question later. + * + * `expand` surrounds an element's whole expansion, which makes it a step in + * the walk that reached it and not a walk of its own. Treating it as one would + * count every element as a separate expansion to pause and release, and a + * structural element crossing this seam would release the hold its enclosing + * walk is still standing in. It inherits the walk it was reached through, and + * holding it stops that walk before the element resolves. */ export const COMPONENT_BOUNDARIES: BoundaryPartition = { walks: ["content", "tryContent"], gates: [ + "expand", "importComponent", "applyModifiers", "applyBoundModifiers", @@ -184,7 +194,16 @@ export interface ExpansionController { * where the install runs: a controller installed inside a resource body would * be invisible to the execution the caller is about to start. */ -export function* useExpansionController(): Operation { +export function* useExpansionController( + /** + * Where a held walk says the element it stopped is waiting, or none. + * + * A hold is the one wait this controller owns: the element has reached a + * boundary and is standing there until Continue releases it, which is a + * different thing from the work being slow. + */ + waits?: ReplWaits, +): Operation { const walks = new Set(); const states = createSignal(); const crossings = new Map(); @@ -258,7 +277,12 @@ export function* useExpansionController(): Operation { if (walk === undefined) { return; } - yield* hold(walk); + const release = yield* waits?.hold("expansion") ?? noExpansionWait(); + try { + yield* hold(walk); + } finally { + release(); + } } function* bracket(boundary: string, body: () => Operation): Operation { @@ -363,3 +387,9 @@ export function* useExpansionController(): Operation { }, }; } + +/** No reading is counting waits, so nothing is held and nothing is released. */ +// deno-lint-ignore require-yield +function* noExpansionWait(): Operation<() => void> { + return () => {}; +} diff --git a/packages/cli/src/repl/fitting.ts b/packages/cli/src/repl/fitting.ts new file mode 100644 index 000000000..7777f6f1e --- /dev/null +++ b/packages/cli/src/repl/fitting.ts @@ -0,0 +1,561 @@ +/** + * How wide text actually is, asked of the engine that will draw it. + * + * The installed engine answers natural width through a fit-sized element: give + * it the same text operations a row will be drawn with, read the bounds it + * reports, and that is the number of cells those operations occupy. There is no + * public line-break API and no renderer change, so wrapping is built on top of + * that one question asked repeatedly. + * + * ## Why not `text.length` + * + * A UTF-16 code unit is not a cell. `界` is one unit and two cells; a combining + * accent is one unit and no cells at all; an emoji is a surrogate pair and two + * cells. Every width decision here is therefore an engine answer, and every cut + * is at a grapheme boundary — because a cut between a base character and its + * combining mark produces two rows neither of which is a character. + * + * UTF-16 offsets index source. They never stand in for terminal width. + * + * ## Why batched + * + * Each measurement is a render, and the engine's reported geometry is only + * valid until the next one — so every answer is copied out before another + * measurement is asked for. Candidates are therefore grouped: one measurement + * answers for every row still looking for its cut, and the search narrows all + * of them together rather than one row at a time. + * + * ## Why it refuses rather than clips + * + * A region with no room for even one grapheme cannot hold the reading, and + * shortening somebody's source to fit would be this screen editing the content + * it exists to show. So an unfittable row is a frame refusal, which already + * recovers on resize. + */ + +import { close, fit, open, text } from "@bomb.sh/tty"; +import type { Op } from "@bomb.sh/tty"; +import { Err, Ok } from "effection"; +import type { Operation, Result } from "effection"; +import type { ReplRenderer } from "./renderer.ts"; +import type { ReplTerminalSize } from "./terminal.ts"; +import type { ReplReadingLine } from "./source-reading.ts"; +import { READING_STATUSES } from "./source-reading.ts"; +import { runText, tokenRuns } from "./description.ts"; +import type { ReplTokenRun } from "./description.ts"; + +/** What a reading could not be fitted into. */ +export class ReplFitRefusal extends Error { + constructor(message: string) { + super(message); + this.name = "ReplFitRefusal"; + } +} + +/** + * One visual row of a reading, fitted to a measured width. + * + * Immutable frame preparation data. The same list answers how many rows there + * are, which of them a window admits, what each description says and which + * boxes a frame places — so a row count and a drawn row cannot disagree about + * what was prepared. + */ +export interface ReplPreparedRow { + readonly key: string; + /** Exactly the characters this row draws. Never trimmed, never shortened. */ + readonly text: string; + /** What each part of that text is. Concatenates to exactly `text`. */ + readonly runs: readonly ReplTokenRun[]; + /** The logical line this row came from. */ + readonly line: string; + /** Whether a previous row already carried the start of that line. */ + readonly continuation: boolean; + readonly rail: ReplReadingLine["rail"]; + readonly depth: number; + /** + * How far this row is set in from the start of the content region. + * + * The nesting depth's own indent, and on a continuation the leading + * whitespace of the logical line it continues as well — so a line that + * wrapped is still read at the column it was written at. Without it a + * continuation sits further left than the row above it and reads as a new, + * shallower line, which is a claim about structure that the source does not + * make. + * + * Display only. No source byte and no offset moves, and concatenating a + * line's rows still recovers exactly its text. + */ + readonly indent: number; + /** Present on a logical line's first row only. */ + readonly badge: readonly ReplTokenRun[] | undefined; + readonly style: ReplReadingLine["style"]; + /** How many cells the engine says `text` occupies. */ + readonly width: number; +} + +/** What a prepared reading reserved before any of its text was fitted. */ +export interface ReplFitReservation { + /** The rail column, measured rather than counted. */ + readonly rail: number; + /** The gap between the rail and the text. */ + readonly separator: number; + /** The status column, wide enough for every reading a badge can hold. */ + readonly status: number; + /** What is left for text after all three. */ + readonly content: number; + /** How wide the engine says each badge reading is, by its exact text. */ + readonly statusWidths: ReadonlyMap; +} + +/** A reading prepared for one measured width. */ +export interface ReplPreparedReading { + readonly reservation: ReplFitReservation; + readonly rows: readonly ReplPreparedRow[]; +} + +/** The rail glyph, and the gap between it and the text beside it. */ +const RAIL = "│"; +const SEPARATOR = " "; + +/** + * Ask the engine how wide each of these texts is. + * + * One fit-sized element per text, each under its own structural id, all in one + * render — and every answer copied into a plain map before returning, because + * the engine's geometry is only valid until the next measurement. + */ +function* widths( + renderer: ReplRenderer, + size: ReplTerminalSize, + texts: readonly string[], +): Operation> { + if (texts.length === 0) { + return Ok([]); + } + const ops: Op[] = [ + open("fit:root", { layout: { width: fit(), height: fit(), direction: "ttb" } }), + ]; + for (const [index, one] of texts.entries()) { + ops.push(open(`fit:${index}`, { layout: { width: fit(), height: fit() } })); + if (one.length > 0) { + ops.push(text(one)); + } + ops.push(close()); + } + ops.push(close()); + const measured = yield* renderer.measure(ops, size); + if (!measured.ok) { + return measured; + } + const answers: number[] = []; + for (const [index, one] of texts.entries()) { + // An empty text occupies nothing; the engine gives its element no bounds to + // report, which is the same answer said differently. + if (one.length === 0) { + answers.push(0); + continue; + } + const bounds = measured.value.boundsOf(`fit:${index}`); + if (bounds === undefined) { + return Err(new ReplFitRefusal(`the engine measured no width for ${JSON.stringify(one)}`)); + } + answers.push(Math.ceil(bounds.width)); + } + return Ok(answers); +} + +/** + * What a reading reserves before any of its text is fitted. + * + * The rail, the gap after it and the status column, every one of them an engine + * answer rather than a character count — `◀ EXIT · ● WAITING` is nineteen code + * units and nineteen cells only by coincidence, and the glyphs in it are + * exactly the characters a count gets wrong. + * + * The status column is the widest reading a badge can hold, including the + * cleanup wait that says two things at once. Reserving per row instead would + * make every phase change re-cut the source beside it, so a reader watching an + * element settle would see their own document reflow. + */ +export function* reserveFor( + renderer: ReplRenderer, + size: ReplTerminalSize, + room: number, +): Operation> { + const measured = yield* widths(renderer, size, [RAIL, SEPARATOR, ...READING_STATUSES]); + if (!measured.ok) { + return measured; + } + const [rail, separator, ...statuses] = measured.value; + const status = statuses.reduce((widest, one) => Math.max(widest, one), 0); + const content = room - rail - separator - separator - status; + if (content < 1) { + return Err( + new ReplFitRefusal( + `a reading needs ${rail + separator + separator + status + 1} columns and has ${room}`, + ), + ); + } + const statusWidths = new Map(); + for (const [index, one] of READING_STATUSES.entries()) { + statusWidths.set(one, statuses[index]); + } + return Ok(Object.freeze({ rail, separator, status, content, statusWidths })); +} + +/** + * One reading's logical lines as the rows they are drawn in. + * + * Every nonempty iteration consumes source, so this terminates and nothing is + * dropped: concatenating one logical line's rows recovers exactly its text, + * whitespace and all. A blank logical line stays one row, because it is a line + * of somebody's document. + */ +export function* prepareReading( + renderer: ReplRenderer, + size: ReplTerminalSize, + lines: readonly ReplReadingLine[], + room: number, +): Operation> { + const reserved = yield* reserveFor(renderer, size, room); + if (!reserved.ok) { + return reserved; + } + const reservation = reserved.value; + const cut: ReplPreparedRow[] = []; + // Nesting is drawn inside the content region rather than beside it, so a + // deeply nested element loses text room rather than pushing the status column + // off the pane it was measured against. + for (const line of lines) { + const indent = Math.min(line.depth * 2, Math.max(0, reservation.content - 1)); + const fitted = yield* fitLine(renderer, size, line, reservation.content - indent); + if (!fitted.ok) { + return fitted; + } + cut.push(...fitted.value); + } + // The finished candidates, measured again as the text they ended up being. + // The search above narrowed over prefixes; this is the row, and a row that + // does not fit the region it was cut for is a frame refusal rather than + // something to clip. + const measured = yield* widths( + renderer, + size, + cut.map((one) => one.text), + ); + if (!measured.ok) { + return measured; + } + const rows: ReplPreparedRow[] = []; + for (const [index, one] of cut.entries()) { + const width = measured.value[index]; + const indent = Math.min(one.depth * 2 + one.indent, Math.max(0, reservation.content - 1)); + if (width > reservation.content - indent) { + return Err( + new ReplFitRefusal( + `row ${one.key} measured ${width} columns in ${reservation.content - indent}`, + ), + ); + } + rows.push(Object.freeze({ ...one, width })); + } + return Ok(Object.freeze({ reservation, rows: Object.freeze(rows) })); +} + +/** One line of read-only text, fitted to a width. */ +export interface ReplFittedLine { + readonly key: string; + readonly text: string; + readonly runs: readonly ReplTokenRun[]; + /** Whether a previous row already carried the start of this line. */ + readonly continuation: boolean; + /** The column this line was written at, carried onto its continuations. */ + readonly indent: number; +} + +/** + * Fit read-only text to a width, with nothing reserved beside it. + * + * The reading's own `prepareReading` reserves a rail and a status column + * because every row of it carries both. A drawer's message carries neither: it + * is somebody's text in a box, and all of the box is for the text. + * + * The same measurement and the same cuts otherwise — engine widths, grapheme + * boundaries, word boundaries preferred, whitespace kept, blank lines kept — + * so concatenating one line's rows recovers exactly the line. A drawer that + * showed one row per logical line could only ever show the first screenful of + * a long one, and scrolling cannot reveal a row that was never prepared. + */ +export function* fitPlain( + renderer: ReplRenderer, + size: ReplTerminalSize, + lines: readonly { + readonly key: string; + readonly text: string; + readonly runs?: readonly ReplTokenRun[]; + }[], + room: number, +): Operation> { + if (room < 1) { + return Ok([]); + } + const fitted: ReplFittedLine[] = []; + for (const one of lines) { + const runs = one.runs ?? tokenRuns([{ text: one.text, token: "source" }]); + const rows = yield* fitLine( + renderer, + size, + Object.freeze({ + key: one.key, + text: one.text, + runs, + rail: "rail-pending" as const, + depth: 0, + badge: undefined, + style: Object.freeze({ role: "source" as const, selected: false, inspected: false }), + }), + room, + ); + if (!rows.ok) { + return rows; + } + for (const row of rows.value) { + fitted.push( + Object.freeze({ + key: row.key, + text: row.text, + runs: row.runs, + continuation: row.continuation, + indent: row.indent, + }), + ); + } + } + return Ok(Object.freeze(fitted)); +} + +/** One logical line as its rows. */ +function* fitLine( + renderer: ReplRenderer, + size: ReplTerminalSize, + line: ReplReadingLine, + room: number, +): Operation> { + // A blank line is a row. Measuring it would ask the engine the width of + // nothing, and dropping it would edit the shape of somebody's document. + if (line.text.length === 0) { + return Ok([row(line, "", 0, false, 0)]); + } + const whole = yield* widths(renderer, size, [line.text]); + if (!whole.ok) { + return whole; + } + if (whole.value[0] <= room) { + return Ok([row(line, line.text, 0, false, 0)]); + } + // Where this line begins, so its continuations begin there too. Bounded well + // inside the region: an indent that left no room for text would make a row + // that consumes nothing, and this loop would not terminate. + const hanging = Math.min( + line.text.length - line.text.trimStart().length, + Math.max(0, Math.floor(room / 2)), + ); + const rows: ReplPreparedRow[] = []; + let at = 0; + while (at < line.text.length) { + const rest = line.text.slice(at); + const cut = yield* cutAt(renderer, size, rest, at > 0 ? room - hanging : room); + if (!cut.ok) { + return cut; + } + rows.push(row(line, rest.slice(0, cut.value), at, at > 0, hanging)); + at += cut.value; + } + return Ok(Object.freeze(rows)); +} + +/** + * How many code units of `rest` fit in `room`, cut where a reader would cut. + * + * The longest prefix the engine measures as fitting, then back to its last + * whitespace boundary if it has one — whitespace kept in the prefix, because it + * is part of the line and the next row does not begin with somebody else's + * space. A word longer than the region has no such boundary and is broken at + * the longest fitting grapheme instead, which is the only way text wider than + * its pane can be read at all. + */ +function* cutAt( + renderer: ReplRenderer, + size: ReplTerminalSize, + rest: string, + room: number, +): Operation> { + const stops = graphemeStops(rest); + // Narrow by halves over grapheme boundaries, asking the engine for one batch + // of candidates per round rather than one render per candidate. + let low = 0; + let high = stops.length - 1; + while (low < high) { + const middle = Math.ceil((low + high) / 2); + const measured = yield* widths(renderer, size, [rest.slice(0, stops[middle])]); + if (!measured.ok) { + return measured; + } + if (measured.value[0] <= room) { + low = middle; + } else { + high = middle - 1; + } + } + if (low === 0) { + // Not one grapheme fits. Clipping it would show the reader a row that is + // not what their file says, so the frame refuses instead. + return Err( + new ReplFitRefusal( + `${room} columns hold no grapheme of ${JSON.stringify(rest.slice(0, stops[1] ?? 1))}`, + ), + ); + } + const longest = stops[low]; + const preferred = wordBoundary(rest, longest); + return Ok(preferred === undefined ? longest : preferred); +} + +/** + * The last whitespace boundary at or before `longest`, or none. + * + * Counted as the offset *after* the run of whitespace, so the spaces a reader + * typed stay on the row that ends with them. A boundary at zero is not one: it + * would consume nothing and loop. + */ +function wordBoundary(text: string, longest: number): number | undefined { + let at: number | undefined; + for (let index = 0; index < longest; index += 1) { + if (/\s/.test(text[index]) && !/\s/.test(text[index + 1] ?? "x")) { + at = index + 1; + } + } + return at === undefined || at === 0 ? undefined : at; +} + +/** + * Every grapheme boundary of a string, as offsets into it. + * + * `Intl.Segmenter` is what decides, so a base character and its combining + * marks are one stop and an emoji sequence is one stop. The list starts at zero + * and ends at the string's length, so `stops[n]` is always a slice offset. + */ +function graphemeStops(text: string): readonly number[] { + const stops: number[] = [0]; + for (const { index } of segmenter.segment(text)) { + if (index > 0) { + stops.push(index); + } + } + stops.push(text.length); + return stops; +} + +const segmenter = new Intl.Segmenter(undefined, { granularity: "grapheme" }); + +/** + * One prepared row. + * + * The runs are cut by the same offsets the text was, so a token that spans a + * break becomes two runs of one role rather than one row of ordinary prose. + * Only the first row of a logical line carries its badge. + */ +function row( + line: ReplReadingLine, + shown: string, + from: number, + continuation: boolean, + hanging: number, +): ReplPreparedRow { + return Object.freeze({ + // Replaced by the engine's answer for this exact row before the reading is + // returned; nothing reads it in between. + width: 0, + indent: continuation ? hanging : 0, + key: continuation ? `${line.key}+${from}` : line.key, + text: shown, + runs: slicedRuns(line.runs, from, from + shown.length), + line: line.key, + continuation, + rail: line.rail, + depth: line.depth, + badge: continuation ? undefined : line.badge, + style: line.style, + }); +} + +/** + * The part of a run list covering one half-open slice of its text. + * + * Roles survive the cut: a quoted value broken across two rows is a quoted + * value on both of them. A run the slice does not touch is absent rather than + * empty, which is what keeps the list concatenating to exactly the row's text. + */ +function slicedRuns( + runs: readonly ReplTokenRun[], + from: number, + to: number, +): readonly ReplTokenRun[] { + const parts: { readonly text: string; readonly token: ReplTokenRun["token"] }[] = []; + let at = 0; + for (const run of runs) { + const start = Math.max(at, from); + const end = Math.min(at + run.text.length, to); + if (end > start) { + parts.push({ text: run.text.slice(start - at, end - at), token: run.token }); + } + at += run.text.length; + } + return tokenRuns(parts); +} + +/** + * One prepared row as the runs a description carries. + * + * Rail, the gap after it, the nesting indent, the row's own text, then the + * badge against the region's right inner edge. One row of runs, which is what + * the existing presentation already draws as several text operations inside one + * measured element — so this introduces no second rendering path and inherits + * the bound that keeps a row inside its own region. + * + * The padding is arithmetic over engine answers, not over character counts: + * every width in it was measured, and the badge column is the reservation every + * row of the reading shares. + */ +export function readingRuns( + prepared: ReplPreparedReading, + one: ReplPreparedRow, +): readonly ReplTokenRun[] { + const { reservation } = prepared; + const indent = Math.min(one.depth * 2 + one.indent, Math.max(0, reservation.content - 1)); + const parts: { readonly text: string; readonly token: ReplTokenRun["token"] }[] = [ + { text: RAIL, token: one.rail }, + { text: SEPARATOR, token: "punctuation" }, + ]; + if (indent > 0) { + parts.push({ text: " ".repeat(indent), token: "punctuation" }); + } + for (const run of one.runs) { + parts.push({ text: run.text, token: run.token }); + } + const badge = one.badge; + const shown = badge === undefined ? "" : runText(badge); + const width = badge === undefined ? 0 : (reservation.statusWidths.get(shown) ?? shown.length); + // Everything between the end of the text and the start of the status column, + // so a row with no badge still ends where the reading ends and the column is + // the same column on every row of it. + const gap = + reservation.content - indent - one.width + reservation.separator + (reservation.status - width); + if (gap > 0) { + parts.push({ text: " ".repeat(gap), token: "punctuation" }); + } + for (const run of badge ?? []) { + parts.push({ text: run.text, token: run.token }); + } + return tokenRuns(parts); +} diff --git a/packages/cli/src/repl/lifecycle.ts b/packages/cli/src/repl/lifecycle.ts new file mode 100644 index 000000000..d233f4e4a --- /dev/null +++ b/packages/cli/src/repl/lifecycle.ts @@ -0,0 +1,276 @@ +/** + * What this process is doing right now, element by element. + * + * Core says where one expansion has reached; this is where the session keeps + * what it was told, for as long as the entry that caused it is open. It is a + * reading, not a record: nothing here reaches the Journal, a cold process + * reconstructs none of it, and an entry that goes takes its observations with + * it. + * + * ## One call, not one name + * + * An expansion id names a logical element. A document that writes the same + * element twice, or writes one inside a ``, produces several *calls* of + * it, and a reader watching the second must not be shown the first. Each + * actual call is given an occurrence key of this entry's own, and the + * enclosing key is bound around the delegation, so the calls under one element + * are its children and a repeated id never overwrites anything. + * + * The lineage is presentation and only presentation. It is bound in a context + * a descendant can rebind, which is exactly why nothing that must be + * authoritative reads it: a generated fragment inherits where it is being + * shown while its capability environment stays as narrow as it was. + * + * ## Consumers are siblings of the dispatch, never children + * + * Core publishes an element's terminal phase after the whole middleware + * dispatch has unwound. A consumer spawned on the handler's own frame is torn + * down before that, so it never learns the element finished — and one the + * handler *joined* would wait for a phase that cannot arrive until the handler + * returns. Consumers therefore run on the entry's own owner, started before + * the delegation and outliving it. + */ + +import { createSignal, each, ensure, useScope } from "effection"; +import type { Operation, Scope, Stream } from "effection"; +import { Component } from "@executablemd/core/api"; +import type { ComponentExpansionPhase, ComponentExpansionRequest } from "@executablemd/core/api"; +import { createContext } from "effection"; +import type { Context } from "effection"; +import type { ExecutionInstallation } from "@executablemd/core/host"; +import type { SourcePosition } from "@executablemd/core"; + +/** Where one call of one element has reached, as a reader sees it. */ +export type ReplLifecyclePhase = "enter" | "active" | "exit" | "settled" | "failed" | "cancelled"; + +/** One actual call of one authored element. */ +export interface ReplOccurrence { + /** This entry's own key for this call. Never a logical id. */ + readonly key: string; + /** The logical element this is a call of. */ + readonly expansion: string; + readonly name: string; + /** The call this one was reached through, or none at the top. */ + readonly parent: string | undefined; + readonly position: Readonly | undefined; + readonly phase: ReplLifecyclePhase; + /** Why an element that has not settled is waiting, if it is. */ + readonly waiting: readonly ReplWaitReason[]; +} + +/** What an element is waiting on, counted rather than guessed. */ +export type ReplWaitReason = "question" | "permission" | "expansion"; + +/** + * Counting one reason an element is waiting, for whoever actually waits. + * + * The narrow half of the lifecycle, handed to a provider so it can say "this + * element is waiting on me" without being able to read, change or end anything + * else. A provider given none counts nothing, which is what an execution with + * no session reading behaves like. + */ +export interface ReplWaits { + hold(reason: ReplWaitReason): Operation<() => void>; +} + +/** Everything this entry is doing, detached and immutable. */ +export interface ReplLifecycleReading { + /** The entry these observations belong to, or none between entries. */ + readonly entry: string | undefined; + /** Every call, in the order it was first observed. */ + readonly occurrences: readonly ReplOccurrence[]; +} + +/** Nothing observed. What a reading is between entries, and after one closes. */ +export const NO_LIFECYCLE: ReplLifecycleReading = Object.freeze({ + entry: undefined, + occurrences: Object.freeze([]), +}); + +export interface ReplLifecycle { + /** Installed into one execution, carrying this session's observation. */ + readonly installation: ExecutionInstallation; + /** Begin this entry's generation. Updates from an older one are refused. */ + open(entry: string): void; + /** End it, and drop what it was holding. */ + close(entry: string): void; + /** What this entry is doing now. Detached: the caller may keep it. */ + reading(): ReplLifecycleReading; + /** + * Count one reason the element being expanded is waiting, until released. + * + * An operation, because the call it belongs to is read from the lineage + * where the wait is taken: two concurrent questions are two waits on two + * elements rather than one session that is "busy". Releasing is idempotent, + * and a wait taken outside any element releases nothing. + */ + hold(reason: ReplWaitReason): Operation<() => void>; + /** Announced whenever the reading changed. */ + readonly changes: Stream; +} + +/** + * The call a descendant was reached through. + * + * Presentation lineage, nothing else. It is an ordinary context, so a + * descendant may rebind it for its own descendants — which is why no + * authority, identity or durable decision reads it. + */ +const CurrentOccurrence: Context = createContext( + "repl.lifecycle.occurrence", + undefined, +); + +interface Observed { + key: string; + expansion: string; + name: string; + parent: string | undefined; + position: Readonly | undefined; + phase: ReplLifecyclePhase; + waiting: ReplWaitReason[]; +} + +export function useReplLifecycle(): Operation { + return { + *[Symbol.iterator]() { + // The owner every consumer runs under: the session's own scope, which + // outlives any one element's dispatch. + const owner: Scope = yield* useScope(); + const changes = createSignal(); + let generation: string | undefined; + let sequence = 0; + const observed = new Map(); + + const announce = (): void => { + changes.send(reading()); + }; + const reading = (): ReplLifecycleReading => + Object.freeze({ + entry: generation, + occurrences: Object.freeze( + [...observed.values()].map((one) => + Object.freeze({ + key: one.key, + expansion: one.expansion, + name: one.name, + parent: one.parent, + position: one.position, + phase: one.phase, + waiting: Object.freeze([...one.waiting]), + }), + ), + ), + }); + + const settle = (key: string, phase: ComponentExpansionPhase): void => { + const one = observed.get(key); + if (one === undefined) { + return; + } + if (phase.phase === "complete") { + one.phase = phase.result.ok ? "settled" : "failed"; + } else if (phase.phase === "cancelled") { + one.phase = "cancelled"; + } else if (phase.phase === "exit") { + one.phase = "exit"; + } else { + one.phase = phase.phase; + } + announce(); + }; + + const lifecycle: ReplLifecycle = { + installation: { + *install(): Operation { + yield* Component.around({ + *expand([request]: [ComponentExpansionRequest], next) { + const entry = generation; + if (entry === undefined) { + // Nothing is open to attribute this to. Observation is never + // a reason to change what executes, so it is delegated as it + // is and nothing is recorded. + yield* next(request); + return; + } + sequence += 1; + const key = `${entry}#${sequence}`; + const parent = yield* CurrentOccurrence.get(); + observed.set(key, { + key, + expansion: request.expansion.id, + name: request.expansion.name, + parent, + position: request.expansion.position, + phase: "enter", + waiting: [], + }); + announce(); + // Started before the delegation and owned by the session, so + // it is still there when the terminal phase is published after + // this handler has unwound. + owner.run(function* () { + for (const phase of yield* each(request.phases)) { + if (generation === entry) { + settle(key, phase); + } + yield* each.next(); + } + }); + // Bound *around* the delegation rather than set on this + // frame, which is what makes it the enclosing key for + // everything the call reaches — generated descendants + // included — and for nothing beside it. + yield* CurrentOccurrence.with(key, () => next(request)); + }, + }); + }, + }, + open(entry: string): void { + generation = entry; + sequence = 0; + observed.clear(); + announce(); + }, + close(entry: string): void { + if (generation !== entry) { + return; + } + generation = undefined; + observed.clear(); + announce(); + }, + reading, + *hold(reason: ReplWaitReason): Operation<() => void> { + // The call that is waiting is the one this wait was taken inside. + const key = yield* CurrentOccurrence.get(); + const one = key === undefined ? undefined : observed.get(key); + if (one === undefined) { + return () => {}; + } + one.waiting.push(reason); + announce(); + let released = false; + return () => { + if (released) { + return; + } + released = true; + const at = one.waiting.indexOf(reason); + if (at >= 0) { + one.waiting.splice(at, 1); + } + announce(); + }; + }, + changes, + }; + + yield* ensure(() => { + generation = undefined; + observed.clear(); + }); + return lifecycle; + }, + }; +} diff --git a/packages/cli/src/repl/presentation-style.ts b/packages/cli/src/repl/presentation-style.ts index a0a62ad3b..ad3425d3b 100644 --- a/packages/cli/src/repl/presentation-style.ts +++ b/packages/cli/src/repl/presentation-style.ts @@ -104,7 +104,20 @@ export type ReplPresentationRole = /** The line the next entry is typed on. */ | "draft" /** A retained position, and the band that holds them. */ - | "history"; + | "history" + /** The rail beside a reading, saying which region an element occupies. */ + | "rail-pending" + | "rail-active" + | "rail-waiting" + | "rail-exit" + | "rail-settled" + /** What one call of one element is doing, as its glyph and its word. */ + | "lifecycle-enter" + | "lifecycle-active" + | "lifecycle-waiting" + | "lifecycle-exit" + | "lifecycle-settled" + | "lifecycle-failed"; /** * One row's presentation facts, as the reading settles them. @@ -193,6 +206,31 @@ export const REPL_PALETTE = Object.freeze({ bindingsSurface: 0x0a0c0e, /** A pane's edge. */ edge: 0x161c21, + /** An element one call of which has been entered. */ + lifecycleEnter: 0x5aa87c, + /** One that is expanding. */ + lifecycleActive: 0x7fd3e8, + /** One that is waiting on somebody. */ + lifecycleWaiting: 0xc99a3f, + /** One whose body returned and whose resources are being released. */ + lifecycleExit: 0xc2766e, + /** One that finished. */ + lifecycleSettled: 0x4f8468, + /** One that failed. */ + lifecycleFailed: 0xd24b3f, + /** + * The rails beside a reading, one per phase the occupied region is in. + * + * Dimmer than the badge of the same phase on purpose: a rail runs the whole + * height of an element and says only *where* it is, so it must not compete + * with the text it encloses or with the badge that says what the phase is. + */ + railActive: 0x2f6272, + railWaiting: 0x4a3a1c, + railExit: 0x5d3c39, + railSettled: 0x2e3c35, + /** Source nothing has been observed in. Not a phase: an absence of one. */ + railPending: 0x252b30, }); /** @@ -329,6 +367,28 @@ function colourOf(role: ReplPresentationRole, inspected: boolean): number { return REPL_PALETTE.waiting; case "history": return REPL_PALETTE.historical; + case "rail-pending": + return REPL_PALETTE.railPending; + case "rail-active": + return REPL_PALETTE.railActive; + case "rail-waiting": + return REPL_PALETTE.railWaiting; + case "rail-exit": + return REPL_PALETTE.railExit; + case "rail-settled": + return REPL_PALETTE.railSettled; + case "lifecycle-enter": + return REPL_PALETTE.lifecycleEnter; + case "lifecycle-active": + return REPL_PALETTE.lifecycleActive; + case "lifecycle-waiting": + return REPL_PALETTE.lifecycleWaiting; + case "lifecycle-exit": + return REPL_PALETTE.lifecycleExit; + case "lifecycle-settled": + return REPL_PALETTE.lifecycleSettled; + case "lifecycle-failed": + return REPL_PALETTE.lifecycleFailed; case "source": case "status": case "draft": diff --git a/packages/cli/src/repl/program.ts b/packages/cli/src/repl/program.ts index f0fb6b704..efe4c0183 100644 --- a/packages/cli/src/repl/program.ts +++ b/packages/cli/src/repl/program.ts @@ -55,6 +55,7 @@ import { permissionWithdrawn, presentationFor, readinessOf, + readingOf, reduceRepl, refusedView, stateFor, @@ -82,6 +83,9 @@ import type { ReplSession } from "./session.ts"; import { useReplFrames } from "./frame.ts"; import type { ReplFrames } from "./frame.ts"; import { committedOps, flatten, profileFor, skeletonOps } from "./layout.ts"; +import { fitPlain, prepareReading } from "./fitting.ts"; +import { readingLines } from "./source-reading.ts"; +import { sourceRuns } from "./presentation-text.ts"; import type { ReplBox, ReplLayoutManifest, ReplRegion } from "./layout.ts"; import { capacityOf, NOTHING_ADMITTED } from "./layout-admission.ts"; import type { ReplAdmission } from "./layout-admission.ts"; @@ -772,7 +776,13 @@ function keyOfFocus(tree: ReplTree): string | undefined { return node === undefined ? undefined : tree.keyOf(node); } -/** This process's overlay, as the application reads it. */ +/** + * This process's overlay, as the application reads it. + * + * Everything this process knows and no record holds. What a reading frozen at + * a recorded position leaves out of it is stated once, in `viewFor`, rather + * than decided again here. + */ function liveOf(session: ReplSession): ReplLive { return { output: session.overlay.output, @@ -781,6 +791,7 @@ function liveOf(session: ReplSession): ReplLive { pausable: session.controller !== undefined, running: session.live, agent: session.agent, + lifecycle: session.lifecycle, }; } @@ -1356,6 +1367,11 @@ function measuringAt( entriesWindowed, entriesRows, capture: "capture", + // The measuring pass describes no width-dependent text, so it has no + // fitted reading or preview either: text prepared against a width this + // pass is still asking for would be text fitted to nothing. + reading: undefined, + preview: undefined, }; } @@ -1406,10 +1422,53 @@ export function* prepareFrame( if (!third.ok) { return third; } + // The entry reading, fitted to the transcript's measured inner width. It + // happens here — after the widths are known and before anything is admitted — + // because the rows it produces are what the window has to admit and what the + // descriptions have to say. Preparing it later would admit a count from one + // list and describe another. + // + // Independent probe elements, so nothing about this measurement touches the + // tree the frame is about to draw. + // + // A profile with no pane for it has no reading: a frame refused for being too + // small publishes no transcript and no routed outlet, so there is no measured + // region to fit one to. That is different from a pane too narrow to hold a + // grapheme, which is a refusal and stays one. + const reading = + widths.surface < 1 + ? Ok(undefined) + : yield* prepareReading(renderer, view.size, readingLines(readingOf(view)), widths.surface); + if (!reading.ok) { + return reading; + } + // The open question's message, fitted to the drawer's own width. Prepared + // here for the same reason the reading is: a drawer row is one row, so a + // long preview line is only reachable if it was cut into rows before the + // window counted them. + const asked = view.live.question; + const preview = + asked === undefined || widths.drawer < 1 + ? Ok(undefined) + : yield* fitPlain( + renderer, + view.size, + asked.message.split("\n").map((text, offset) => ({ + key: `message:${offset}`, + text, + runs: sourceRuns(text), + })), + widths.drawer, + ); + if (!preview.ok) { + return preview; + } const admission = admissionFor({ view, manifest: measured.manifest, widths, + reading: reading.value, + preview: preview.value, boundsOf: (id: string) => third.value.boundsOf(id), }); const context: ReplPresentationContext = { @@ -1419,6 +1478,8 @@ export function* prepareFrame( entriesWindowed: windowed, entriesRows, capture: "capture", + reading: reading.value, + preview: preview.value, }; return Ok({ context, admission, presentation: presentationFor(view, context) }); } diff --git a/packages/cli/src/repl/session.ts b/packages/cli/src/repl/session.ts index e5c5164e5..7f577d388 100644 --- a/packages/cli/src/repl/session.ts +++ b/packages/cli/src/repl/session.ts @@ -95,6 +95,8 @@ import { filter } from "@effectionx/stream-helpers"; import { consumeAdmissions } from "./admission.ts"; import type { ReplAgentAuthority, ReplAgentReading } from "./agent.ts"; import { useReplElicitation } from "./elicitation.ts"; +import { useReplLifecycle } from "./lifecycle.ts"; +import type { ReplLifecycleReading } from "./lifecycle.ts"; import type { ReplElicitations, ReplQuestion } from "./elicitation.ts"; import { EntrySegmentStream, entryKey, partitionEntrySegments } from "./entries.ts"; import { useExpansionController } from "./expansion.ts"; @@ -263,6 +265,17 @@ export interface ReplSession { readonly permissions: ReplAgentAuthority; /** Whether an entry's execution is still running in this process. */ readonly live: boolean; + /** + * What this process is doing element by element, for the entry that is open. + * + * Ephemeral and detached, like the Agent reading beside it: a cold process + * reconstructs none of it, and an entry that closes takes its observations + * with it. Static syntax and a retained admission prove nothing about it — + * what is here is what this process actually watched happen. + */ + readonly lifecycle: ReplLifecycleReading; + /** Every change to that reading, as it changes. */ + readonly lifecycleChanges: Stream; /** Each reprojection, as the history grows under it. */ readonly changes: Stream; /** @@ -432,13 +445,17 @@ function* start( } provisional.run(function* () { - const expansion = yield* useExpansionController(); - const elicitation = yield* useReplElicitation(); + // Created before anything can start an entry, and owned by this session: + // its consumers outlive every element's dispatch, which is the only place + // a terminal observation can reach them. + const lifecycle = yield* useReplLifecycle(); + const expansion = yield* useExpansionController(lifecycle); + const elicitation = yield* useReplElicitation(lifecycle); // Created here and installed into each entry's execution below, so the // middleware it owns belongs to this session's scope and dies with it. An // execution elsewhere would otherwise inherit an observer watching for a // session that is gone. - const agent = yield* useReplAgent(permissionMode ?? "deny-all"); + const agent = yield* useReplAgent(permissionMode ?? "deny-all", lifecycle); // The scope every entry task is started in. One session, one owner: a task // started anywhere else would outlive the observer and the kernels it // reports through. @@ -500,6 +517,10 @@ function* start( get live() { return live; }, + get lifecycle() { + return lifecycle.reading(); + }, + lifecycleChanges: lifecycle.changes, changes, outputs, join(): Operation> { @@ -644,6 +665,7 @@ function* start( // not what this one has printed, and the Journal holds that one's anyway. output = ""; live = true; + lifecycle.open(key); const task = owner.run(function* () { // Registered before anything this task acquires, and therefore released // after all of it: destructors run in reverse order of registration. By @@ -663,6 +685,19 @@ function* start( running = undefined; } }); + // Registered here, so it runs after everything this entry acquired has + // been released: the execution and its providers, the output consumer + // and the Agent attachment all come down first, and only then does the + // session say it is no longer live. The two readings that describe + // work in flight go with it, because by now there is none. + yield* ensure(() => { + live = false; + lifecycle.close(key); + // Said on the ordinary path, because nothing else will: an entry + // that appended no record and took no input still stopped running, + // and a reader waiting for that is waiting for this. + changes.send(model); + }); const outcome = yield* runEntry(key, source, view, initialBindings); settle(outcome); // A settled entry is past whatever its segment retained, whichever way @@ -710,7 +745,7 @@ function* start( // Installed into this exact execution, and nowhere else: the live // observer and the permission policy are this session's, not the // process's, and the attachment carries the entry whose turns they are. - [...installations, installation], + [...installations, lifecycle.installation, installation], { initialBindings }, ); // Consumed as it arrives, inside this session's scope. Collecting until @@ -731,7 +766,6 @@ function* start( } }); const outcome = yield* started; - live = false; if (!outcome.ok && isReconstructionFailure(outcome.error)) { refuse(outcome.error); return outcome; @@ -739,7 +773,6 @@ function* start( open(); return outcome; } catch (error) { - live = false; const raised = error instanceof Error ? error : new Error(String(error)); if (isReconstructionFailure(raised)) { refuse(raised); diff --git a/packages/cli/src/repl/source-reading.ts b/packages/cli/src/repl/source-reading.ts new file mode 100644 index 000000000..00f4bc747 --- /dev/null +++ b/packages/cli/src/repl/source-reading.ts @@ -0,0 +1,757 @@ +/** + * One entry, as the thing a reader came to look at: what it produced, and the + * source that produced it with what each part of it is doing. + * + * Private to this REPL. It takes a frozen model entry, whatever the session's + * lifecycle reading currently says, and the live output of a run still going, + * and answers one immutable list of logical lines. It measures nothing, draws + * nothing and reads no context: a description asks it what to show and the + * frame decides how wide that is. + * + * ## Output first, then source + * + * A reader opens an entry to see what it came to. The source is why it came to + * that, and it is long — so the result is above it and both share one window, + * which is what makes scrolling down from an answer into the reason for it one + * movement rather than two panes to reconcile. + * + * ## Static syntax is not execution + * + * `inspectSource` says where the executable elements of a text are written. It + * says nothing about whether any of them ran: a retained prefix shows the same + * elements it always showed, and a cold process reconstructs no phase at all. + * So an element acquires a phase only from an observation actually attributed + * to it — matched on this entry, its lineage and its recorded source position — + * and source nothing was observed in gets the rail that means *nothing was + * observed here*, never a borrowed one. + * + * ## Generated source replaces, and keeps its own offsets + * + * An `` that admitted a generated fragment shows that fragment where + * its producer was written, because the producer is an implementation detail of + * text the reader is being asked to read. The enclosure bytes stay exactly as + * authored and only what they enclose is substituted, so the opening and + * closing tags a reader sees are the ones in their file. + * + * The fragment is a *child region* rather than a splice: its offsets index its + * own text, which is the only space its own recorded positions were ever + * counted in. Splicing would renumber them and attribute its elements to the + * wrong bytes of somebody else's file. + */ + +import { inspectSource } from "@executablemd/core"; +import type { SourceElement } from "@executablemd/core"; +import type { ReplEntry, ReplGenerated, ReplScope } from "./model.ts"; +import type { ReplLifecycleReading, ReplOccurrence } from "./lifecycle.ts"; +import { styleOf } from "./presentation-style.ts"; +import type { ReplPresentationRole, ReplRowStyle } from "./presentation-style.ts"; +import { tokenRuns } from "./description.ts"; +import type { ReplTokenRun } from "./description.ts"; +import { sourceRuns } from "./presentation-text.ts"; + +/** What this terminal draws for each phase, glyph and word together. */ +const BADGES: Readonly< + Record< + "enter" | "active" | "waiting" | "exit" | "settled" | "failed", + { + readonly glyph: string; + readonly word: string; + readonly role: ReplPresentationRole; + } + > +> = Object.freeze({ + enter: Object.freeze({ glyph: "▶", word: "ENTER", role: "lifecycle-enter" }), + active: Object.freeze({ glyph: "●", word: "ACTIVE", role: "lifecycle-active" }), + waiting: Object.freeze({ glyph: "●", word: "WAITING", role: "lifecycle-waiting" }), + exit: Object.freeze({ glyph: "◀", word: "EXIT", role: "lifecycle-exit" }), + settled: Object.freeze({ glyph: "✓", word: "SETTLED", role: "lifecycle-settled" }), + failed: Object.freeze({ glyph: "×", word: "FAILED", role: "lifecycle-failed" }), +}); + +/** + * Every reading a badge can hold, so the stable status column is a measurement + * rather than a guess. + * + * The widest is a cleanup wait, which is two readings at once; a frame reserves + * for that and every phase-only change afterwards reuses the same reservation + * and the same source cuts. Cancellation keeps this terminal's existing word + * rather than acquiring a glyph the archive never gave it. + */ +export const READING_STATUSES: readonly string[] = Object.freeze([ + ...Object.values(BADGES).map((one) => `${one.glyph} ${one.word}`), + `${BADGES.exit.glyph} ${BADGES.exit.word} · ${BADGES.waiting.glyph} ${BADGES.waiting.word}`, + "cancelled", +]); + +/** One line of a reading, before anything has been fitted to a width. */ +export interface ReplReadingLine { + /** Stable within one reading, so a window and a target agree on a row. */ + readonly key: string; + /** The logical text of this line. Never trimmed, never shortened. */ + readonly text: string; + /** What each part of that text is. Concatenates to exactly `text`. */ + readonly runs: readonly ReplTokenRun[]; + /** Which region this line is in, as a rail role. */ + readonly rail: ReplPresentationRole; + /** How deeply nested the region is, counted from the entry's own source. */ + readonly depth: number; + /** + * What the element owning this line is doing, or none. + * + * Present on the first line of an observed element's opening delimiter and on + * the first line of its closing one, and nowhere else: a badge on every + * wrapped continuation would say the phase changed between two halves of one + * tag. + */ + readonly badge: readonly ReplTokenRun[] | undefined; + readonly style: ReplRowStyle; +} + +/** One captioned part of a reading. */ +export interface ReplReadingSection { + readonly caption: string; + readonly captionStyle: ReplRowStyle; + readonly lines: readonly ReplReadingLine[]; +} + +/** One entry's whole reading: what it produced, then what produced it. */ +export interface ReplReading { + readonly output: ReplReadingSection; + readonly source: ReplReadingSection; +} + +/** Nothing to read. What a prefix that admitted no entry shows. */ +export const NO_READING: ReplReading = Object.freeze({ + output: Object.freeze({ + caption: "No entry output recorded at this checkpoint.", + captionStyle: styleOf("metadata"), + lines: Object.freeze([]), + }), + source: Object.freeze({ + caption: "Source", + captionStyle: styleOf("pane-heading"), + lines: Object.freeze([]), + }), +}); + +/** + * One reading as the flat list of logical lines it is drawn as. + * + * Captions included, because a caption is a row of the reading rather than a + * fixed heading above it: the two halves share one vertical window, so + * scrolling from a result down into the source that produced it has to carry + * the words that say which is which. + */ +export function readingLines(reading: ReplReading): readonly ReplReadingLine[] { + return Object.freeze([ + caption("output", reading.output), + ...reading.output.lines, + caption("source", reading.source), + ...reading.source.lines, + ]); +} + +function caption(key: string, section: ReplReadingSection): ReplReadingLine { + return Object.freeze({ + key: `caption:${key}`, + text: section.caption, + runs: tokenRuns([{ text: section.caption, token: section.captionStyle.role }]), + rail: "rail-pending", + depth: 0, + badge: undefined, + style: section.captionStyle, + }); +} + +/** What a reading is being built for. */ +export interface ReplReadingRequest { + readonly entry: ReplEntry; + /** This session's observations, or an empty reading for a retained prefix. */ + readonly lifecycle: ReplLifecycleReading; + /** Text a run still going has emitted, or empty. */ + readonly live: string; + /** Whether this reading is a retained position rather than the live head. */ + readonly inspected: boolean; +} + +/** + * One entry as output then source. + * + * The two halves are decided from the typed record, never from the text that + * came out of it: a document whose own prose contains the word `failed` is + * still output, and a root that closed `err` is a failure whatever it rendered. + */ +export function entryReading(request: ReplReadingRequest): ReplReading { + return Object.freeze({ + output: outputSection(request), + source: sourceSection(request), + }); +} + +/** + * What this entry produced. + * + * Four readings, and exactly one of them is true at a time. Live text is + * labelled as live because it is still arriving and the reader must not take a + * partial document for a finished one; a retained final result replaces it once + * and stops being labelled. A root that settled having rendered nothing is not + * an absence of information — it is the outcome, which is said instead. + */ +function outputSection(request: ReplReadingRequest): ReplReadingSection { + const { entry, live, inspected } = request; + const terminal = entry.terminal; + if (terminal === undefined) { + // Still going. Its own emitted text, if any has arrived. + if (live.length > 0) { + return Object.freeze({ + caption: "Output · live", + captionStyle: styleOf("waiting", { inspected }), + lines: plain(live, "output:live", styleOf("output", { inspected })), + }); + } + return Object.freeze({ + caption: "No entry output recorded at this checkpoint.", + captionStyle: styleOf("metadata", { inspected }), + lines: Object.freeze([]), + }); + } + if (terminal.output.length > 0) { + return Object.freeze({ + caption: "Output", + captionStyle: styleOf("pane-heading", { inspected }), + lines: plain(terminal.output, "output", styleOf("output", { inspected })), + }); + } + // Settled with nothing rendered. The outcome is the reading, and it is read + // from the recorded status rather than from any word in any prose. + const outcome: ReplPresentationRole = + terminal.status === "ok" + ? "successful-outcome" + : terminal.status === "err" + ? "failed-outcome" + : "waiting"; + const said = terminal.status === "cancelled" ? "cancelled" : `closed ${terminal.status}`; + // The reason, and only where there is one to give. An `ok`, a cancellation and + // an entry that never settled have no reason to show, and inventing text for + // them would describe a failure that did not happen. Its own row, so the + // outcome and the reason are two rows a window can move rather than one row + // this pane has to break in the middle. + const reason = + terminal.status === "err" && terminal.message !== undefined + ? plain( + `failed: ${terminal.message}`, + "output:reason", + styleOf("failed-outcome", { inspected }), + ) + : []; + return Object.freeze({ + caption: "No rendered output.", + captionStyle: styleOf("metadata", { inspected }), + lines: Object.freeze([ + ...plain(said, "output:outcome", styleOf(outcome, { inspected })), + ...reason, + ]), + }); +} + +/** Text as lines nothing has been observed in. */ +function plain(text: string, prefix: string, style: ReplRowStyle): readonly ReplReadingLine[] { + return Object.freeze( + text.split("\n").map((one, index) => + Object.freeze({ + key: `${prefix}:${index}`, + text: one, + runs: tokenRuns([{ text: one, token: style.role }]), + rail: "rail-pending" as const, + depth: 0, + badge: undefined, + style, + }), + ), + ); +} + +/** + * The source this entry admitted, with what each part of it is doing. + * + * A refusal is not a blank pane: inspection refusing this text means the + * scanner would refuse it too, and a reader looking at source the engine will + * not accept needs to be told that rather than shown unmarked prose. The exact + * text is still shown, because it is theirs. + */ +function sourceSection(request: ReplReadingRequest): ReplReadingSection { + const { entry, inspected } = request; + const lines: ReplReadingLine[] = []; + const build = new Build(request, lines); + build.region(entry.source, "document", entry.scope, undefined, 0, "src"); + return Object.freeze({ + caption: "Source", + captionStyle: styleOf("pane-heading", { inspected }), + lines: Object.freeze(lines), + }); +} + +/** Where one element's phase and its rail came from. */ +interface Observation { + readonly rail: ReplPresentationRole; + readonly opening: readonly ReplTokenRun[]; + readonly closing: readonly ReplTokenRun[] | undefined; +} + +/** Source nothing has been observed in. Not a phase: an absence of one. */ +const UNOBSERVED: Observation = Object.freeze({ + rail: "rail-pending", + opening: Object.freeze([]), + closing: undefined, +}); + +/** + * One reading under construction. + * + * A class rather than a pile of threaded parameters because the recursion + * carries six things through every generated child region and one of them — the + * row counter — has to stay unique across all of them. + */ +class Build { + readonly #request: ReplReadingRequest; + readonly #lines: ReplReadingLine[]; + #next = 0; + + constructor(request: ReplReadingRequest, lines: ReplReadingLine[]) { + this.#request = request; + this.#lines = lines; + } + + /** + * One text as reading lines, recursing into the elements it is written with. + * + * `owner` is the scope whose recorded positions index this text, and + * `generated` names the fragment this text *is* when it is one. The two + * together are what an observation is matched on, which is why neither is + * ever guessed from a name or a sibling. + */ + region( + text: string, + kind: "document" | "fragment", + owner: ReplScope | undefined, + generated: string | undefined, + depth: number, + prefix: string, + ): void { + const read = inspectSource(text, kind); + if (!read.ok) { + // The scanner refuses this text. Its exact bytes are still the reader's, + // and the refusal is said rather than drawn as ordinary prose. + this.#say( + `this source cannot be read: ${read.error.message}`, + "failed-outcome", + UNOBSERVED.rail, + depth, + `${prefix}:refused`, + ); + this.#plain(text, depth, UNOBSERVED.rail, prefix); + return; + } + this.#elements(text, read.value, 0, text.length, owner, generated, depth, prefix); + } + + /** + * The elements written between two offsets, and the prose between them. + * + * Nesting comes from canonical interval containment and from nothing else — + * not from a name, not from indentation, not from the order two tags happen + * to appear in. An element whose whole span lies inside another's is that + * one's child; one that merely follows it is its sibling. + */ + #elements( + text: string, + elements: readonly SourceElement[], + from: number, + to: number, + owner: ReplScope | undefined, + generated: string | undefined, + depth: number, + prefix: string, + ): void { + let at = from; + for (const [index, element] of elements.entries()) { + const span = spanOf(element); + if (span.start < from || span.end > to) { + continue; + } + if (span.start < at) { + // Inside one already emitted as a child of an earlier element. + continue; + } + if (span.start > at) { + this.#plain( + text.slice(at, span.start), + depth, + UNOBSERVED.rail, + `${prefix}:${index}:before`, + ); + } + this.#element(text, elements, element, owner, generated, depth, `${prefix}:${index}`); + at = span.end; + } + if (at < to) { + this.#plain(text.slice(at, to), depth, UNOBSERVED.rail, `${prefix}:tail`); + } + } + + /** + * One element: its opening delimiter, what it encloses, its closing one. + * + * The rail runs the whole way through, continuations and generated children + * included, because it says which region a row is in. The badge appears on + * the first line of each delimiter only. + */ + #element( + text: string, + elements: readonly SourceElement[], + element: SourceElement, + owner: ReplScope | undefined, + generated: string | undefined, + depth: number, + prefix: string, + ): void { + // One reading per actual call, in the order they were observed, each in + // its own source group. An element nothing was observed of is read once + // and unobserved, which is the same shape with no call in it. + const calls = this.#occurrences(element, owner, generated); + if (calls.length === 0) { + this.#call(text, elements, element, owner, generated, depth, prefix, undefined); + return; + } + for (const [index, one] of calls.entries()) { + this.#call( + text, + elements, + element, + owner, + generated, + depth, + calls.length === 1 ? prefix : `${prefix}@${index}`, + one, + ); + } + } + + /** One call of one element, as its delimiters and what they enclose. */ + #call( + text: string, + elements: readonly SourceElement[], + element: SourceElement, + owner: ReplScope | undefined, + generated: string | undefined, + depth: number, + prefix: string, + call: ReplOccurrence | undefined, + ): void { + const observed = this.#observed(call); + // A self-closing element has one delimiter, so whatever would have been + // said beside its closing tag is said beside the only tag it has. Dropping + // it would leave an element that plainly settled reading as unobserved. + const opening = + element.closing === undefined ? (observed.closing ?? observed.opening) : observed.opening; + this.#delimiter( + text.slice(element.opening.start, element.opening.end), + depth, + observed.rail, + `${prefix}:open`, + opening, + ); + // What this element's enclosure holds: the fragment it generated, if it + // admitted one, and otherwise the source its author wrote there. + const replacement = this.#generated(element, owner, generated); + if (replacement !== undefined) { + this.region( + replacement.source, + "fragment", + replacement.scope, + replacement.identity, + depth + 1, + `${prefix}:gen`, + ); + } else if (element.closing !== undefined) { + this.#elements( + text, + elements, + element.opening.end, + element.closing.start, + owner, + generated, + depth + 1, + `${prefix}:in`, + ); + } + if (element.closing !== undefined) { + this.#delimiter( + text.slice(element.closing.start, element.closing.end), + depth, + observed.rail, + `${prefix}:close`, + observed.closing, + ); + } + } + + /** + * A delimiter as lines, with its badge on the first of them. + * + * A tag written across several lines is several lines here too, and only the + * first carries the reading — a second badge on a continuation would say the + * phase belonged to half a tag. + */ + #delimiter( + text: string, + depth: number, + rail: ReplPresentationRole, + prefix: string, + badge: readonly ReplTokenRun[] | undefined, + ): void { + const shown = badge !== undefined && badge.length > 0 ? badge : undefined; + for (const [index, one] of text.split("\n").entries()) { + this.#push({ + key: `${prefix}:${index}`, + text: one, + runs: sourceRuns(one), + rail, + depth, + badge: index === 0 ? shown : undefined, + style: styleOf("source", { inspected: this.#request.inspected }), + }); + } + } + + /** + * Prose as lines, each cut only at its own explicit newline. + * + * A blank logical line stays one row: dropping it would edit the shape of + * somebody's document, and the whole point of this pane is that it is theirs. + */ + #plain(text: string, depth: number, rail: ReplPresentationRole, prefix: string): void { + if (text.length === 0) { + return; + } + const pieces = text.split("\n"); + // A region that began right after a newline ends its predecessor's row + // rather than opening an empty one of its own. + for (const [index, one] of pieces.entries()) { + if (one.length === 0 && (index === 0 || index === pieces.length - 1)) { + continue; + } + this.#push({ + key: `${prefix}:${index}`, + text: one, + runs: sourceRuns(one), + rail, + depth, + badge: undefined, + style: styleOf("source", { inspected: this.#request.inspected }), + }); + } + } + + #say( + text: string, + role: ReplPresentationRole, + rail: ReplPresentationRole, + depth: number, + key: string, + ): void { + this.#push({ + key, + text, + runs: tokenRuns([{ text, token: role }]), + rail, + depth, + badge: undefined, + style: styleOf(role, { inspected: this.#request.inspected }), + }); + } + + #push(line: Omit & { readonly key: string }): void { + this.#next += 1; + this.#lines.push(Object.freeze({ ...line, key: `${line.key}#${this.#next}` })); + } + + /** + * What has actually been observed of this element, or nothing. + * + * Matched on the entry's own lifecycle reading, the recorded source position + * and the fragment that position belongs to. A retained prefix has no + * observations at all, so every element in it reads as unobserved — which is + * the truth: static syntax does not prove execution. + */ + #observed(one: ReplOccurrence | undefined): Observation { + if (one === undefined) { + return UNOBSERVED; + } + const waiting = one.waiting.length > 0; + switch (one.phase) { + case "enter": + return reading(waiting ? "rail-waiting" : "rail-active", waiting ? ["waiting"] : ["enter"]); + case "active": + return reading( + waiting ? "rail-waiting" : "rail-active", + waiting ? ["waiting"] : ["active"], + ); + // An element whose body returned is releasing what it acquired. The wait + // is said as well as the exit, and the rail stays the exit's: a cleanup + // wait is a thing happening *during* exit, not instead of it. + case "exit": + return reading("rail-exit", [], waiting ? ["exit", "waiting"] : ["exit"]); + case "settled": + return reading("rail-settled", [], ["settled"]); + case "failed": + return reading("rail-settled", [], ["failed"]); + // This terminal's existing word for a reading that stopped. The archive + // gave cancellation no glyph, and inventing one would make it a seventh + // phase nobody specified. + case "cancelled": + return Object.freeze({ + rail: "rail-settled" as const, + opening: Object.freeze([]), + closing: tokenRuns([{ text: "cancelled", token: "waiting" }]), + }); + } + } + + /** + * Every call of this element, in the order they were observed. + * + * A document that writes one element inside a ``, or writes the same + * element twice, produces several *calls* of it at one source position. Each + * is its own reading: showing only the last would say the earlier ones never + * happened, and showing one badge for all of them would say several calls + * were one. + */ + #occurrences( + element: SourceElement, + owner: ReplScope | undefined, + generated: string | undefined, + ): readonly ReplOccurrence[] { + const { lifecycle, entry } = this.#request; + if (lifecycle.entry !== entry.key) { + return []; + } + const path = owner?.path; + const found: ReplOccurrence[] = []; + for (const one of lifecycle.occurrences) { + const position = one.position; + if (position === undefined || position.offset !== element.opening.start) { + continue; + } + if (position.generatedSource !== generated) { + continue; + } + if (generated === undefined && path !== undefined && position.path !== path) { + continue; + } + found.push(one); + } + return found; + } + + /** + * The generated fragment this element admitted, or nothing. + * + * Exactly one `Evaluate` opening at the owner's canonical recorded position. + * Nothing is matched by name, by sibling order or by what the fragment looks + * like: an ambiguous or absent ownership keeps the producer on screen, which + * is what the author actually wrote, instead of attributing somebody's + * generated text to the wrong enclosure. + */ + #generated( + element: SourceElement, + owner: ReplScope | undefined, + generated: string | undefined, + ): + | { readonly source: string; readonly scope: ReplScope | undefined; readonly identity: string } + | undefined { + if (owner === undefined || element.name !== "Evaluate") { + return undefined; + } + const admitted = owner.generated.filter((one) => admittedAt(one, element, owner, generated)); + if (admitted.length !== 1) { + return undefined; + } + const one = admitted[0]; + const source = one.source; + if (source === undefined) { + return undefined; + } + // The scope this fragment was admitted as, so its own elements are matched + // against its own recorded positions. + const scope = owner.scopes.find( + (child) => child.kind === "generated" && child.marker === one.marker, + ); + return { source, scope, identity: one.marker }; + } +} + +/** + * Whether one admitted fragment belongs to this element's enclosure. + * + * A refusal keeps the producer visible; so does a fragment whose recorded + * position is not this opening, or belongs to a different text than the one + * being read. + */ +function admittedAt( + one: ReplGenerated, + element: SourceElement, + owner: ReplScope, + generated: string | undefined, +): boolean { + if (one.decision !== "admitted" || one.source === undefined) { + return false; + } + const scope = owner.scopes.find( + (child) => child.kind === "generated" && child.marker === one.marker, + ); + const position = scope?.position; + if (position === undefined) { + return false; + } + return position.offset === element.opening.start && position.generatedSource === generated; +} + +function reading( + rail: ReplPresentationRole, + opening: readonly (keyof typeof BADGES)[], + closing?: readonly (keyof typeof BADGES)[], +): Observation { + return Object.freeze({ + rail, + opening: badgeRuns(opening), + closing: closing === undefined ? undefined : badgeRuns(closing), + }); +} + +/** + * One badge as its runs. + * + * Glyph and word together, as the accepted presentation insists, so a reading + * survives a reader who cannot tell two of these colours apart. Two readings in + * one badge keep their own accents and are joined by a separator that is + * neither of them. + */ +function badgeRuns(phases: readonly (keyof typeof BADGES)[]): readonly ReplTokenRun[] { + const parts: { readonly text: string; readonly token: ReplPresentationRole }[] = []; + for (const [index, phase] of phases.entries()) { + const badge = BADGES[phase]; + if (index > 0) { + parts.push({ text: " · ", token: "punctuation" }); + } + parts.push({ text: `${badge.glyph} ${badge.word}`, token: badge.role }); + } + return tokenRuns(parts); +} + +function spanOf(element: SourceElement): { readonly start: number; readonly end: number } { + return { + start: element.opening.start, + end: element.closing === undefined ? element.opening.end : element.closing.end, + }; +} diff --git a/packages/cli/tests/fixtures/repl/presentation.ts b/packages/cli/tests/fixtures/repl/presentation.ts index cc9ef6e5a..b548f658a 100644 --- a/packages/cli/tests/fixtures/repl/presentation.ts +++ b/packages/cli/tests/fixtures/repl/presentation.ts @@ -705,6 +705,10 @@ export function measuringContext(size: ReplTerminalSize): ReplPresentationContex entriesWindowed: false, entriesRows: undefined, capture: "capture", + // The measuring pass has no fitted reading or preview, which is what makes + // it the pass that asks how wide the panes holding them are. + reading: undefined, + preview: undefined, }; } diff --git a/packages/cli/tests/fixtures/repl/reference-style.ts b/packages/cli/tests/fixtures/repl/reference-style.ts index e42f5fdd1..ad8f2c3cc 100644 --- a/packages/cli/tests/fixtures/repl/reference-style.ts +++ b/packages/cli/tests/fixtures/repl/reference-style.ts @@ -99,6 +99,45 @@ export const SYNTAX = Object.freeze({ nul: 0x8b959c, }); +/** + * The archive's `L`: what one call of one element is doing. + * + * Glyph *and* word, as the archive's own comment insists — "never colour + * alone". A reader who cannot distinguish #5aa87c from #4f8468, or who is + * reading a capture through a filter, still has `ENTER` and `SETTLED` to read. + */ +export const LIFECYCLE = Object.freeze({ + /** `enter` — a call has begun. */ + enter: Object.freeze({ glyph: "\u25b6", word: "ENTER", colour: 0x5aa87c }), + /** `active` — it is expanding. */ + active: Object.freeze({ glyph: "\u25cf", word: "ACTIVE", colour: 0x7fd3e8 }), + /** `hold` — it is waiting on somebody. */ + hold: Object.freeze({ glyph: "\u25cf", word: "WAITING", colour: 0xc99a3f }), + /** `exit` — its body returned and its resources are being released. */ + exit: Object.freeze({ glyph: "\u25c0", word: "EXIT", colour: 0xc2766e }), + /** `settled` — it finished. */ + settled: Object.freeze({ glyph: "\u2713", word: "SETTLED", colour: 0x4f8468 }), + /** `fail` — it failed. */ + fail: Object.freeze({ glyph: "\u00d7", word: "FAILED", colour: 0xd24b3f }), +}); + +/** + * The archive's `RAIL`: the column beside a reading, one value per phase. + * + * `held` is the archive's rail for a group whose body is collapsed, which this + * terminal has no equivalent of; it is transcribed for completeness and not + * sampled. `pend` is the one a reading uses for source nothing has been + * observed in. + */ +export const RAIL = Object.freeze({ + held: 0x3c4a44, + active: 0x2f6272, + exit: 0x5d3c39, + settled: 0x2e3c35, + hold: 0x4a3a1c, + pend: 0x252b30, +}); + /** * The two cues this terminal keeps that the archive states differently. * diff --git a/packages/cli/tests/fixtures/repl/surface.ts b/packages/cli/tests/fixtures/repl/surface.ts index 2eeecf665..ea6c05393 100644 --- a/packages/cli/tests/fixtures/repl/surface.ts +++ b/packages/cli/tests/fixtures/repl/surface.ts @@ -451,11 +451,11 @@ export function fixturePairs( ): readonly ReplBox[] => { const id = `box:${window}:viewport`; viewports.push(Object.freeze({ id, region, window })); - const less = paired(CONTROL, `${window}:earlier`, "[^ earlier]", region, { + const less = paired(CONTROL, `${window}:earlier`, "[↑ earlier]", region, { control: true, extra: { window, delta: -1 }, }); - const more = paired(CONTROL, `${window}:later`, "[v later]", region, { + const more = paired(CONTROL, `${window}:later`, "[↓ later]", region, { control: true, extra: { window, delta: 1 }, }); @@ -601,11 +601,11 @@ export function fixturePairs( const id = `box:${DRAWER}:viewport`; viewports.push(Object.freeze({ id, region: "drawer", window: DRAWER })); regions.push({ region: "drawer", id: "box:drawer:layer" }); - const less = paired(CONTROL, "drawer:earlier", "[^ earlier]", "drawer", { + const less = paired(CONTROL, "drawer:earlier", "[↑ earlier]", "drawer", { control: true, extra: { window: DRAWER, delta: -1 }, }); - const more = paired(CONTROL, "drawer:later", "[v later]", "drawer", { + const more = paired(CONTROL, "drawer:later", "[↓ later]", "drawer", { control: true, extra: { window: DRAWER, delta: 1 }, }); diff --git a/packages/cli/tests/repl-agent-execution.test.ts b/packages/cli/tests/repl-agent-execution.test.ts index 0ff0936ec..eff91a92f 100644 --- a/packages/cli/tests/repl-agent-execution.test.ts +++ b/packages/cli/tests/repl-agent-execution.test.ts @@ -1163,6 +1163,67 @@ describe("P1 — exact policy and per-turn suspension", () => { }); }); +describe("L2 — a pending permission is a waiting element (#881 PR 2)", () => { + beforeAll(() => useTempFileCompiler()); + + it("L2: the element the request was asked on behalf of is waiting, and stops when it is answered", function* () { + const stub = createStub({ + one: { permission: { toolCallId: "call-1", kind: "execute" }, deltas: ["reply"] }, + }); + yield* useStub(stub); + const holder = execution(); + const session = opened(yield* start(holder, ONE_PROMPT, "approve-reads")); + + yield* spawn(function* () { + yield* reported(session, "a pending request", (reading) => reading.requests.length === 1); + const pending = session.agent.requests[0]!; + + // The waiting is counted on the element the request was asked inside — + // the `` whose turn owns it — and not on the session. A reading + // that said "busy" would name every element here. + const waiting = session.lifecycle.occurrences.filter((one) => one.waiting.length > 0); + expect(waiting.map((one) => `${one.name}:${one.waiting.join()}`)).toEqual([ + "Prompt:permission", + ]); + + // Answering it releases that element and nothing else. + expect(session.permissions.choose(pending.key, "once")).toBe(true); + yield* sleep(0); + expect(session.lifecycle.occurrences.filter((one) => one.waiting.length > 0)).toEqual([]); + }); + + const outcome = yield* session.join(); + expect(outcome.ok).toBe(true); + // And the reading goes with the entry, the way every other one does. + yield* sleep(0); + expect(session.lifecycle.occurrences).toEqual([]); + }); + + it("L2: a request abandoned at teardown releases its wait without deciding anything", function* () { + const stub = createStub({ + one: { permission: { toolCallId: "call-1", kind: "execute" }, deltas: ["reply"] }, + }); + yield* useStub(stub); + const holder = execution(); + // Held inside a scope of its own, so leaving the scope is the teardown. + const left = yield* scoped(function* () { + const session = opened(yield* start(holder, ONE_PROMPT, "approve-reads")); + yield* reported(session, "a pending request", (reading) => reading.requests.length === 1); + expect(session.lifecycle.occurrences.some((one) => one.waiting.includes("permission"))).toBe( + true, + ); + // Nobody answers. + return session; + }); + yield* sleep(0); + // The scope went away and took the reading with it. No decision was + // invented on the way out: a denial recorded here would be a decision + // nobody made. + expect(left.lifecycle.occurrences).toEqual([]); + expect(stub.outcomes.get("call-1")).toBe(undefined); + }); +}); + describe("P2 — one live request, one direct settlement", () => { beforeAll(() => useTempFileCompiler()); diff --git a/packages/cli/tests/repl-agent-interface.test.ts b/packages/cli/tests/repl-agent-interface.test.ts index 09c84067a..a54c5bf86 100644 --- a/packages/cli/tests/repl-agent-interface.test.ts +++ b/packages/cli/tests/repl-agent-interface.test.ts @@ -65,6 +65,7 @@ import { presentationFor, viewFor, } from "../src/repl/application.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; import type { ReplAction, ReplIntent, @@ -462,6 +463,7 @@ function liveReading(session: ReplSession): ReplLive { pausable: session.controller !== undefined, running: session.live, agent: session.agent, + lifecycle: NO_LIFECYCLE, }; } @@ -1115,10 +1117,10 @@ describe("U1 — one chronology, filtered, undisturbed by the background", () => expect((yield* turnRows(filtered)).map((one) => one.label.includes(later))).toEqual([true]); expect( (yield* keysOf(filtered)).filter( - (key) => key.startsWith("entry:") || key.startsWith("line:"), + (key) => key.startsWith("entry:") || key.startsWith("reading:"), ), ).toEqual( - (yield* keysOf(view)).filter((key) => key.startsWith("entry:") || key.startsWith("line:")), + (yield* keysOf(view)).filter((key) => key.startsWith("entry:") || key.startsWith("reading:")), ); // And All puts every turn back, removing only the filter. @@ -1752,7 +1754,7 @@ describe("U3 — what each accepted frame mounts, and nothing else", () => { // column are not mounted, so they are in no frame, no target map and no // pointer's way. const mounted = tree.mounted().map((id) => tree.keyOf(id) ?? ""); - for (const prefix of ["entry:", "scope:", "line:", "binding:", "elicit:"]) { + for (const prefix of ["entry:", "scope:", "reading:", "binding:", "elicit:"]) { expect(mounted.filter((key) => key.startsWith(prefix))).toEqual([]); expect((yield* keysOf(view)).filter((key) => key.startsWith(prefix))).toEqual([]); } @@ -1763,7 +1765,9 @@ describe("U3 — what each accepted frame mounts, and nothing else", () => { const placed = frame.keys; expect(placed.some((key) => key.startsWith("sessions:turn:"))).toBe(true); expect(placed).toContain("entries:heading"); - expect(placed.filter((key) => key.startsWith("entry:") || key.startsWith("line:"))).toEqual([]); + expect(placed.filter((key) => key.startsWith("entry:") || key.startsWith("reading:"))).toEqual( + [], + ); // Every target this frame offers is a control, and every one of them is // mounted: nothing offers itself to a pointer and then does nothing. for (const key of placed.filter((one) => frame.targetable(one))) { @@ -3368,10 +3372,14 @@ describe("U9 — two entries whose Prompts share one durable name", () => { /** The transcript lines one view draws, in order, excluding the live overlay. */ function* transcriptLines(view: ReplView): Operation { - return (yield* describedBy(view)) - .filter((one) => one.key.startsWith("line:") && !one.key.startsWith("line:live:")) - .map((one) => one.label.trim()) - .filter((label) => label.length > 0); + return ( + (yield* describedBy(view)) + .filter((one) => one.key.startsWith("reading:") && !one.key.includes(":live")) + // Every reading row opens with its rail, which says which region the row is + // in rather than what the row says. + .map((one) => one.label.replace(/^[\u2502]\s*/, "").trim()) + .filter((label) => label.length > 0) + ); } /** The catalog rows one view describes, in the order it describes them. */ diff --git a/packages/cli/tests/repl-agent-journey.test.ts b/packages/cli/tests/repl-agent-journey.test.ts index 8d0d06b55..ef4e8c17d 100644 --- a/packages/cli/tests/repl-agent-journey.test.ts +++ b/packages/cli/tests/repl-agent-journey.test.ts @@ -333,7 +333,9 @@ describe("J1 — the Story, from one entry to one README", () => { expect(shows(terminal, "generated generated")).toBe(true); // And both Plan reviews are retained where they were asked, which is // inside that scope rather than in the entry. - expect(occurrences(terminal, "answered @executablemd/cli/Plan.md")).toBe(2); + expect( + yield* answeredUnder(terminal, "component Plan", "answered @executablemd/cli/Plan"), + ).toBe(2); // Two turns in one conversation, and nothing else was asked. expect(fake.prompts).toHaveLength(2); @@ -635,12 +637,19 @@ describe("J3 — durable truth, and a cold process over it", () => { // Both turns are recorded rows rather than a live process's readings, and // both Plan reviews are retained where they were asked. expect(occurrences(second.terminal, "completed, recorded")).toBe(2); - expect(occurrences(second.terminal, "answered @executablemd/cli/Plan.md")).toBe(2); + expect( + yield* answeredUnder(second.terminal, "component Plan", "answered @executablemd/cli/Plan"), + ).toBe(2); // The program's own two questions are retained with the answers they // took. Their positions carry no path — generated source is not a file — // so they are shown under the fragment that produced them. - expect(occurrences(second.terminal, 'answered 4:1 {"project":"Ledger"')).toBe(1); - expect(occurrences(second.terminal, 'answered 12:1 {"decision":"Approv')).toBe(1); + yield* click(second.terminal, "generated generated"); + yield* settled(40); + // The answers themselves are read from the reading above, which shows + // them in full; what this counts is that each question is retained under + // the fragment that asked it, at the position it was written. + expect(occurrences(second.terminal, "answered 4:1")).toBe(1); + expect(occurrences(second.terminal, "answered 12:1")).toBe(1); // And the branch that was not taken is on the screen once, as source. expect(occurrences(second.terminal, STOPPED)).toBe(1); @@ -1323,13 +1332,13 @@ describe("J2 — what filtering Sessions leaves on the screen (#875 R1)", () => /** * The rows the Sessions reading itself is drawn in. * - * The sidebar band, cut at the reading's own `[v later]` control: below that + * The sidebar band, cut at the reading's own `[↓ later]` control: below that * sit the Entries heading and the catalog, which a conversation filter does * not touch, and beside it are columns holding a journal that grows. */ const listBand = (terminal: Terminal): string[] => { const rows = screenOf(terminal).map((line) => line.slice(0, SIDEBAR)); - const bottom = rows.findIndex((line) => line.includes("[v later]")); + const bottom = rows.findIndex((line) => line.includes("[↓ later]")); return rows.slice(0, bottom === -1 ? rows.length : bottom); }; @@ -1382,7 +1391,7 @@ describe("J2 — what filtering Sessions leaves on the screen (#875 R1)", () => const before = listBand(terminal); const lastBefore = lastWritten(before); // Pre-assert: the reading really is windowed, so there is a reading to - // shrink and a `[v later]` to bound it by. + // shrink and a `[↓ later]` to bound it by. expect(before.length).toBeGreaterThan(0); expect(lastBefore).toBeGreaterThan(0); @@ -2281,7 +2290,7 @@ function historyMarkers(terminal: Terminal): string[] { const found: string[] = []; for (let row = at + 1; row < rows.length; row += 1) { const label = (rows[row] ?? "").trim().replace(/^>\s*/, ""); - if (label === "[^ earlier]") { + if (label === "[↑ earlier]") { // How the window moves, not a position in it. This drawer scrolls (#875), // and both window controls sit outside the content they scroll. continue; @@ -2314,6 +2323,32 @@ function occurrences(terminal: Terminal, text: string): number { return screenOf(terminal).filter((line) => line.includes(text)).length; } +/** + * Select one scope by its catalog row, and count what it says was answered. + * + * #881 PR 2 gave the Transcript the selected entry's reading, so the records of + * every scope no longer appear there at once; the answers a nested scope + * retained are read by selecting that scope, which is where the product says + * they were asked. Selecting is the claim: a row counted under the wrong scope + * would be an answer attributed to somewhere it never happened. + */ +function* answeredUnder(terminal: Terminal, scope: string, text: string): Operation { + yield* click(terminal, scope); + yield* settled(40); + const found = occurrences(terminal, text); + if (found === 0) { + throw new Error( + `after selecting ${scope}, no row said ${text}: ${JSON.stringify( + screenOf(terminal) + .map((line) => line.trimEnd()) + .filter((line) => line.length > 0) + .slice(0, 12), + )}`, + ); + } + return found; +} + /** Every row of the screen holding this text, trimmed, in order. */ function shown(terminal: Terminal, text: string): string[] { return screenOf(terminal) @@ -2471,8 +2506,12 @@ function* showing(terminal: Terminal, expected: string): Operation { function coordinateOf( terminal: Terminal, label: string, + from = 0, ): { readonly column: number; readonly row: number } | undefined { for (const [row, line] of screenOf(terminal).entries()) { + if (row < from) { + continue; + } const column = line.indexOf(label); if (column !== -1) { return { column, row }; @@ -2481,6 +2520,25 @@ function coordinateOf( return undefined; } +/** + * Where the open drawer begins, as the row its own top rule is on. + * + * Needed to tell a control inside the drawer from text that merely reads the + * same outside it: #881 PR 2 gave the Transcript the entry's source, and a + * question's own schema is written in that source — so `Project name` is on the + * screen twice, once as the field to click and once as the `title` the author + * typed. A pointer aimed at the second lands behind the modal and does nothing. + */ +function drawerTop(terminal: Terminal): number { + for (const [row, line] of screenOf(terminal).entries()) { + const said = line.trim(); + if (said.length > 8 && /^\u2500+$/.test(said)) { + return row; + } + } + return 0; +} + /** * Click one control, by finding it on the screen and pressing there. * @@ -2488,8 +2546,8 @@ function coordinateOf( * pointer needs no focus, so it can reach a control while an execution is still * moving. The protocol counts from one and the screen counts from zero. */ -function* click(terminal: Terminal, label: string): Operation { - const at = coordinateOf(terminal, label); +function* click(terminal: Terminal, label: string, from = 0): Operation { + const at = coordinateOf(terminal, label, from); if (at === undefined) { throw new Error(`no control labelled ${label} is on the screen`); } @@ -2651,13 +2709,14 @@ function* focusDraft(terminal: Terminal): Operation { /** Type a value into the field this label names. */ function* answer(terminal: Terminal, label: string, value: string): Operation { - yield* click(terminal, label); + // Inside the drawer that is asking, not wherever the words first appear. + yield* click(terminal, label, drawerTop(terminal)); terminal.bytes(BYTES.encode(value)); yield* settled(20); } /** Submit the open form. */ function* submit(terminal: Terminal): Operation { - yield* click(terminal, "[submit]"); + yield* click(terminal, "[submit]", drawerTop(terminal)); yield* settled(40); } diff --git a/packages/cli/tests/repl-entries.test.ts b/packages/cli/tests/repl-entries.test.ts index 59795ead6..0a6838bf2 100644 --- a/packages/cli/tests/repl-entries.test.ts +++ b/packages/cli/tests/repl-entries.test.ts @@ -68,6 +68,7 @@ import { viewFor, withoutAbsentEntry, } from "../src/repl/application.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; import type { ReplAction, ReplLive, ReplState, ReplView } from "../src/repl/application.ts"; import { flatten, inspectionWidth, NARROW, sidebarWidth } from "../src/repl/layout.ts"; import type { ReplBounds, ReplRegion } from "../src/repl/layout.ts"; @@ -1458,6 +1459,7 @@ const NOTHING_LIVE: ReplLive = Object.freeze({ pausable: false, running: false, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }); /** This process's overlay, exactly as the program reads it into a view. */ @@ -1469,6 +1471,7 @@ function liveReading(session: ReplSession): ReplLive { pausable: session.controller !== undefined, running: session.live, agent: session.agent, + lifecycle: NO_LIFECYCLE, }; } @@ -2314,19 +2317,44 @@ function saying(what: string): string { return `${what}\n`; } -/** The transcript lines this view draws, in order. */ +/** + * The reading's rows this view draws, in order, as the text in each of them. + * + * #881 PR 2 replaced the transcript's record rows with the entry's reading, so + * these are its rows. Each one opens with its rail, which says which region the + * row is in rather than what the row says — so the rail and the gap after it are + * dropped here, leaving what a reader reads. + */ function* transcriptOf(view: ReplView): Operation { return (yield* describedBy(view)) - .filter((one) => one.key.startsWith("line:") && !one.key.startsWith("line:live:")) - .map((one) => one.label.trim()) + .filter((one) => one.key.startsWith("reading:") && !one.key.startsWith("reading:output:live")) + .map((one) => said(one.label)) .filter((label) => label.length > 0); } -/** The live overlay lines this view draws, in order. */ +/** + * The Output half of the reading this view draws, caption included. + * + * For the cases whose claim is about what an entry *produced*, which the + * reading states separately from the source that produced it. + */ +function* outputOf(view: ReplView): Operation { + return (yield* describedBy(view)) + .filter((one) => one.key === "reading:caption:output" || one.key.startsWith("reading:output:")) + .map((one) => said(one.label)) + .filter((label) => label.length > 0); +} + +/** The live overlay rows this view draws, in order. */ function* overlayOf(view: ReplView): Operation { return (yield* describedBy(view)) - .filter((one) => one.key.startsWith("line:live:")) - .map((one) => one.label.trim()); + .filter((one) => one.key.startsWith("reading:output:live")) + .map((one) => said(one.label)); +} + +/** What one reading row says, without the rail that says where it is. */ +function said(label: string): string { + return label.replace(/^[\u2502]\s*/, "").trim(); } /** The Sessions rows this view draws, in order, with their labels. */ @@ -2345,11 +2373,15 @@ describe("REPL entries: the transcript belongs to the entry that is selected", ( expect(model.entries.map((entry) => entry.key)).toEqual(["entry-1", "entry-2"]); const standing = initialState(EXECUTION); - // Nothing selected is the whole execution, which is what a one-entry - // execution has always shown and what this must not change. + // Re-anchored for #881 PR 2. The transcript used to concatenate every + // entry's records when nothing was selected; it now holds one entry's + // reading, and with nothing selected that is the last entry this prefix + // admitted — which is still what a one-entry execution shows. The claim + // this case exists for is unchanged and is the one below it: no reading + // shows another entry's. const whole = yield* transcriptOf(reading(standing, model, NOTHING_LIVE, WIDE)); - expect(whole).toContain("ALPHA-ONE"); expect(whole).toContain("BRAVO-TWO"); + expect(whole).not.toContain("ALPHA-ONE"); const first = yield* transcriptOf( reading(selecting(standing, "entry-1"), model, NOTHING_LIVE, WIDE), @@ -2388,7 +2420,11 @@ describe("REPL entries: the transcript belongs to the entry that is selected", ( const standing = initialState(EXECUTION); // Reading the entry that is running: the overlay is its own, so it shows. const running = reading(selecting(standing, "entry-2"), model, live, WIDE); - expect(yield* overlayOf(running)).toContain("… CHARLIE-LIVE"); + // Re-anchored for #881 PR 2: the Output half says once that its text is + // still arriving, so each row no longer repeats an ellipsis. Both claims + // are kept — the text is shown, and it is marked as live. + expect(yield* overlayOf(running)).toContain("CHARLIE-LIVE"); + expect(yield* transcriptOf(running)).toContain("Output · live"); // Reading the settled entry before it: the overlay is somebody else's // run, and attributing it here would show text this entry never produced. @@ -2397,9 +2433,9 @@ describe("REPL entries: the transcript belongs to the entry that is selected", ( expect(yield* transcriptOf(earlier)).toContain("ALPHA-ONE"); expect(yield* transcriptOf(earlier)).not.toContain("CHARLIE-LIVE"); - // With nothing selected the locus is the execution, which includes - // whatever is running in it — unchanged from a one-entry execution. - expect(yield* overlayOf(reading(standing, model, live, WIDE))).toContain("… CHARLIE-LIVE"); + // With nothing selected the locus is the last entry admitted, which is + // the one that can still be running — so its live text is what shows. + expect(yield* overlayOf(reading(standing, model, live, WIDE))).toContain("CHARLIE-LIVE"); // Sessions is execution-wide and the same reading under every selection, // row for row and label for label. @@ -2689,10 +2725,20 @@ describe("REPL entries: what each prefix of a two-entry history shows", () => { const state = frozenAt(drafting(initialState(EXECUTION), "still typing"), marker); return yield* transcriptOf(reading(state, projected(events, marker), NOTHING_LIVE, WIDE)); }; + // The Output half alone. Re-anchored for #881 PR 2: the reading also shows + // the source each prefix admitted, and at the admission checkpoint that + // source is the second entry's own — so the claim being made here, that no + // prefix shows output it had not retained, is read off the half that + // carries output. + const outputAt = function* (marker: string): Operation { + const state = frozenAt(drafting(initialState(EXECUTION), "still typing"), marker); + return yield* outputOf(reading(state, projected(events, marker), NOTHING_LIVE, WIDE)); + }; expect(yield* linesAt(before)).toContain("ALPHA-ONE"); expect(yield* linesAt(before)).not.toContain("BRAVO-TWO"); - expect(yield* linesAt(admission)).not.toContain("BRAVO-TWO"); - expect(yield* linesAt(terminal)).toContain("BRAVO-TWO"); + expect(yield* outputAt(admission)).not.toContain("BRAVO-TWO"); + expect(yield* outputAt(admission)).toContain("No entry output recorded at this checkpoint."); + expect(yield* outputAt(terminal)).toContain("BRAVO-TWO"); // The draft is current route state throughout: it is not a thing a prefix // retained, so freezing the durable view does not freeze it. @@ -3042,6 +3088,7 @@ describe("REPL entries: what a frozen position says is unavailable", () => { pausable: true, running: true, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }; // Projected *at* the position, because a prefix is a different reading of @@ -3210,8 +3257,8 @@ describe("REPL entries: what a failed entry says it failed with", () => { /** The transcript rows this model describes, by label. */ function* transcript(model: ReplModel): Operation { return (yield* describedBy(reading(initialState(EXECUTION), model, NOTHING_LIVE, WIDE))) - .filter((one) => one.key.startsWith("line:")) - .map((one) => one.label); + .filter((one) => one.key.startsWith("reading:")) + .map((one) => said(one.label)); } it("UI13: the recorded reason is on the screen, bounded, and a cold reopen says the same", function* () { @@ -3296,21 +3343,25 @@ describe("REPL entries: what a failed entry says it failed with", () => { const frame = yield* drawn(tree, view, size); const cells = frame.keys - .filter((key) => key.startsWith("line:")) + .filter((key) => key.startsWith("reading:")) .map((key) => ({ key, text: frame.cell(key) ?? "", bounds: frame.bounds(key) })); - const failed = cells.find((cell) => cell.text.startsWith("failed: ")); - const region = frame.region("transcript"); + // The rail opens every reading row, so the reason is found by what the + // row says rather than by what the row begins with. + const failed = cells.find((cell) => cell.text.includes("failed: ")); + // The pane the reading is placed in at this size. A narrow frame routes + // one outlet and the Entries surface is it, so the reading is in + // `content` there and in `transcript` at the two wider profiles — the + // same claim about the same rows, in whichever region the profile has + // for them. + const region = frame.region("transcript") ?? frame.region("content"); if (region === undefined) { - // The narrow profile mounts one routed outlet, and the transcript is - // not one of them: `content` carries the navigation and whichever - // catalog the route chose. No transcript row is placed at this size at - // all, which is asserted rather than assumed — a reason that cannot be - // drawn cannot overflow, and a row claiming to measure one here would - // be measuring nothing. - expect([size.columns, cells.length]).toEqual([size.columns, 0]); - expect([size.columns, failed]).toEqual([size.columns, undefined]); + throw new Error(`the ${size.columns}-column frame placed the reading nowhere`); } else { + // Every profile places it, the narrow one included: routing a reader + // to the surface their entry is on and then showing them nothing of + // it is the defect this leg of the case exists for. + expect([size.columns, cells.length > 0]).toEqual([size.columns, true]); expect([size.columns, failed !== undefined]).toEqual([size.columns, true]); const drawn = failed?.bounds; // Inside its own region, by its own geometry: it starts where the @@ -3322,11 +3373,18 @@ describe("REPL entries: what a failed entry says it failed with", () => { ]).toEqual([size.columns, true]); // One row, which is what keeps everything below it where it was. expect([size.columns, drawn?.height]).toEqual([size.columns, 1]); - // And it says so wherever it had to be shortened, rather than ending - // mid-word with the mark cut off by the renderer. - if ((failed?.text.length ?? 0) === region.width) { - expect([size.columns, failed?.text.endsWith("…")]).toEqual([size.columns, true]); - } + // Re-anchored for #881 PR 2: a reason too long for the pane is now + // *wrapped* rather than shortened with a mark, so what used to be the + // tail of one clipped row is the next row of the reading. The claim + // the ellipsis stood for — no character of the recorded reason is + // thrown away — is the stronger one, and is asserted here by + // recovering the whole reason from the rows it was cut into. + const reason = cells + .filter((cell) => cell.key.startsWith("reading:output:reason:")) + .map((cell) => cell.text.replace(/^[\u2502]\s*/, "").trimEnd()) + .join(""); + expect([size.columns, reason.startsWith("failed: ")]).toEqual([size.columns, true]); + expect([size.columns, reason.includes("…")]).toEqual([size.columns, false]); } // The footer is exactly where it always is: the reason did not push a diff --git a/packages/cli/tests/repl-forms.test.ts b/packages/cli/tests/repl-forms.test.ts index 69c01160d..7699f704e 100644 --- a/packages/cli/tests/repl-forms.test.ts +++ b/packages/cli/tests/repl-forms.test.ts @@ -35,6 +35,7 @@ import { reduceRepl, viewFor, } from "../src/repl/application.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; import type { ReplAction, ReplLive, @@ -142,6 +143,7 @@ function asking(question: ReplQuestion | undefined): ReplLive { pausable: false, running: false, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }; } @@ -865,6 +867,134 @@ describe("F3 — complete content and reachable navigation", () => { } }); + it("F3: a long preview line is readable to its last character at 72x20", function* () { + // R4. A drawer row is one row, so a message split only at its newlines + // puts a long line's tail nowhere: the window can scroll to rows that + // were prepared and cannot recover the rest of one that was not. The + // reviewed build lost `ng coding agents."` off the end of the README + // summary at this size, with Approve still reachable — so reaching the + // controls is not what discriminates this. + // + // Walked through the real tree, the real admission and the real scroll, + // concatenating what the drawer actually placed. + const summary = "A lightweight workspace for coordinating coding agents."; + const long = [ + "Proposed README.md:", + "", + "{", + ` "project": "Northstar",`, + ` "summary": "${summary}"`, + "}", + "", + ].join("\n"); + const asked = yield* askingFor(PLAN_SCHEMA, long); + const live = asking(asked.question); + const tree = yield* useReplTree(); + + let state = opened(live); + const rows: string[] = []; + const seenKeys = new Set(); + for (let press = 0; press < 160; press++) { + const view = reading(state, live, EMPTY_MODEL, NARROW, undefined); + for (const [key] of yield* placedKeys(tree, view)) { + if (key.startsWith("drawer:message:") && !seenKeys.has(key)) { + seenKeys.add(key); + } + } + for (const row of yield* framed(state, live)) { + if ( + row.key.startsWith("drawer:message:") && + !rows.includes(row.key + "\u0000" + row.label) + ) { + rows.push(row.key + "\u0000" + row.label); + } + } + const next = reduceRepl( + state, + { kind: "scroll", delta: 1 }, + EMPTY_MODEL, + live, + yield* admissionOf(state, live), + ).state; + if (next.form.offset === state.form.offset) { + break; + } + state = next; + } + + // Every row the drawer showed, in the order its keys number them. + // + // Compared with runs of whitespace collapsed, because the display adds + // two kinds of its own: `drawerLine` pads each row to the drawer's width, + // and a continuation is set in to the column its line was written at. + // Neither is content, and neither can be told from the content's own + // spaces in a drawn row — so the recovery rule is "every character, in + // order, up to the whitespace the display inserted". That is exactly the + // rule that catches the defect: a dropped tail is missing characters, not + // extra spaces. + const flat = (text: string): string => text.replace(/\s+/g, " ").trim(); + const whole = flat( + rows + .map((one) => one.split("\u0000")) + .sort((a, b) => Number(a[0].split(":")[2]) - Number(b[0].split(":")[2])) + .map(([, label]) => label) + .join(" "), + ); + + // The property that makes the text *drawable*, and the one the reviewed + // build failed. A description's label always held the whole line — the + // loss happened at the row, where a label wider than the drawer has + // nowhere to put its tail and the window has no second row to scroll to. + // So every message row must fit the width the drawer was measured at. + const room = + (yield* contextOf(reading(opened(live), live, EMPTY_MODEL, NARROW, undefined))).widths + ?.drawer ?? 0; + expect(room).toBeGreaterThan(0); + const overflowing = rows + .map((one) => one.split("\u0000")) + .filter(([, label]) => label.replace(/\s+$/, "").length > room); + expect(overflowing.map(([key]) => key)).toEqual([]); + // And there really are more rows than logical lines, so this is wrapping + // rather than a message that happened to be short enough. + expect(rows.length).toBeGreaterThan(long.split("\n").length); + + // The tail the reviewed build dropped. + expect(whole).toContain("ng coding agents."); + // And the value entire, with its closing quote. + expect(whole).toContain(flat(`"summary": "${summary}"`)); + // Every logical line is in there whole, so nothing was shortened and + // nothing was invented. + for (const line of long.split("\n").filter((one) => one.trim().length > 0)) { + expect([line, whole.includes(flat(line))]).toEqual([line, true]); + } + // The decisions are still reachable afterwards, which is the half the + // reviewed build already satisfied. + const reached = new Set(); + let at = opened(live); + for (let press = 0; press < 160; press++) { + for (const [key, targetable] of yield* placedKeys( + tree, + reading(at, live, EMPTY_MODEL, NARROW, undefined), + )) { + if (targetable) { + reached.add(key); + } + } + const next = reduceRepl( + at, + { kind: "scroll", delta: 1 }, + EMPTY_MODEL, + live, + yield* admissionOf(at, live), + ).state; + if (next.form.offset === at.form.offset) { + break; + } + at = next; + } + expect(reached.has("drawer:form:submit")).toBe(true); + }); + it("F3: the viewport walks the whole message before the form controls", function* () { const asked = yield* askingFor(PLAN_SCHEMA, DRAFT); const live = asking(asked.question); @@ -1271,6 +1401,7 @@ function liveReading(session: ReplSession): ReplLive { pausable: session.controller !== undefined, running: session.live, agent: session.agent, + lifecycle: NO_LIFECYCLE, }; } diff --git a/packages/cli/tests/repl-journey.test.ts b/packages/cli/tests/repl-journey.test.ts index 292a671c3..b942b5740 100644 --- a/packages/cli/tests/repl-journey.test.ts +++ b/packages/cli/tests/repl-journey.test.ts @@ -48,6 +48,7 @@ import { reduceRepl, refusedView, } from "../src/repl/application.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; import { readDescription } from "../src/repl/description.ts"; import { runReplProgram } from "../src/repl/program.ts"; import type { ReplExecutionProfile } from "../src/repl-profile.ts"; @@ -1028,10 +1029,10 @@ function drawerMarkers(terminal: Terminal): string[] { title = true; continue; } - if (label === "[close]" || label === "[v later]") { + if (label === "[close]" || label === "[↓ later]") { break; } - if (label === "[^ earlier]") { + if (label === "[↑ earlier]") { // How the window moves, not a position in it. Both window controls stay // outside the content they scroll, so neither is one of these. continue; @@ -1044,7 +1045,7 @@ function drawerMarkers(terminal: Terminal): string[] { /** * Every content row the open drawer can show, gathered by scrolling it. * - * Through `[v later]`, the way a person reaches the rest of a long record: + * Through `[↓ later]`, the way a person reaches the rest of a long record: * pressing it moves the window by a row, and what the window holds is what is * mounted. Stops when a press adds nothing new, which is the end of the reading. */ @@ -1064,7 +1065,7 @@ function* drawerContent( // What one frame holds, before anything has been scrolled. const first = drawerMarkers(terminal); take(); - yield* focusOn(terminal, "[v later]"); + yield* focusOn(terminal, "[↓ later]"); // Stopped when the window itself stops moving, not when a press reveals no // label this walk had not already collected. A serialized value repeats rows — // `}` closes every object — so a press that only brought a duplicate into view @@ -1096,6 +1097,36 @@ function* histories(root: string): Operation { return entries.filter((name) => name.endsWith(".jsonl")).sort(); } +/** + * The Transcript column's own rows, as the columns of the screen they occupy. + * + * Sliced between the two headings that name the columns either side of it, so + * what comes back is this pane's text and not the sidebar's or the Bindings + * column's. Never trimmed: the padding between one row's text and the next is + * what a wrapped line's break became, and dropping it would join two words the + * reading kept apart. + */ +function transcriptColumn(terminal: Terminal): string[] { + const rows = screenOf(terminal); + const header = rows.findIndex((line) => line.includes("Transcript")); + if (header < 0) { + throw new Error(`this screen has no Transcript column; it has ${JSON.stringify(rows)}`); + } + const left = rows[header].indexOf("Transcript"); + const right = rows[header].indexOf("Bindings"); + // The pane's edge and the reading's rail become spaces: they are the frame + // saying where a row is, not part of what the row says, and a wrapped line's + // break is the whitespace either side of them. + return rows.map((line) => + line.slice(left, right < 0 ? undefined : right).replace(/[\u2502\u2500]/g, " "), + ); +} + +/** One string with every run of whitespace as one space, for comparing two. */ +function flattened(text: string): string { + return text.replace(/\s+/g, " ").trim(); +} + /** The last row of the screen that contains this text, or none. */ function lastRowContaining(rows: readonly string[], text: string): number { for (let row = rows.length - 1; row >= 0; row -= 1) { @@ -2202,23 +2233,28 @@ describe("REPL journey: the whole of it, from raw bytes", () => { yield* until_(terminal, "the answer's output", (t) => shows(t, "Decision: approve")); expect(shows(terminal, "[ok]")).toBe(true); - // The generated fragment's source is on screen above the row that admits it. - // Which two rows those are comes from the model rather than from a guess at - // their wording; which order they are in comes from the screen. + // The generated fragment's source is on screen, inside the enclosure its + // author wrote. Which text that is comes from the model rather than from a + // guess at its wording; where it sits comes from the screen. + // + // Re-anchored for #881 PR 2. This used to assert that the fragment's + // source row came above the effect row admitting it, both of which were + // transcript records. The reading replaces that with the stronger claim: + // the fragment is shown *where its producer was written*, between the + // opening and closing tags the author typed. const settledModel = yield* projectionOf(root, files[0]); const at = settledModel.transcript.findIndex((row) => row.kind === "generated"); expect(at).toBeGreaterThanOrEqual(0); const sourceRow = settledModel.transcript[at]; - const admission = settledModel.transcript.slice(at + 1).find((row) => row.kind === "effect"); - if (sourceRow?.kind !== "generated" || admission?.kind !== "effect") { - throw new Error("the settled history holds a generated fragment and its admission"); + if (sourceRow?.kind !== "generated" || sourceRow.source === undefined) { + throw new Error("the settled history holds an admitted generated fragment"); } - const rows = screenOf(terminal).map((line) => line.trim()); - const sourceAt = rows.findIndex((line) => line.includes(sourceRow.source ?? "«none»")); - const admittedAt = lastRowContaining(rows, `${admission.type} ${admission.status}`); - expect(sourceAt).toBeGreaterThanOrEqual(0); - expect(admittedAt).toBeGreaterThanOrEqual(0); - expect(sourceAt).toBeLessThan(admittedAt); + // Recovered from the rows it was wrapped into, because the reading wraps + // rather than clips: the whole of it is on the screen, across as many rows + // as the pane needed, and concatenating the pane's own rows is what gets + // it back. + const column = transcriptColumn(terminal).join(""); + expect(flattened(column)).toContain(flattened(sourceRow.source)); // 7. The complete binding value, from the drawer that holds it. yield* activate(terminal, "1. [ok] entry-1"); @@ -2354,7 +2390,12 @@ describe("REPL journey: the whole of it, from raw bytes", () => { second.terminal.feed("\x1b"); yield* settled(30); expect(shows(second.terminal, "component Checklist")).toBe(true); - expect(shows(second.terminal, "generated admitted:")).toBe(true); + // Re-anchored for #881 PR 2: the transcript's `generated admitted:` + // record row became the entry's reading, and the fragment this cold + // process reconstructed is the catalog row that names it. The claim is + // unchanged — a reopened process shows the generated region the first one + // admitted. + expect(shows(second.terminal, "generated generated")).toBe(true); expect(shows(second.terminal, "Decision: approve")).toBe(true); // The same complete binding value. @@ -2665,13 +2706,18 @@ describe("REPL journey: the same product at every size", () => { yield* settled(60); terminal.feed("\r"); yield* dismissQuestion(terminal); - yield* until_(terminal, "the transcript", (t) => shows(t, "component Checklist")); + yield* until_(terminal, "the transcript", (t) => shows(t, "About to evaluate:")); // A transcript with something in it — recorded before anything covers it, // so what follows is a claim about coverage rather than about absence. // Texts that belong to the transcript and to nothing the drawer lists, so // finding one inside the drawer means it showed through. - const underneath = ["import_component ok", "About to evaluate:"]; + // Re-anchored for #881 PR 2: the transcript's record rows became the + // entry's reading, so these are two lines of what that reading shows. + // Both belong to it and to nothing this drawer lists — the retained + // positions name `summarySource` and ` admitted`, so + // neither `Source` nor `Checklist` would discriminate anything. + const underneath = ["### Ship the REPL", "2 steps remain."]; for (const text of underneath) { expect(shows(terminal, text)).toBe(true); } @@ -3135,6 +3181,7 @@ describe("REPL journey: what it settles before it acts", () => { pausable: true, running: true, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }); expect(pausing.intent.kind).toBe("none"); expect(pausing.state.refusal).toContain("not paused"); @@ -3146,6 +3193,7 @@ describe("REPL journey: what it settles before it acts", () => { pausable: true, running: true, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }); expect(held.intent.kind).toBe("continue"); expect(held.state.refusal).toBe(undefined); @@ -3227,7 +3275,13 @@ describe("REPL journey: output nothing records still reaches the screen", () => } // No question is pending, so nothing but output has woken this screen. - expect(shows(terminal, "Decide?")).toBe(false); + // + // Re-anchored for #881 PR 2: the Transcript now shows the entry's own + // source, and `Decide?` is written in it — so the absence of a question + // is asserted as the absence of a question rather than as the absence of + // its words. That is the stronger claim and the one the reading makes: + // source on the screen never proves anything is waiting. + expect(askedRow(terminal)).toBeUndefined(); // The text after the last record this document writes. Nothing recorded // it, nothing reprojected because of it, and it is on the screen. expect(shows(terminal, PRINTED)).toBe(true); @@ -3845,8 +3899,15 @@ describe("REPL journey: what an open drawer covers, through the terminal", () => // And none of the lines it covered is anywhere inside it. Read as cells // rather than as descriptions: the blank interior of a short modal line // is exactly where a drawer without a background lets text through. + // A window control the drawer offers *itself* cannot discriminate: both + // panes spell `[^ earlier]` and `[v later]` the same way, so finding one + // inside the rectangle says the drawer has its own, not that something + // behind it showed through. Narrow routes one outlet holding the catalog + // and the entry's reading, each with its own pair, so a fragment of one + // of those labels is now among the covered lines. + const shared = ["[\u2191 earlier]", "[\u2193 later]", "[close]"]; for (const line of covered.split("\n").map((one) => one.trim())) { - if (line.length < 4) { + if (line.length < 4 || shared.some((control) => control.includes(line))) { continue; } expect([label, line, inside.includes(line)]).toEqual([label, line, false]); @@ -4784,7 +4845,7 @@ describe("REPL first use: UI10 narrow drawer guidance", () => { // And a control in the same drawer that does something else entirely. It // moves the window over a message too long to draw at once, so the row says // that instead of promising an answer to somebody who is still reading. - yield* focusOn(terminal, "[v later]"); + yield* focusOn(terminal, "[↓ later]"); const onScroll = guidanceRow(terminal); expect(onScroll).toBe("Entry 1 question · Enter scrolls · Esc closes · Tab/Shift+Tab move"); diff --git a/packages/cli/tests/repl-lifecycle.test.ts b/packages/cli/tests/repl-lifecycle.test.ts new file mode 100644 index 000000000..47a3da69a --- /dev/null +++ b/packages/cli/tests/repl-lifecycle.test.ts @@ -0,0 +1,251 @@ +/** + * Tier L — what this process is doing, element by element (#881 PR 2). + * + * Real sessions over a real stream, executing real entries. What is asserted + * is the reading the session actually published while work was in flight — + * collected from its own change stream, the way the screen consumes it — and + * not a shape a fixture assembled. Every case also checks the two things an + * observation must never do: change what executed, or outlive its entry. + */ + +import { beforeAll, describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { useTempFileCompiler } from "@executablemd/core"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { each, ensure, scoped, sleep, spawn } from "effection"; +import type { Operation, Result } from "effection"; + +import { openReplSession } from "../src/repl/session.ts"; +import type { ReplSession } from "../src/repl/session.ts"; +import type { ReplExecution } from "../src/repl/journal.ts"; +import type { ReplLifecycleReading, ReplOccurrence } from "../src/repl/lifecycle.ts"; + +function execution(): ReplExecution { + return { id: "lifecycle", stream: new InMemoryStream([]) }; +} + +function opened(result: Result): ReplSession { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +function accepted(result: Result): void { + if (!result.ok) { + throw result.error; + } +} + +/** Every reading one session published, in order. */ +interface Watched { + readonly readings: ReplLifecycleReading[]; + /** Every call of one element seen at any point, by key. */ + calls(name: string): ReplOccurrence[]; +} + +function watching(session: ReplSession): Operation { + return { + *[Symbol.iterator]() { + const readings: ReplLifecycleReading[] = []; + yield* spawn(function* () { + for (const reading of yield* each(session.lifecycleChanges)) { + readings.push(reading); + yield* each.next(); + } + }); + return { + readings, + calls(name: string): ReplOccurrence[] { + const byKey = new Map(); + for (const reading of readings) { + for (const one of reading.occurrences) { + if (one.name === name) { + byKey.set(one.key, one); + } + } + } + return [...byKey.values()]; + }, + }; + }, + }; +} + +describe("Tier L — the reading one entry leaves behind", () => { + beforeAll(() => useTempFileCompiler()); + + it("L1: a repeated element is two calls, never one overwritten by the other", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + const watched = yield* watching(session); + accepted(yield* session.submit('\n\n\n')); + yield* session.join(); + yield* sleep(0); + + const seen = watched.calls("Json"); + expect(seen.length).toBe(2); + expect(new Set(seen.map((one) => one.key)).size).toBe(2); + // Both reached a terminal observation of their own. + expect(seen.map((one) => one.phase).sort()).toEqual(["settled", "settled"]); + }); + + it("L1: a nested element is a child of the call that reached it", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + const watched = yield* watching(session); + accepted(yield* session.submit('\n')); + yield* session.join(); + yield* sleep(0); + + const outer = watched.calls("Each"); + const inner = watched.calls("Json"); + expect(outer.length).toBe(1); + expect(inner.length).toBe(2); + // Each iteration's element names the call it was reached through, and + // the two iterations are separate calls rather than one id twice. + for (const one of inner) { + expect(one.parent).toBe(outer[0].key); + } + expect(new Set(inner.map((one) => one.key)).size).toBe(2); + }); + + it("L2: an element waiting on a question says so, and stops when it is answered", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + accepted( + yield* session.submit( + 'Answer this.\n', + ), + ); + // Several turns: the phases a consumer is told travel through its own + // queue, and the question itself is published from inside the element. + for (let turn = 0; turn < 5; turn += 1) { + yield* sleep(0); + } + + const waiting = session.lifecycle.occurrences.filter((one) => one.waiting.length > 0); + // The element that is waiting, what it is waiting on, and that it is + // the only one: a session-wide "busy" would name every element here. + expect( + session.lifecycle.occurrences.map((one) => `${one.name}:${one.phase}:${one.waiting.join()}`), + ).toEqual(["Elicit:active:question"]); + expect(waiting.map((one) => one.name)).toEqual(["Elicit"]); + + const question = session.elicitation.pending; + if (question === undefined) { + throw new Error("this session is waiting on no question"); + } + question.submit({ note: "answered" }); + yield* session.join(); + yield* sleep(0); + // The entry closed, so its reading went with it. + expect(session.lifecycle.occurrences).toEqual([]); + expect(session.live).toBe(false); + }); + + it("L2: ordinary work is not a wait", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + const watched = yield* watching(session); + accepted(yield* session.submit('\n')); + yield* session.join(); + yield* sleep(0); + + const waited = watched.readings.flatMap((reading) => + reading.occurrences.filter((one) => one.waiting.length > 0), + ); + expect(waited).toEqual([]); + }); + + it("L3: an entry that closes takes its reading, and says it is no longer live", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + accepted(yield* session.submit('\n')); + yield* session.join(); + yield* sleep(0); + + expect(session.lifecycle.entry).toBe(undefined); + expect(session.lifecycle.occurrences).toEqual([]); + expect(session.live).toBe(false); + // The entry really ran: the reading going is the reading going, not the + // work never having happened. + expect(session.model.entries.length).toBe(1); + expect(session.model.entries[0]?.terminal?.status).toBe("ok"); + }); + + it("L3: everything this entry held is released before the session says it is not live", function* () { + // The frozen ordering, observed rather than read off the comment that + // states it. An installation of this entry's own records, at the moment it + // is released, what the session was still saying about itself. If the + // liveness finalizer had been registered after the things this entry + // acquires, it would run before them and this would read `false` — and a + // reader woken then would be told the entry was over while its provider, + // its middleware and its Agent attachment were still standing. + let liveWhenReleased: boolean | undefined; + let released = false; + const session = opened( + yield* openReplSession({ + execution: execution(), + installations: [ + { + *install(): Operation { + yield* ensure(() => { + released = true; + liveWhenReleased = session.live; + }); + }, + }, + ], + }), + ); + const watched = yield* watching(session); + accepted(yield* session.submit('\n')); + yield* session.join(); + yield* sleep(0); + + // The installation really was released, so this is an ordering rather + // than an assertion about something that never ran. + expect(released).toBe(true); + expect(liveWhenReleased).toBe(true); + // And by now the session says what it is. + expect(session.live).toBe(false); + // The wake arrived with no input: nothing was typed, nothing activated, + // and a reader waiting for the entry to end was still told. + expect(watched.readings.length).toBeGreaterThan(0); + expect(watched.readings[watched.readings.length - 1].occurrences).toEqual([]); + expect(watched.readings[watched.readings.length - 1].entry).toBe(undefined); + }); + + it("L3: a late producer update cannot revive a reading the entry took with it", function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + accepted(yield* session.submit('\n')); + yield* session.join(); + yield* sleep(0); + expect(session.lifecycle.occurrences).toEqual([]); + + // Whatever is still draining from the entry that closed — a terminal + // phase published as its dispatch unwound — arrives after the generation + // it belonged to is gone, and is refused by generation rather than + // appended to nothing. + for (let turn = 0; turn < 5; turn += 1) { + yield* sleep(0); + } + expect(session.lifecycle.occurrences).toEqual([]); + expect(session.lifecycle.entry).toBe(undefined); + }); + + it("L3: observing changes neither the outcome nor what the document rendered", function* () { + const source = '\n'; + const rendered: string[] = []; + for (const collect of [true, false]) { + yield* scoped(function* () { + const session = opened(yield* openReplSession({ execution: execution() })); + if (collect) { + yield* watching(session); + } + accepted(yield* session.submit(source)); + yield* session.join(); + rendered.push(session.model.entries[0]?.terminal?.output ?? ""); + }); + } + expect(rendered[0]).toBe(rendered[1]); + expect(rendered[0]).not.toBe(""); + }); +}); diff --git a/packages/cli/tests/repl-presentation-style.test.ts b/packages/cli/tests/repl-presentation-style.test.ts index 7521ab234..e760cb22a 100644 --- a/packages/cli/tests/repl-presentation-style.test.ts +++ b/packages/cli/tests/repl-presentation-style.test.ts @@ -45,6 +45,7 @@ import { reduceRepl, viewFor, } from "../src/repl/application.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; import { useReplElicitation } from "../src/repl/elicitation.ts"; import type { ReplQuestion } from "../src/repl/elicitation.ts"; import { encodeLocation } from "../src/repl/route.ts"; @@ -58,7 +59,7 @@ import type { import type { ReplDispatched } from "../src/repl/reconcile.ts"; import type { ReplDrawerRef } from "../src/repl/route.ts"; import { FOOTER_ROWS, HISTORY_LABEL, HISTORY_ROWS, NARROW } from "../src/repl/layout.ts"; -import { ACTION_ROW, DRAWER_WINDOW } from "../src/repl/application.ts"; +import { ACTION_ROW, DRAWER_WINDOW, READING_WINDOW } from "../src/repl/application.ts"; import type { ReplBounds } from "../src/repl/layout.ts"; import { BOLD, REPL_PALETTE } from "../src/repl/presentation-style.ts"; import { @@ -68,6 +69,8 @@ import { SEMANTIC, SURFACE, SYNTAX, + LIFECYCLE, + RAIL, } from "./fixtures/repl/reference-style.ts"; import type { ReplTerminalSize } from "../src/repl/terminal.ts"; import { @@ -120,8 +123,49 @@ const NOTHING_LIVE: ReplLive = Object.freeze({ pausable: false, running: false, agent: NO_AGENT, + lifecycle: NO_LIFECYCLE, }); +/** + * This process observing one element of the entry it is running. + * + * The lifecycle reading the session would publish, so a frame can be asked + * what an observed element's badge looks like where it actually lands. + */ +function observing( + entry: string, + name: string, + offset: number, + phase: "active", + path: string, +): ReplLive { + return Object.freeze({ + ...NOTHING_LIVE, + running: true, + pausable: true, + lifecycle: Object.freeze({ + entry, + occurrences: Object.freeze([ + Object.freeze({ + key: `${entry}#1`, + expansion: "x", + name, + parent: undefined, + position: Object.freeze({ + path, + generatedSource: undefined, + offset, + line: 1, + column: 1, + }), + phase, + waiting: Object.freeze([]), + }), + ]), + }), + }); +} + /** This process part-way through an entry: output on the overlay, nothing closed. */ function streaming(output: string): ReplLive { return Object.freeze({ ...NOTHING_LIVE, output, running: true, pausable: true }); @@ -176,6 +220,30 @@ function* submittedByPath(): Operation { return projected(yield* physical.readAll()); } +/** + * One line of prose longer than any pane this frame has, and one unbroken + * token no word boundary can break. + * + * The case R3 needs: text that would reach the pane beside it if nothing fitted + * it. It is a document's own prose rather than a path, because what a reader + * came for is exactly the text this screen must not shorten — so the only + * honest way to keep it inside its pane is to measure it and wrap it. + */ +const LONG_SOURCE = [ + "One line of ordinary prose, written long enough that no pane this frame has " + + "room for could hold it on a single row without reaching into the column " + + "beside it, which is the artifact this case exists to catch.", + "", + `t${"t".repeat(200)}`, + "", +].join("\n"); + +function* submittedWithLongSource(): Operation { + const physical = new InMemoryStream(); + yield* runHostEntry(physical, HOST_ENTRY_PATH, LONG_SOURCE); + return projected(yield* physical.readAll()); +} + /** The projected model, or the failure that stopped it. */ function projected(events: readonly DurableEvent[]): ReplModel { const result = projectRepl(events); @@ -337,6 +405,15 @@ function textOf(grid: TerminalGrid, bounds: ReplBounds): string { return grid.textIn(bounds)[0] ?? ""; } +/** The sidebar the catalog is drawn in. */ +function sidebarRegion(observed: Observed): ReplBounds { + const bounds = regionBounds(observed, "sidebar"); + if (bounds === undefined) { + throw new Error("this frame published no sidebar region"); + } + return bounds; +} + /** Whether one rectangle is inside another, so a row can be held to its pane. */ function within(inner: ReplBounds, outer: ReplBounds): boolean { return ( @@ -347,17 +424,20 @@ function within(inner: ReplBounds, outer: ReplBounds): boolean { ); } -/** The first transcript row of one kind, as the key the frame drew it under. */ -function transcriptKey(model: ReplModel, kind: string): string { - const at = model.transcript.findIndex((row) => row.kind === kind); - if (at < 0) { - throw new Error( - `this prefix holds no ${kind} row; it holds ${model.transcript - .map((row) => row.kind) - .join(", ")}`, - ); +/** + * One subordinate row, as the frame drew it. + * + * Before #881 PR 2 this was a transcript record row — a fact about the event + * that produced the output beside it. The transcript now holds the entry's + * reading, so the subordinate reading is taken from a row that still carries + * it: what the frame says where it has nothing retained to list. + */ +function metadataKey(observed: Observed): string { + const key = observed.keys.find((one) => one === "sessions:empty"); + if (key === undefined) { + throw new Error(`this frame drew no subordinate row; it drew ${observed.keys.join(", ")}`); } - return `line:${at}:0`; + return key; } /** Every mounted row whose cells carry the focus foreground. */ @@ -413,7 +493,11 @@ describe("REPL presentation: what one row's cells say it is", () => { it("G1: output, metadata and the two outcomes are four different readings", function* () { const model = yield* settledAndFailed(); const presenter = yield* usePresenter(WIDE); - const observed = yield* presenter.commit(reading(stateWith({}), model)); + // The entry whose result is under test, selected: a reading belongs to an + // entry, so which entry is being read is part of asking what it says. + const observed = yield* presenter.commit( + reading(selecting(stateWith({}), model.entries[0].key), model, NOTHING_LIVE, WIDE), + ); const { grid } = presenter; const transcript = regionBounds(observed, "transcript"); @@ -421,16 +505,23 @@ describe("REPL presentation: what one row's cells say it is", () => { throw new Error("this frame published no transcript region"); } - // What the run produced, and a fact about the event that produced it. The - // same eval block wrote both, so they are adjacent lines in one column and - // nothing but the style tells a reader which is which. - const output = placed(observed, transcriptKey(model, "output")); - const metadata = placed(observed, transcriptKey(model, "binding")); - expect(inkOf(grid, output).foreground).toBe(REPL_PALETTE.output); + // What the run produced, and a fact about the event that produced it. + // + // Re-anchored for #881 PR 2. These were adjacent transcript rows written by + // one eval block; the transcript now holds the entry's *reading* — its + // output and then its source — and the facts about events are the inspection + // column's. The claim is unchanged: a result and a fact about the event that + // produced it are two readings, and nothing but the style says which is + // which. + const output = placed(observed, "reading:output:0"); + const said = model.entries[0].terminal?.output.split("\n")[0] ?? ""; + expect(said.length).toBeGreaterThan(0); + const metadata = placed(observed, metadataKey(observed)); + expect(inkOfSpan(grid, output, said).foreground).toBe(REPL_PALETTE.output); expect(inkOf(grid, metadata).foreground).toBe(REPL_PALETTE.muted); - expect(inkOf(grid, output).foreground).not.toBe(inkOf(grid, metadata).foreground); + expect(inkOfSpan(grid, output, said).foreground).not.toBe(inkOf(grid, metadata).foreground); expect(within(output, transcript)).toBe(true); - expect(within(metadata, transcript)).toBe(true); + expect(within(metadata, sidebarRegion(observed))).toBe(true); // The two outcomes the file recorded, in the catalog that promises them. const sidebar = regionBounds(observed, "sidebar"); @@ -448,7 +539,7 @@ describe("REPL presentation: what one row's cells say it is", () => { // Four distinct readings, and the words survive without any of them. const colours = [ - inkOf(grid, output).foreground, + inkOfSpan(grid, output, said).foreground, inkOf(grid, metadata).foreground, inkOf(grid, settled).foreground, inkOf(grid, failed).foreground, @@ -477,10 +568,21 @@ describe("REPL presentation: what one row's cells say it is", () => { reading(selecting(stateWith({}), settled.key), model, NOTHING_LIVE, WIDE), ); const { grid } = presenter; - const first = placed(observed, `line:${at}:0`); - expect(textOf(grid, first)).toContain(record.output.split("\n")[0]); - expect(inkOf(grid, first).foreground).toBe(REPL_PALETTE.output); - expect(inkOf(grid, first).foreground).not.toBe(REPL_PALETTE.success); + // Re-anchored for #881 PR 2: the entry's result is the Output half of its + // reading rather than a transcript row keyed by the record's position. The + // claim is unchanged — the rendered document reads as output, and not as the + // word the root closed with. + const first = placed(observed, "reading:output:0"); + const said = record.output.split("\n")[0]; + expect(textOf(grid, first)).toContain(said); + // The text's own ink, not the row's first cell: every reading row opens + // with its rail, which says which region the row is in rather than what the + // row is. + expect(inkOfSpan(grid, first, said).foreground).toBe(REPL_PALETTE.output); + expect(inkOfSpan(grid, first, said).foreground).not.toBe(REPL_PALETTE.success); + // And it is labelled as the retained result rather than as live text. + expect(textOf(grid, placed(observed, "reading:caption:output"))).toContain("Output"); + expect(textOf(grid, placed(observed, "reading:caption:output"))).not.toContain("live"); // The outcome keeps its accent where an outcome is what is shown. expect(inkOf(grid, placed(observed, `entry:${settled.key}`)).foreground).toBe( @@ -507,12 +609,18 @@ describe("REPL presentation: what one row's cells say it is", () => { const { grid } = presenter; // With nothing recorded to show, the outcome itself is the row, and it keeps // the outcome accent; the reason recorded beside it is its own row. - const closed = placed(observed, `line:${at}:0`); + // Re-anchored for #881 PR 2 onto the reading's Output half; both claims and + // both accents are unchanged, and the caption now says that the absence of + // rendered text is what is being reported. + expect(textOf(grid, placed(observed, "reading:caption:output"))).toContain( + "No rendered output.", + ); + const closed = placed(observed, "reading:output:outcome:0"); expect(textOf(grid, closed)).toContain("closed err"); - expect(inkOf(grid, closed).foreground).toBe(REPL_PALETTE.failure); - const reason = placed(observed, `line:${at}:1`); + expect(inkOfSpan(grid, closed, "closed err").foreground).toBe(REPL_PALETTE.failure); + const reason = placed(observed, "reading:output:reason:0"); expect(textOf(grid, reason)).toContain("failed:"); - expect(inkOf(grid, reason).foreground).toBe(REPL_PALETTE.failure); + expect(inkOfSpan(grid, reason, "failed:").foreground).toBe(REPL_PALETTE.failure); }); it("G1: an entry that never closed reads as waiting, and says so", function* () { @@ -531,10 +639,12 @@ describe("REPL presentation: what one row's cells say it is", () => { ); // The overlay is what this process is producing, so it reads as output - // rather than as one more fact about an event. - const live = placed(observed, "line:live:0"); + // rather than as one more fact about an event. Re-anchored for #881 PR 2 + // onto the reading's Output half, which now also says it is still arriving. + const live = placed(observed, "reading:output:live:0"); expect(textOf(grid, live)).toContain("building the plan"); - expect(inkOf(grid, live).foreground).toBe(REPL_PALETTE.output); + expect(inkOfSpan(grid, live, "building the plan").foreground).toBe(REPL_PALETTE.output); + expect(textOf(grid, placed(observed, "reading:caption:output"))).toContain("live"); }); it("G1: an empty screen's placeholders are subordinate, and the draft is a surface", function* () { @@ -1147,12 +1257,19 @@ describe("REPL presentation: what a drawer says it is showing", () => { describe("REPL presentation: a row stays inside the pane it was measured for", () => { beforeAll(() => useTempFileCompiler()); - it("R3: long metadata is fitted, and the row beneath it is its own", function* () { + it("R3: a long reading is fitted, and the row beneath it is its own", function* () { // An entry submitted by its host path, which in this checkout is longer // than any pane the frame has room for. The engine clips nothing, so a row // written unbounded wraps over the cells of the row beneath it: that is the // artifact the complete gallery caught in the Transcript pane. - const model = yield* submittedByPath(); + // + // Re-anchored for #881 PR 2. The rows under test were the transcript's + // metadata records, bounded by shortening them; the transcript now holds + // the entry's reading, fitted by measuring it. Both claims are kept — what + // is drawn in a row is what that row contributed, and no row reaches the + // pane beside it — and the case still needs a row that would overflow if + // nothing bounded it. + const model = yield* submittedWithLongSource(); let exercised = false; for (const size of [WIDE, MEDIUM]) { @@ -1165,8 +1282,7 @@ describe("REPL presentation: a row stays inside the pane it was measured for", ( throw new Error(`the ${size.columns}x${size.rows} frame published no transcript interior`); } - const lines = observed.keys.filter((key) => key.startsWith("line:")); - const recordOf = (key: string) => model.transcript[Number(key.split(":")[1])]; + const lines = observed.keys.filter((key) => key.startsWith("reading:")); // Every transcript row holds what its own node contributed and nothing // else. This is the whole discrimination: an unbounded row leaves its @@ -1179,35 +1295,23 @@ describe("REPL presentation: a row stays inside the pane it was measured for", ( expect([size.columns, key, drawn]).toEqual([size.columns, key, contributed]); } - // The rows that name where something came from: an entry's path, a - // scope's name, an agent's identity. What a document or a provider - // *said* stays unbounded on purpose — that text is the thing a reader - // came for. - // - // A generated fragment's source is the one member of this class left - // unbounded. An accepted journey reads the branch a program did not take - // off this screen, which only whole source can show, so bounding it is a - // decision about what the screen owes a reader rather than a formatting - // one; it is reported with this correction rather than taken. - const bounded = new Set(["entry", "scope", "agent"]); - const metadata = lines.filter((key) => { - const record = recordOf(key); - return record !== undefined && bounded.has(record.kind); - }); - expect(metadata.length).toBeGreaterThan(0); - - for (const key of metadata) { + // Every row of the reading, held to the pane it was measured for. Each + // one is as wide as the interior and ends where the interior ends, so + // nothing of it reaches the pane beside it — which is what the engine, + // left to itself, does not do. + expect(lines.length).toBeGreaterThan(0); + for (const key of lines) { const bounds = placed(observed, key); - // As wide as the inside of its pane, and no wider. expect([key, bounds.width]).toEqual([key, inside.width]); - // And ending where the interior ends, so nothing of it reaches the - // pane beside it. expect([size.columns, key, bounds.x + bounds.width]).toEqual([ size.columns, key, inside.x + inside.width, ]); - if ((observed.cells.get(key) ?? "").length >= inside.width) { + // A row that is the continuation of a longer line: without the + // measurement that cut it, that line is what would have landed on the + // cells of the row beneath. + if (key.startsWith("reading:src:") && key.includes("+")) { exercised = true; } } @@ -1258,9 +1362,12 @@ describe("REPL presentation: the one guidance row", () => { // What is drawn in the row is what the row contributed, and so is every // transcript row beneath it: an overlong sentence lands on the cells of // the row below, which is where it was seen. - for (const key of ["guidance", ...observed.keys.filter((one) => one.startsWith("line:"))]) { - // A narrow frame describes the transcript's rows and places none of - // them, so there is nothing of theirs on this screen to read. + for (const key of [ + "guidance", + ...observed.keys.filter((one) => one.startsWith("reading:")), + ]) { + // A narrow frame describes the reading's rows and places none of them, + // so there is nothing of theirs on this screen to read. const at = key === "guidance" ? bounds : observed.boundsOf(key); if (at === undefined) { continue; @@ -1584,6 +1691,114 @@ describe("REPL presentation: focus marks the row without repainting it", () => { describe("REPL presentation: the surfaces a reading is drawn on", () => { beforeAll(() => useTempFileCompiler()); + it("P1-T3: an observed badge ends at the pane's right inner edge, in its own ink", function* () { + // The frozen requirement is that these align at the *measured source + // pane's right inner edge*. Everything asserting it so far has read the + // composed run strings, which is the model's answer — the same shape of + // claim that let a clipped drawer preview pass its own regression. This + // reads the cells. + // An entry whose source holds an element the scanner reports — a fence is + // not one, so observing at its offset would match nothing and prove + // nothing. + // Long enough that its opening delimiter wraps: "only the first visual row + // carries each badge" says nothing about an element that fits on one row, + // and a check written against one cannot see a badge repeated. + const source = '\n'; + const model = yield* scoped(function* () { + const physical = new InMemoryStream(); + yield* runEntry(physical, source); + return projected(yield* physical.readAll()); + }); + const entry = model.entries[0]; + const at = entry.source.indexOf(" key.startsWith("reading:")) + .map((key) => ({ key, bounds: observed.boundsOf(key) })) + .find((one) => one.bounds !== undefined && textOf(grid, one.bounds).includes(badge)); + if (carrying?.bounds === undefined) { + throw new Error( + `no reading row says ${badge}; they say ${observed.keys + .filter((key) => key.startsWith("reading:")) + .map((key) => textOf(grid, observed.boundsOf(key) ?? inside).trim()) + .join(" | ")}`, + ); + } + const row = textOf(grid, carrying.bounds); + const starts = row.indexOf(badge); + // Its last cell is the pane interior's last cell: right inner edge, in + // cells, not in a string this test composed for itself. + expect([carrying.key, carrying.bounds.x + starts + badge.length]).toEqual([ + carrying.key, + inside.x + inside.width, + ]); + // And it is drawn in the archive's own literal for that phase. + expect(inkOfSpan(grid, carrying.bounds, badge).foreground).toBe(LIFECYCLE.active.colour); + // The source beside it keeps its own colours, so the badge is not + // repainting the row it sits on. + expect(inkOfSpan(grid, carrying.bounds, "<").foreground).not.toBe(LIFECYCLE.active.colour); + + // Exactly one row says it. "Only the first visual row carries each badge" + // is frozen, and a second one would say the phase changed between two + // halves of one element — counted in cells, because that is where a + // reader would see it twice. + const saying = observed.keys + .filter((key) => key.startsWith("reading:")) + .map((key) => observed.boundsOf(key)) + .filter((bounds) => bounds !== undefined && textOf(grid, bounds).includes(badge)); + expect(saying.length).toBe(1); + + // And every row of the reading opens with its rail, in cells. The rail is + // what says which region a row is in, so a reading drawn without one is a + // reading with its structure removed — and until now nothing asked the + // screen whether it was there. + const railed = observed.keys + .filter((key) => key.startsWith("reading:src:") || key.startsWith("reading:output:")) + .map((key) => ({ key, bounds: observed.boundsOf(key) })) + .filter((one) => one.bounds !== undefined); + expect(railed.length).toBeGreaterThan(0); + for (const one of railed) { + const bounds = one.bounds; + if (bounds === undefined) { + continue; + } + expect([one.key, grid.at(bounds.x, bounds.y)]).toEqual([one.key, "\u2502"]); + // In the rail's own ink, which for an observed region is not the + // unobserved one: a rail painted the same everywhere says nothing. + expect([one.key, grid.styleAt(bounds.x, bounds.y).foreground]).not.toEqual([ + one.key, + REPL_PALETTE.source, + ]); + } + // The observed region really is drawn in the archive's active rail, so + // this is not satisfied by painting every row alike. + const activeRail = railed.filter( + (one) => + one.bounds !== undefined && + grid.styleAt(one.bounds.x, one.bounds.y).foreground === RAIL.active, + ); + expect(activeRail.length).toBeGreaterThan(0); + }); + it("P1-T3: an empty screen's panes, edges and uncovered area are each their own", function* () { const model = yield* settledAndFailed(); const presenter = yield* usePresenter(WIDE); @@ -1606,9 +1821,17 @@ describe("REPL presentation: the surfaces a reading is drawn on", () => { region, surface, ]); + // The bottom of the pane's own moving content, not the bottom of the + // pane. Re-anchored for #881 PR 2: the transcript's last row is now the + // reading's `[\u2193 later]` control, and a control carries the draft's + // surface wherever it is — which is this terminal's existing rule, not a + // new one. The claim is unchanged: the area a pane holds that no control + // covers is that pane's own surface. + const floor = + viewportBounds(observed, region === "transcript" ? READING_WINDOW : region) ?? bounds; expect([ region, - grid.styleAt(bounds.x + bounds.width - 1, bounds.y + bounds.height - 1).background, + grid.styleAt(bounds.x + bounds.width - 1, floor.y + floor.height - 1).background, ]).toEqual([region, surface]); } diff --git a/packages/cli/tests/repl-source-reading.test.ts b/packages/cli/tests/repl-source-reading.test.ts new file mode 100644 index 000000000..5e9b86718 --- /dev/null +++ b/packages/cli/tests/repl-source-reading.test.ts @@ -0,0 +1,890 @@ +/** + * The reading an entry is shown as (#881 PR 2, rows S1–S4). + * + * One real layout engine answers every width here. Nothing in this file works + * out where a character lands: a test that counted code units would agree with + * itself while the terminal drew something else, and the two cases this tier + * exists to catch — a wide character and a combining mark — are exactly the + * ones a count gets wrong. + * + * What is under test: + * + * - **S1** generated source *replaces* its producer inside the enclosure the + * author wrote, at the recorded position, in the owner that recorded it — + * and a missing or ambiguous ownership keeps the producer rather than + * guessing; + * - **S2** every logical line is recoverable from the rows it was cut into, + * with its roles, across a long schema, an unbroken token, wide and + * combining text and explicit blank lines; + * - **S3** the output half says what actually happened, read from the record + * rather than from any word in the prose; + * - **S4** a phase-only change reuses the same cuts and the same status + * column, and static syntax acquires no phase. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation } from "effection"; + +import { entryReading, NO_READING, READING_STATUSES } from "../src/repl/source-reading.ts"; +import type { ReplReadingLine } from "../src/repl/source-reading.ts"; +import { prepareReading, readingRuns, reserveFor } from "../src/repl/fitting.ts"; +import type { ReplPreparedReading } from "../src/repl/fitting.ts"; +import { useReplRenderer } from "../src/repl/renderer.ts"; +import type { ReplRenderer } from "../src/repl/renderer.ts"; +import { runText } from "../src/repl/description.ts"; +import { NO_LIFECYCLE } from "../src/repl/lifecycle.ts"; +import type { ReplLifecycleReading, ReplOccurrence } from "../src/repl/lifecycle.ts"; +import type { ReplEntry, ReplScope, ReplTerminal } from "../src/repl/model.ts"; +import type { ReplTerminalSize } from "../src/repl/terminal.ts"; +import { LIFECYCLE, RAIL } from "./fixtures/repl/reference-style.ts"; +import { REPL_PALETTE, runStyleOf } from "../src/repl/presentation-style.ts"; + +const WIDE: ReplTerminalSize = { columns: 160, rows: 36 }; + +function withRenderer(body: (renderer: ReplRenderer) => Operation): Operation { + return scoped(function* () { + return yield* body(yield* useReplRenderer(WIDE)); + }); +} + +function scope(over: Partial = {}): ReplScope { + return Object.freeze({ + key: "entry-1", + kind: "entry", + name: "entry", + path: "repl://entry-1", + source: "", + position: undefined, + marker: "m1", + bindings: Object.freeze([]), + elicitations: Object.freeze([]), + generated: Object.freeze([]), + scopes: Object.freeze([]), + ...over, + }); +} + +function entryOf(source: string, over: Partial = {}): ReplEntry { + return Object.freeze({ + key: "entry-1", + order: 1, + source, + path: "repl://entry-1", + scope: scope({ source }), + transcript: Object.freeze([]), + checkpoints: Object.freeze([]), + terminal: undefined, + settled: false, + bindings: Object.freeze([]), + turns: Object.freeze([]), + ...over, + }); +} + +function settled(status: "ok" | "err" | "cancelled", output: string): ReplTerminal { + return Object.freeze({ status, output, message: undefined, location: undefined }); +} + +function observing(occurrences: readonly Partial[]): ReplLifecycleReading { + return Object.freeze({ + entry: "entry-1", + occurrences: Object.freeze( + occurrences.map((one, index) => + Object.freeze({ + key: `entry-1#${index + 1}`, + expansion: "x", + name: "Elicit", + parent: undefined, + position: undefined, + phase: "active" as const, + waiting: Object.freeze([]), + ...one, + }), + ), + ), + }); +} + +function at(offset: number) { + return Object.freeze({ + path: "repl://entry-1", + generatedSource: undefined, + offset, + line: 1, + column: 1, + }); +} + +/** Every logical line's text, recovered from the rows it was cut into. */ +function recovered(prepared: ReplPreparedReading): Map { + const joined = new Map(); + for (const row of prepared.rows) { + joined.set(row.line, (joined.get(row.line) ?? "") + row.text); + } + return joined; +} + +/** + * Whether every badge of a reading ends where the reading ends. + * + * The accepted presentation aligns these at the pane's right inner edge, so + * what has to agree across rows is where a badge *finishes*, not where it + * starts: `\u25b6 ENTER` and `\u2713 SETTLED` are different widths and would start in + * different columns if they were right-aligned correctly. + */ +function rightAligned(prepared: ReplPreparedReading): readonly boolean[] { + const ends: boolean[] = []; + for (const row of prepared.rows) { + if (row.badge === undefined) { + continue; + } + ends.push(runText(readingRuns(prepared, row)).endsWith(runText(row.badge))); + } + return ends; +} + +describe("the reading one entry is shown as", () => { + describe("S1 generated source replaces its producer where it was written", () => { + // `` is written at offset 0 and the fragment it admitted is + // recorded against that offset in this owner. + const SOURCE = '\nconst x = 1;\n\n'; + const FRAGMENT = '# generated\n\n\n'; + + function owning(decision: "admitted" | "refused", offset: number): ReplEntry { + const child = scope({ + key: "entry-1/generated-1", + kind: "generated", + name: "generated", + marker: "g1", + source: FRAGMENT, + position: Object.freeze({ + path: "repl://entry-1", + generatedSource: undefined, + offset, + line: 1, + column: 1, + }), + }); + return entryOf(SOURCE, { + scope: scope({ + source: SOURCE, + generated: Object.freeze([ + Object.freeze({ + marker: "g1", + source: decision === "admitted" ? FRAGMENT : undefined, + decision, + construct: undefined, + }), + ]), + scopes: Object.freeze([child]), + }), + }); + } + + it("shows the fragment inside the author's own enclosure bytes", function* () { + const reading = entryReading({ + entry: owning("admitted", 0), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const text = reading.source.lines.map((one) => one.text); + // The enclosure the author wrote, exactly as they wrote it. + expect(text).toContain(''); + expect(text).toContain(""); + // What it produced, where its producer was. + expect(text).toContain("# generated"); + expect(text).toContain(''); + // And not the producer itself. + expect(text).not.toContain("const x = 1;"); + // Replaced, never appended: the fragment is between the two delimiters. + const open = text.indexOf(''); + const close = text.indexOf(""); + expect(open).toBeLessThan(text.indexOf("# generated")); + expect(text.indexOf("# generated")).toBeLessThan(close); + }); + + it("keeps the fragment's own offsets, so its elements are its own", function* () { + const reading = entryReading({ + entry: owning("admitted", 0), + // The `` is at offset 14 of the *fragment*, which is nowhere + // near offset 14 of the entry's own source. + lifecycle: observing([ + { + name: "Elicit", + position: Object.freeze({ + path: undefined, + generatedSource: "g1", + offset: FRAGMENT.indexOf(" one.text.includes(" one.text); + expect(text).toContain("const x = 1;"); + expect(text).not.toContain("# generated"); + }); + + it("keeps the producer when the recorded position is not this opening", function* () { + const reading = entryReading({ + entry: owning("admitted", 99), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + expect(reading.source.lines.map((one) => one.text)).toContain("const x = 1;"); + }); + + it("keeps the producer when two admissions claim the same opening", function* () { + const base = owning("admitted", 0); + const twice = entryOf(SOURCE, { + scope: scope({ + source: SOURCE, + generated: Object.freeze([ + ...base.scope.generated, + Object.freeze({ + marker: "g2", + source: "# other\n", + decision: "admitted" as const, + construct: undefined, + }), + ]), + scopes: Object.freeze([ + ...base.scope.scopes, + scope({ + key: "entry-1/generated-2", + kind: "generated", + marker: "g2", + source: "# other\n", + position: Object.freeze({ + path: "repl://entry-1", + generatedSource: undefined, + offset: 0, + line: 1, + column: 1, + }), + }), + ]), + }), + }); + const reading = entryReading({ + entry: twice, + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const text = reading.source.lines.map((one) => one.text); + expect(text).toContain("const x = 1;"); + expect(text).not.toContain("# generated"); + expect(text).not.toContain("# other"); + }); + + it("gives a self-closing Evaluate a connected child region underneath", function* () { + const source = '\n'; + const fragment = "read\n"; + const reading = entryReading({ + entry: entryOf(source, { + scope: scope({ + source, + generated: Object.freeze([ + Object.freeze({ + marker: "g1", + source: fragment, + decision: "admitted" as const, + construct: undefined, + }), + ]), + scopes: Object.freeze([ + scope({ + key: "entry-1/generated-1", + kind: "generated", + marker: "g1", + source: fragment, + position: Object.freeze({ + path: "repl://entry-1", + generatedSource: undefined, + offset: 0, + line: 1, + column: 1, + }), + }), + ]), + }), + }), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const text = reading.source.lines.map((one) => one.text); + // Its own row is kept, and the region is under it. + expect(text[0]).toBe(''); + expect(text).toContain("read"); + expect(reading.source.lines.find((one) => one.text === "read")?.depth).toBe(1); + }); + }); + + describe("S2 every logical line is recoverable from the rows it was cut into", () => { + const PROFILES = [48, 72, 100]; + + // A long schema line, an unbroken token no boundary can break, wide and + // combining text, and explicit blank lines. + const SOURCE = [ + '', + " Enter the project details, including the owner and the budget it is held against.", + "", + "", + `x${"x".repeat(159)}`, + "", + "界é🙂 and é combining", + "", + ].join("\n"); + + for (const room of PROFILES) { + it(`recovers every line and its roles at ${room} columns`, () => + withRenderer(function* (renderer) { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const prepared = yield* prepareReading(renderer, WIDE, reading.source.lines, room); + expect(prepared.ok).toBe(true); + if (!prepared.ok) { + return; + } + const joined = recovered(prepared.value); + for (const line of reading.source.lines) { + // Nothing dropped, nothing trimmed, nothing shortened. + expect(joined.get(line.key)).toBe(line.text); + } + // Every row's runs still concatenate to exactly its text, so no + // character lost its role on the way through the cut. + for (const row of prepared.value.rows) { + expect(runText(row.runs)).toBe(row.text); + } + // The unbroken token did get broken — into rows, not into nothing. + const unbroken = reading.source.lines.find((one) => one.text.startsWith("xxx")); + expect(unbroken).toBeDefined(); + const pieces = prepared.value.rows.filter((one) => one.line === unbroken?.key); + expect(pieces.length).toBeGreaterThan(1); + // One owner, and no second badge on any continuation. + expect(pieces.filter((one) => one.continuation).length).toBe(pieces.length - 1); + expect(pieces.filter((one) => one.badge !== undefined).length).toBeLessThanOrEqual(1); + // A blank logical line is one row. + for (const line of reading.source.lines.filter((one) => one.text === "")) { + expect(prepared.value.rows.filter((one) => one.line === line.key).length).toBe(1); + } + })); + } + + it("sets a continuation in to the column its own line was written at", () => + withRenderer(function* (renderer) { + // An indented line, long enough to wrap. Its continuations have to + // start where it started: a continuation further left than the row it + // continues reads as a new, shallower line, which is a claim about + // structure the source does not make. + const indented = + " " + + "an indented line of prose, written long enough that it cannot be held on one row"; + const reading = entryReading({ + entry: entryOf(indented + "\n"), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const prepared = yield* prepareReading(renderer, WIDE, reading.source.lines, 56); + expect(prepared.ok).toBe(true); + if (!prepared.ok) { + return; + } + const rows = prepared.value.rows; + expect(rows.length).toBeGreaterThan(1); + // The first row carries the source's own whitespace in its text; each + // continuation carries the same amount as a display indent instead. + expect(rows[0].indent).toBe(0); + for (const row of rows.slice(1)) { + expect(row.indent).toBe(6); + } + // Each composed row therefore begins its text at the same column. + const columns = rows.map((one) => { + const whole = runText(readingRuns(prepared.value, one)); + return whole.length - whole.trimStart().length; + }); + expect(new Set(columns).size).toBe(1); + // And the line is still recoverable exactly: the indent is display, + // and no source byte moved. + expect(recovered(prepared.value).get(reading.source.lines[0].key)).toBe(indented); + })); + + it("never measures a row wider than the region it was cut for", () => + withRenderer(function* (renderer) { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + const prepared = yield* prepareReading(renderer, WIDE, reading.source.lines, 48); + expect(prepared.ok).toBe(true); + if (!prepared.ok) { + return; + } + for (const row of prepared.value.rows) { + expect(row.width).toBeLessThanOrEqual(prepared.value.reservation.content); + } + })); + + it("cuts wide and combining text at graphemes the engine agrees on", () => + withRenderer(function* (renderer) { + // Four code units, five cells, and two of the three graphemes are not + // one unit wide. + const line: ReplReadingLine = Object.freeze({ + key: "one", + text: "界é🙂", + runs: Object.freeze([Object.freeze({ text: "界é🙂", token: "source" as const })]), + rail: "rail-pending" as const, + depth: 0, + badge: undefined, + style: Object.freeze({ role: "source" as const, selected: false, inspected: false }), + }); + const reservation = yield* reserveFor(renderer, WIDE, 80); + expect(reservation.ok).toBe(true); + if (!reservation.ok) { + return; + } + // A region with room for three cells holds `界` and `é` and not the + // emoji — which a code-unit count would get wrong in both directions. + const room = + reservation.value.rail + reservation.value.separator * 2 + reservation.value.status + 3; + const prepared = yield* prepareReading(renderer, WIDE, [line], room); + expect(prepared.ok).toBe(true); + if (!prepared.ok) { + return; + } + expect(prepared.value.rows.map((one) => one.text)).toEqual(["界é", "🙂"]); + })); + + it("refuses rather than clipping when not one grapheme fits", () => + withRenderer(function* (renderer) { + const line: ReplReadingLine = Object.freeze({ + key: "one", + text: "界界界", + runs: Object.freeze([Object.freeze({ text: "界界界", token: "source" as const })]), + rail: "rail-pending" as const, + depth: 0, + badge: undefined, + style: Object.freeze({ role: "source" as const, selected: false, inspected: false }), + }); + const reservation = yield* reserveFor(renderer, WIDE, 80); + expect(reservation.ok).toBe(true); + if (!reservation.ok) { + return; + } + // One column of content, and the narrowest grapheme is two cells wide. + const room = + reservation.value.rail + reservation.value.separator * 2 + reservation.value.status + 1; + const prepared = yield* prepareReading(renderer, WIDE, [line], room); + expect(prepared.ok).toBe(false); + if (prepared.ok) { + return; + } + expect(prepared.error.name).toBe("ReplFitRefusal"); + })); + }); + + describe("S3 the output half says what actually happened", () => { + it("labels text still arriving as live", function* () { + const reading = entryReading({ + entry: entryOf("hello\n"), + lifecycle: NO_LIFECYCLE, + live: "partial outp", + inspected: false, + }); + expect(reading.output.caption).toBe("Output · live"); + expect(reading.output.lines.map((one) => one.text)).toEqual(["partial outp"]); + }); + + it("replaces it with the retained result, once, unlabelled", function* () { + const reading = entryReading({ + entry: entryOf("hello\n", { terminal: settled("ok", "hello"), settled: true }), + lifecycle: NO_LIFECYCLE, + // Live text from the run that just finished must not be shown beside + // the result that replaced it. + live: "partial outp", + inspected: false, + }); + expect(reading.output.caption).toBe("Output"); + expect(reading.output.lines.map((one) => one.text)).toEqual(["hello"]); + }); + + it("says the outcome when nothing was rendered", function* () { + const reading = entryReading({ + entry: entryOf("", { terminal: settled("ok", ""), settled: true }), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + expect(reading.output.caption).toBe("No rendered output."); + expect(reading.output.lines.map((one) => one.text)).toEqual(["closed ok"]); + expect(reading.output.lines[0].style.role).toBe("successful-outcome"); + }); + + it("reads the outcome from the record, not from the prose", function* () { + // Output containing the word `failed` from a root that closed `ok`. + const reading = entryReading({ + entry: entryOf("doc", { + terminal: settled("ok", "the retry failed and the fallback succeeded"), + settled: true, + }), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + expect(reading.output.caption).toBe("Output"); + expect(reading.output.lines[0].style.role).toBe("output"); + }); + + it("says a prefix recorded no entry output rather than borrowing later text", function* () { + const reading = entryReading({ + entry: entryOf("doc"), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: true, + }); + expect(reading.output.caption).toBe("No entry output recorded at this checkpoint."); + expect(reading.output.lines).toEqual([]); + }); + + it("shows a bound value that emitted nothing as no output", function* () { + const reading = entryReading({ + entry: entryOf('1 + 1', { + terminal: settled("ok", ""), + settled: true, + }), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + expect(reading.output.caption).toBe("No rendered output."); + // No synthesized receipt, and no rendered value invented for it. + expect(reading.output.lines.map((one) => one.text)).toEqual(["closed ok"]); + }); + + it("puts output before source", function* () { + const reading = entryReading({ + entry: entryOf('\n', { terminal: settled("ok", "done"), settled: true }), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + // One reading, two captioned sections, and the result is the first. + expect(Object.keys(reading)).toEqual(["output", "source"]); + expect(reading.output.caption).toBe("Output"); + expect(reading.source.caption).toBe("Source"); + }); + + it("reads nothing for a prefix that admitted no entry", function* () { + expect(NO_READING.output.caption).toBe("No entry output recorded at this checkpoint."); + expect(NO_READING.source.lines).toEqual([]); + }); + }); + + describe("S4 phases are observed, never inferred from syntax", () => { + const SOURCE = '\n Details.\n\n'; + + it("gives static source the rail that means nothing was observed", function* () { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: true, + }); + for (const line of reading.source.lines) { + expect(line.rail).toBe("rail-pending"); + expect(line.badge).toBeUndefined(); + } + expect(REPL_PALETTE.railPending).toBe(RAIL.pend); + }); + + it("gives an observed element its own phase, in the archive's literals", function* () { + const cases = [ + { phase: "enter" as const, rail: RAIL.active, badge: LIFECYCLE.enter }, + { phase: "active" as const, rail: RAIL.active, badge: LIFECYCLE.active }, + { phase: "settled" as const, rail: RAIL.settled, badge: LIFECYCLE.settled }, + { phase: "failed" as const, rail: RAIL.settled, badge: LIFECYCLE.fail }, + ]; + for (const one of cases) { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: observing([{ position: at(0), phase: one.phase }]), + live: "", + inspected: false, + }); + const badged = reading.source.lines.filter((line) => line.badge !== undefined); + expect(badged.length).toBe(1); + expect(runText(badged[0].badge ?? [])).toBe(`${one.badge.glyph} ${one.badge.word}`); + // The rail is the archive's value for that phase, read through the + // product's own role rather than compared to the product's palette. + const rail = reading.source.lines[0].rail; + expect(runStyleOf(rail, reading.source.lines[0].style, false).colour).toBe(one.rail); + // And the badge is the archive's colour. + const ink = runStyleOf((badged[0].badge ?? [])[0].token, badged[0].style, false); + expect(ink.colour).toBe(one.badge.colour); + } + }); + + it("says WAITING on the element that is waiting, with the waiting rail", function* () { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: observing([ + { position: at(0), phase: "active", waiting: Object.freeze(["question" as const]) }, + ]), + live: "", + inspected: false, + }); + const badged = reading.source.lines.filter((line) => line.badge !== undefined); + expect(runText(badged[0].badge ?? [])).toBe(`${LIFECYCLE.hold.glyph} ${LIFECYCLE.hold.word}`); + expect(runStyleOf(reading.source.lines[0].rail, badged[0].style, false).colour).toBe( + RAIL.hold, + ); + }); + + it("keeps EXIT's rail during a cleanup wait, and says both readings", function* () { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: observing([ + { position: at(0), phase: "exit", waiting: Object.freeze(["expansion" as const]) }, + ]), + live: "", + inspected: false, + }); + expect( + runStyleOf(reading.source.lines[0].rail, reading.source.lines[0].style, false).colour, + ).toBe(RAIL.exit); + const badged = reading.source.lines.filter((line) => line.badge !== undefined); + // On the closing delimiter, which is where the exit is happening. + expect(badged.length).toBe(1); + expect(badged[0].text).toBe(""); + expect(runText(badged[0].badge ?? [])).toBe( + `${LIFECYCLE.exit.glyph} ${LIFECYCLE.exit.word} · ` + + `${LIFECYCLE.hold.glyph} ${LIFECYCLE.hold.word}`, + ); + // Two readings, two accents, neither of them the separator's. + const inks = (badged[0].badge ?? []).map( + (run) => runStyleOf(run.token, badged[0].style, false).colour, + ); + expect(inks).toContain(LIFECYCLE.exit.colour); + expect(inks).toContain(LIFECYCLE.hold.colour); + expect(new Set(inks).size).toBe(3); + }); + + it("keeps this terminal's word for a reading that stopped", function* () { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: observing([{ position: at(0), phase: "cancelled" }]), + live: "", + inspected: false, + }); + const badged = reading.source.lines.filter((line) => line.badge !== undefined); + expect(runText(badged[0].badge ?? [])).toBe("cancelled"); + }); + + it("reuses the same cuts and the same status column when only a phase changed", () => + withRenderer(function* (renderer) { + const long = + '\n' + + " Enter the project details, including its owner and the budget it is held against.\n" + + "\n"; + const of = (phase: "enter" | "settled") => + entryReading({ + entry: entryOf(long), + lifecycle: observing([{ position: at(0), phase }]), + live: "", + inspected: false, + }); + const first = yield* prepareReading(renderer, WIDE, of("enter").source.lines, 56); + const second = yield* prepareReading(renderer, WIDE, of("settled").source.lines, 56); + expect(first.ok && second.ok).toBe(true); + if (!first.ok || !second.ok) { + return; + } + // The same source, cut the same way. + expect(second.value.rows.map((one) => one.text)).toEqual( + first.value.rows.map((one) => one.text), + ); + // The same reservation, so the column does not move. + expect(second.value.reservation.status).toBe(first.value.reservation.status); + expect(rightAligned(first.value)).toEqual([true]); + expect(rightAligned(second.value)).toEqual([true]); + })); + + it("reserves for every reading a badge can hold, measured not counted", () => + withRenderer(function* (renderer) { + const reservation = yield* reserveFor(renderer, WIDE, 80); + expect(reservation.ok).toBe(true); + if (!reservation.ok) { + return; + } + // Every phase, the cleanup wait that says two things, and the word this + // terminal keeps for a cancellation. + for (const one of READING_STATUSES) { + expect(reservation.value.statusWidths.get(one)).toBeDefined(); + expect(reservation.value.statusWidths.get(one) ?? 0).toBeLessThanOrEqual( + reservation.value.status, + ); + } + expect(READING_STATUSES).toContain("cancelled"); + expect(READING_STATUSES).toContain( + `${LIFECYCLE.exit.glyph} ${LIFECYCLE.exit.word} · ` + + `${LIFECYCLE.hold.glyph} ${LIFECYCLE.hold.word}`, + ); + // The widest is the cleanup wait, which is the point of reserving for + // it: the column cannot grow when an element starts releasing. + expect(reservation.value.status).toBe( + reservation.value.statusWidths.get( + `${LIFECYCLE.exit.glyph} ${LIFECYCLE.exit.word} · ` + + `${LIFECYCLE.hold.glyph} ${LIFECYCLE.hold.word}`, + ), + ); + })); + + it("attributes nothing to an entry whose observations belong to another", function* () { + const reading = entryReading({ + entry: entryOf(SOURCE), + lifecycle: Object.freeze({ + entry: "entry-2", + occurrences: observing([{ position: at(0), phase: "active" }]).occurrences, + }), + live: "", + inspected: false, + }); + for (const line of reading.source.lines) { + expect(line.badge).toBeUndefined(); + expect(line.rail).toBe("rail-pending"); + } + }); + + it("gives each call of one element its own reading, in its own source group", function* () { + // Two calls of one authored element — what a `` produces. Showing + // only the last would say the first never happened, and one badge for both + // would say two calls were one. + const source = '\n'; + const reading = entryReading({ + entry: entryOf(source), + lifecycle: observing([ + { key: "entry-1#1", name: "Step", position: at(0), phase: "settled" }, + { key: "entry-1#2", name: "Step", position: at(0), phase: "active" }, + ]), + live: "", + inspected: false, + }); + const rows = reading.source.lines.filter((one) => one.text.includes(" runText(one.badge ?? []))).toEqual([ + `${LIFECYCLE.settled.glyph} ${LIFECYCLE.settled.word}`, + `${LIFECYCLE.active.glyph} ${LIFECYCLE.active.word}`, + ]); + // And each is its own group: two rows, two keys, two rails. + expect(new Set(rows.map((one) => one.key)).size).toBe(2); + expect(rows.map((one) => one.rail)).toEqual(["rail-settled", "rail-active"]); + }); + + it("reads an element nothing was observed of exactly once", function* () { + const source = '\n'; + const reading = entryReading({ + entry: entryOf(source), + lifecycle: NO_LIFECYCLE, + live: "", + inspected: false, + }); + expect(reading.source.lines.filter((one) => one.text.includes(" + withRenderer(function* (renderer) { + // The gap K1 found. Every other case here reads `line.rail`, which is + // the model's answer; none of them asked whether a rail reaches the + // row. Removing it from the composed runs broke nothing, so a reading + // could have lost its rails entirely and every test would have passed. + const source = '\n Ask.\n\n'; + const reading = entryReading({ + entry: entryOf(source), + lifecycle: observing([{ position: at(0), phase: "active" }]), + live: "", + inspected: false, + }); + const prepared = yield* prepareReading(renderer, WIDE, reading.source.lines, 72); + expect(prepared.ok).toBe(true); + if (!prepared.ok) { + return; + } + expect(prepared.value.rows.length).toBeGreaterThan(0); + for (const row of prepared.value.rows) { + const runs = readingRuns(prepared.value, row); + // It is the first thing in the row, it is the rail glyph, and it is + // drawn under the rail role that says which region the row is in. + expect([row.key, runs[0].text]).toEqual([row.key, "\u2502"]); + expect([row.key, runs[0].token]).toEqual([row.key, row.rail]); + expect([row.key, runText(runs).startsWith("\u2502")]).toEqual([row.key, true]); + } + // And the observed region's rail really is the observed one, so this is + // not satisfied by painting every row the same. + const observed = prepared.value.rows.filter((one) => one.rail === "rail-active"); + expect(observed.length).toBeGreaterThan(0); + })); + + it("aligns every badge at the same measured column", () => + withRenderer(function* (renderer) { + const source = '\n\n'; + const reading = entryReading({ + entry: entryOf(source), + lifecycle: observing([ + { position: at(0), phase: "active" }, + { position: at(source.indexOf(' one.badge !== undefined) + .map((one) => runText(readingRuns(prepared.value, one)).length); + expect(new Set(composed).size).toBe(1); + })); + }); +}); diff --git a/packages/cli/tests/repl-terminal.test.ts b/packages/cli/tests/repl-terminal.test.ts index cddec2628..3d103171b 100644 --- a/packages/cli/tests/repl-terminal.test.ts +++ b/packages/cli/tests/repl-terminal.test.ts @@ -278,9 +278,9 @@ describe("REPL terminal: responsive semantic frames", () => { // sit outside the content they move. expect(textsIn(frame, "drawer")).toEqual([ "Binding: plan", - "[^ earlier]", + "[↑ earlier]", "Close", - "[v later]", + "[↓ later]", "[close]", ]); // One surface, the routed one, and nothing from the others. diff --git a/packages/core/api.ts b/packages/core/api.ts index 5d7a0740b..fee42b6e1 100644 --- a/packages/core/api.ts +++ b/packages/core/api.ts @@ -112,3 +112,19 @@ export type { Structural, } from "./src/execution-declarations.ts"; export type { JournalAdmission } from "./src/execute.ts"; + +/** + * The contextual Api a consumer installs middleware on, and the types one + * `Component.expand` handler is written in. + * + * Exported here rather than from the root because this is the consumer-facing + * boundary: a host that wants to watch an element expand needs the descriptor + * and the shapes it will be handed, and nothing of the engine behind them. + */ +export { Component } from "./src/component-api.ts"; +export type { + ComponentApi, + ComponentExpansionPhase, + ComponentExpansionRequest, +} from "./src/component-api.ts"; +export type { Expansion } from "./src/expansion.ts"; diff --git a/packages/core/mod.ts b/packages/core/mod.ts index b08e21491..be32a01a2 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -68,6 +68,8 @@ export { handleFailure, } from "./src/component-api.ts"; +export { inspectSource } from "./src/source-inspection.ts"; +export type { SourceElement, SourceRange } from "./src/source-inspection.ts"; export { renderSegments } from "./src/render.ts"; export { createReplayStream } from "./src/replay-stream.ts"; diff --git a/packages/core/src/component-api.ts b/packages/core/src/component-api.ts index f65dd7f95..d8fbdbc61 100644 --- a/packages/core/src/component-api.ts +++ b/packages/core/src/component-api.ts @@ -14,10 +14,12 @@ */ import { type Api, createApi, type Operations } from "@effectionx/context-api"; -import type { Operation } from "effection"; +import type { Operation, Result, Stream } from "effection"; import type { EvalScope } from "@effectionx/scope-eval"; import { settle } from "./errors.ts"; +import { ComponentExpansionProtocolError } from "./component-expansion.ts"; import type { BoundExecRequest } from "./bound-exec.ts"; +import type { Expansion } from "./expansion.ts"; import type { CodeBlockContext, CodeBlockResult, @@ -32,7 +34,64 @@ import type { SourcePosition, } from "./types.ts"; +/** + * Where one executable element has reached, as its observers are told. + * + * `enter` before anything is resolved, `active` where accepted work starts, + * `exit` where that work ends and its owned cleanup begins, and exactly one + * terminal observation — `complete` once canonical acceptance has reconciled + * the outcome, or `cancelled` where the element unwound cleanly instead. + * + * `complete` carries `Result` and nothing else: what the element + * produced, what it bound and which error identity canonical execution is + * holding stay private, so an observation says what happened and decides + * nothing about it. + */ +export type ComponentExpansionPhase = + | { readonly phase: "enter" } + | { readonly phase: "active" } + | { readonly phase: "exit"; readonly reason: "returned" | "failed" | "cancelled" } + | { readonly phase: "complete"; readonly result: Result } + | { readonly phase: "cancelled" }; + +/** + * One executable element's complete expansion, offered to whoever is watching. + * + * Issued by canonical execution before resolution, props validation or any + * body runs. It is an offer to surround the work, not a way to do it: the + * request is branded, it delegates once, and nothing on it reaches what the + * element produced. + */ +export interface ComponentExpansionRequest { + /** + * Core's detached description of the element this surrounds. + * + * Presentation information only. It names a logical source element, not a + * process call or a capability, and grants nothing. + */ + readonly expansion: Expansion; + /** + * The latest phase, then each change in order, per subscription. + * + * Independent and scope-owned: every subscriber gets its own, ending after + * the one terminal observation. Reading or cancelling a subscription neither + * starts nor cancels the element, and a reader that is slow or absent delays + * nothing. + */ + readonly phases: Stream; +} + export interface ComponentApi { + /** + * Surround one executable element's complete expansion (spec §5.7). + * + * Core issues the request before resolution or validation; delegate the + * exact request once with `next(request)`. Whatever a handler returns is + * ignored, catching a canonical failure does not rescue it, and completion + * includes owned teardown and final result acceptance. A handler may refuse + * the work by throwing or by returning without delegating. + */ + expand(request: ComponentExpansionRequest): Operation; /** * `"__root__"` imports the root document. * @@ -255,6 +314,21 @@ export function isMissingImportProvider(error: unknown, asked: string): boolean * process, and the lint rule that says so is right. */ const COMPONENT_DEFAULTS: Omit = { + /** + * Nothing issued this, so there is nothing to surround. + * + * The public descriptor's own terminal is reached only by a caller that + * built a request itself or kept one from an expansion that has gone. Either + * way no canonical work is behind it, and running a body for it is the one + * thing this seam must never do. + */ + // deno-lint-ignore require-yield + *expand(_request: ComponentExpansionRequest): Operation { + throw new Error( + "Component.expand() has no canonical expansion to surround: only core issues a request, " + + "and only the expansion that issued one can run it.", + ); + }, // deno-lint-ignore require-yield *importComponent( name: string, @@ -381,6 +455,43 @@ export function importThroughTerminal( return owned.operations.importComponent(name, position); } +/** + * Surround one expansion through a descriptor whose terminal belongs to it. + * + * The same correlation `importThroughTerminal` makes, for the same reason: the + * name shares every installed handler, and the default each `createApi()` + * instance owns is this one element's continuation. Two elements expanding at + * once have two descriptors and two terminals, and a counterfeit descriptor + * built with the same name changes only the calls made through it. + */ +export function expandThroughTerminal( + request: ComponentExpansionRequest, + terminal: (delegated: ComponentExpansionRequest) => Operation, +): Operation { + const owned = createApi("Component", { + ...COMPONENT_DEFAULTS, + registry: new Map(), + *expand(delegated: ComponentExpansionRequest): Operation { + // This terminal is one element's continuation, so the only request it + // may carry is the one that element issued. An authentic request from + // *another* live invocation would otherwise reach the claim below and + // consume somebody else's work while running this body — refusing after + // the body has run is too late, because the effect has happened. + // + // Identity, not shape: a request that merely looks like this one was + // issued for different work. + if (delegated !== request) { + throw new ComponentExpansionProtocolError( + "delegated an expansion this invocation did not issue", + ); + } + yield* terminal(delegated); + }, + }); + return owned.operations.expand(request); +} + +export const expand: Operations["expand"] = Component.operations.expand; export const importComponent: Operations["importComponent"] = Component.operations.importComponent; export const applyModifiers: Operations["applyModifiers"] = diff --git a/packages/core/src/component-expansion.ts b/packages/core/src/component-expansion.ts new file mode 100644 index 000000000..c5ab0b874 --- /dev/null +++ b/packages/core/src/component-expansion.ts @@ -0,0 +1,438 @@ +/** + * Canonical expansion observation (spec §5.7). + * + * One executable element, surrounded from before its resolution through its + * owned cleanup and the acceptance of what it produced. Core issues a request, + * the public `Component.expand` chain composes around it, and exactly one + * delegation of that exact request runs the work. + * + * Observation only. What middleware returns is ignored, what it catches it does + * not rescue, and the outgoing result stays private until canonical acceptance + * has reconciled it — so an observer learns what happened and decides nothing + * about it. The authority is the same one `bound-exec.ts` keeps: a private + * brand nothing outside can forge, a claim that can be spent once, and a + * settlement readable whether or not the chain returned normally. + */ + +import { createQueue, ensure, Err, Ok } from "effection"; +import type { Operation, Queue, Stream } from "effection"; +import { expandThroughTerminal } from "./component-api.ts"; +import type { ComponentExpansionPhase, ComponentExpansionRequest } from "./component-api.ts"; +import type { Expansion } from "./expansion.ts"; +import { durabilityFailure, filesFatalFailure } from "./errors.ts"; + +/** What an observer did that canonical expansion cannot act on. */ +export class ComponentExpansionProtocolError extends Error { + override name = "ComponentExpansionProtocolError"; + constructor(problem: string) { + super( + `Component.expand middleware ${problem}. A handler may observe, refuse or delegate an ` + + "expansion; only canonical execution runs one.", + ); + } +} + +/** + * The latest phase, then what happens next, per subscriber. + * + * Latest rather than every earlier phase: an observer that arrives while an + * element is already active wants to know that it is active, not to be walked + * through a history it missed. One queue per subscription, filled + * synchronously, so a reader that is slow or never reads at all delays nothing — + * execution never waits for an observer, and cancelling a subscription neither + * starts nor stops the work it was watching. + */ +interface PhasePublication { + readonly phases: Stream; + /** Publish one non-terminal phase. */ + publish(phase: ComponentExpansionPhase): void; + /** Publish the one terminal phase and close every subscription with void. */ + finish(phase: ComponentExpansionPhase): void; +} + +function createPhasePublication(): PhasePublication { + const subscribers = new Set>(); + let latest: ComponentExpansionPhase | undefined; + let closed = false; + + const deliver = (phase: ComponentExpansionPhase): void => { + // Frozen before anybody can hold it. One object reaches every subscriber + // and is kept as `latest`, so a reader that changed a field would change + // what the other readers — and every reader registering afterwards — are + // told this element did. An observation is a reading, and a reading + // somebody can edit is not one. + const published = Object.freeze(phase); + latest = published; + for (const queue of subscribers) { + queue.add(published); + } + }; + + return { + phases: { + *[Symbol.iterator]() { + const queue = createQueue(); + // Registered with its latest observation in one step, so nothing can + // land between reading the latest phase and being told the next one. + if (latest !== undefined) { + queue.add(latest); + } + if (closed) { + queue.close(); + return queue; + } + subscribers.add(queue); + yield* ensure(() => { + subscribers.delete(queue); + }); + return queue; + }, + }, + publish(phase: ComponentExpansionPhase): void { + if (!closed) { + deliver(phase); + } + }, + finish(phase: ComponentExpansionPhase): void { + if (closed) { + return; + } + deliver(phase); + closed = true; + for (const queue of subscribers) { + queue.close(); + } + subscribers.clear(); + }, + }; +} + +/** What canonical expansion kept about one element while it ran. */ +interface ExpansionWork { + consumed: boolean; + /** The canonical failure, kept whether or not a handler caught it. */ + failure?: { readonly raised: unknown }; + /** Whether the delegated work ran to its own completion. */ + ran: boolean; +} + +/** + * One element's request, branded so nothing outside this module can build one. + * + * The brand is what makes delegation exact: a copy carrying the same members is + * not this request, and a handler holding one from a sibling expansion cannot + * spend it here. + */ +class CanonicalExpansionRequest implements ComponentExpansionRequest { + readonly #work: ExpansionWork; + readonly expansion: Expansion; + readonly phases: Stream; + + constructor(work: ExpansionWork, expansion: Expansion, publication: PhasePublication) { + this.#work = work; + this.expansion = expansion; + this.phases = publication.phases; + Object.freeze(this); + } + + /** The work `request` speaks for, once, or the refusal saying why it speaks for none. */ + static claim(request: unknown): ExpansionWork { + if (!CanonicalExpansionRequest.own(request)) { + throw new ComponentExpansionProtocolError( + "delegated a request canonical execution did not issue", + ); + } + const work = request.#work; + if (work.consumed) { + throw new ComponentExpansionProtocolError("delegated one expansion more than once"); + } + work.consumed = true; + return work; + } + + /** Whether this class built `value`, answered without trusting it. */ + static own(value: unknown): value is CanonicalExpansionRequest { + if (typeof value !== "object" || value === null) { + return false; + } + try { + return #work in value; + } catch { + return false; + } + } +} + +/** + * What one element's observers are told, handed to the paths that run its body. + * + * Publication and nothing else: a path that takes it can say the work started + * and that it ended, and cannot read an outcome, decide one or reach the + * request it belongs to. + */ +export interface ObservedExpansion { + /** Accepted work is starting. */ + active(): void; + /** The body's own work ended, and why, before its owned cleanup. */ + settled(reason: "returned" | "failed" | "cancelled"): void; +} + +/** What canonical expansion produced for one element. */ +export type ExpansionSettlement = + /** The delegated work ran and settled on its own terms. */ + | { readonly status: "ran" } + /** Canonical expansion raised this exact failure. Not middleware's to rescue. */ + | { readonly status: "raised"; readonly raised: unknown } + /** The terminal was never reached, so no work ran. */ + | { readonly status: "absent"; readonly refusal: ComponentExpansionProtocolError }; + +/** One element's issued request, and what canonical core reads back from it. */ +export interface IssuedExpansion { + readonly request: ComponentExpansionRequest; + /** Publish `enter`, `active` and `exit` as the work reaches them. */ + publish(phase: ComponentExpansionPhase): void; + /** + * Publish the one terminal phase and close every subscription. + * + * Called after the whole dispatch has unwound and canonical acceptance is + * reconciled, which is why it is not the terminal's own business: middleware + * cleanup runs outside the terminal, and a completion published before it + * would say the element had finished while its observers were still running. + */ + finish(phase: ComponentExpansionPhase): void; + /** + * What canonical expansion settled to. + * + * Readable whether or not the public chain returned normally, so a handler + * that catches what canonical expansion raised cannot turn it into silence. + */ + settlement(): ExpansionSettlement; +} + +/** Issue one element's expansion, retaining what canonical core reads back. */ +export function issueComponentExpansion(expansion: Expansion): IssuedExpansion { + const work: ExpansionWork = { consumed: false, ran: false }; + const publication = createPhasePublication(); + return { + request: new CanonicalExpansionRequest(work, expansion, publication), + publish(phase: ComponentExpansionPhase): void { + publication.publish(phase); + }, + finish(phase: ComponentExpansionPhase): void { + publication.finish(phase); + }, + settlement(): ExpansionSettlement { + const failure = work.failure; + if (failure !== undefined) { + return { status: "raised", raised: failure.raised }; + } + if (!work.consumed) { + return { + status: "absent", + refusal: new ComponentExpansionProtocolError("returned without delegating the expansion"), + }; + } + if (!work.ran) { + return { + status: "raised", + raised: new ComponentExpansionProtocolError("returned before the expansion settled"), + }; + } + return { status: "ran" }; + }, + }; +} + +/** + * Run the canonical work for one issued request, once. + * + * Canonical core's terminal calls this with whatever middleware delegated, so a + * rewritten, copied or fabricated request runs nothing rather than running + * something else. `run` is canonical core's own continuation: it is reached only + * from here, so a handler that does not delegate never reaches it. + */ +export function* claimComponentExpansion( + request: unknown, + run: () => Operation, +): Operation { + const work = CanonicalExpansionRequest.claim(request); + try { + yield* run(); + } catch (error) { + // Kept whether or not a handler catches what propagates: a failure + // canonical expansion raised is not middleware's to rescue. + work.failure = { raised: error }; + throw error; + } + work.ran = true; +} + +/** + * One failure as an observation: what it is called and what it said, detached. + * + * A new Error every time, never the one canonical execution is holding. An + * observer learns the selected name and message, and the explanatory Errors + * behind them, and reaches no canonical identity, no resource an error happens + * to carry and nothing `instanceof` would recognize — so catching an + * observation cannot be mistaken for catching the failure, and a reporter that + * serializes one carries away no live object. + * + * Depth is bounded because a cause chain is somebody else's data structure and + * may be cyclic. + */ +export function reported(raised: unknown, depth = 0): Error { + if (!(raised instanceof Error)) { + return Object.freeze(new Error(String(raised))); + } + const explained = depth >= REPORTED_DEPTH ? undefined : explanatory(raised, depth); + const detached = + raised instanceof AggregateError + ? new AggregateError(explained?.members ?? [], raised.message) + : new Error(raised.message); + detached.name = raised.name; + if (explained?.cause !== undefined) { + detached.cause = explained.cause; + } + // Freezing an AggregateError leaves its `errors` array writable, so one + // reader could rewrite the explanation every other reader is holding. The + // members are already detached reports; this is what stops them being + // edited in place. + if (detached instanceof AggregateError) { + Object.freeze(detached.errors); + } + return Object.freeze(detached); +} + +/** How far a detached report follows somebody else's cause chain. */ +const REPORTED_DEPTH = 8; + +function explanatory( + raised: Error, + depth: number, +): { readonly cause: Error | undefined; readonly members: Error[] } { + return { + cause: raised.cause instanceof Error ? reported(raised.cause, depth + 1) : undefined, + members: + raised instanceof AggregateError && Array.isArray(raised.errors) + ? raised.errors.map((member) => reported(member, depth + 1)) + : [], + }; +} + +/** + * Which failure a competing canonical and middleware outcome settles on. + * + * The same order expansion already reconciles bound failures in: durability + * first, then a Files fatal, then the canonical failure over a later ordinary + * one. Stated here rather than shared with `expand.ts`, because the two reach + * it with different shapes and a helper common to both would be a third + * boundary to keep in step. + * + * Within each fatal kind the canonical failure is preferred, so the earlier + * one wins. Middleware still cannot rescue a canonical failure: an ordinary + * middleware failure never displaces it. + */ +function selectedFailure(canonical: unknown, laterFailed: boolean, later: unknown): unknown { + const durable = + durabilityFailure(canonical) ?? (laterFailed ? durabilityFailure(later) : undefined); + if (durable !== undefined) { + return durable; + } + const files = + filesFatalFailure(canonical) ?? (laterFailed ? filesFatalFailure(later) : undefined); + if (files !== undefined) { + return files; + } + return canonical; +} + +/** + * Surround one element's work with its canonical observation. + * + * Every path that expands something an author wrote goes through here, so the + * sequence a reader sees, the authority a handler has and the reconciliation + * between the two are decided once rather than per call site. + * + * What the work produces is returned untouched. The observation says what + * happened and nothing more: it cannot read the value, and a handler cannot + * replace it, rescue a failure or complete the element early. + */ + +export function* observeExpansion( + expansion: Expansion, + work: (observed: ObservedExpansion) => Operation, +): Operation { + const issued = issueComponentExpansion(expansion); + let produced: { readonly value: T } | undefined; + let terminated = false; + const terminate = (phase: ComponentExpansionPhase): void => { + if (!terminated) { + terminated = true; + issued.finish(phase); + } + }; + issued.publish({ phase: "enter" }); + // Said once however many body paths an element has: a second ACTIVE would + // be the same observation twice, and a reader counting changes would see a + // step that did not happen. + let active = false; + const observed: ObservedExpansion = { + active: () => { + if (!active) { + active = true; + issued.publish({ phase: "active" }); + } + }, + settled: (reason) => issued.publish({ phase: "exit", reason }), + }; + + let chainFailure: unknown; + let chainFailed = false; + try { + try { + yield* expandThroughTerminal(issued.request, (delegated) => + claimComponentExpansion(delegated, function* () { + produced = { value: yield* work(observed) }; + }), + ); + } catch (error) { + chainFailure = error; + chainFailed = true; + } + // Reconciled after the whole dispatch has unwound, middleware cleanup + // included. A handler may refuse the work or fail a success; what it + // cannot do is rescue what canonical expansion raised, which is read back + // from the terminal rather than from whether the chain returned normally. + const settlement = issued.settlement(); + if (settlement.status === "raised") { + // Both halves can have failed: the body raised, and middleware's own + // unwind raised something else on the way out. A fatal failure is fatal + // whichever half it came from — an ordinary body error that hid a + // Journal failure would leave enclosing reconciliation believing the + // run is merely incorrect rather than unable to persist. + const selected = selectedFailure(settlement.raised, chainFailed, chainFailure); + terminate({ phase: "complete", result: Err(reported(selected)) }); + throw selected; + } + if (settlement.status === "absent") { + const refusal = chainFailed ? chainFailure : settlement.refusal; + terminate({ phase: "complete", result: Err(reported(refusal)) }); + throw refusal; + } + if (chainFailed) { + terminate({ phase: "complete", result: Err(reported(chainFailure)) }); + throw chainFailure; + } + if (produced === undefined) { + const missing = new ComponentExpansionProtocolError("returned before the expansion settled"); + terminate({ phase: "complete", result: Err(reported(missing)) }); + throw missing; + } + terminate({ phase: "complete", result: Ok(undefined) }); + return produced.value; + } finally { + // Nothing terminal was published, so this element unwound without + // completing: a clean cancellation, told after it has finished unwinding. + terminate({ phase: "cancelled" }); + } +} diff --git a/packages/core/src/definition.ts b/packages/core/src/definition.ts index cef192914..945f0331f 100644 --- a/packages/core/src/definition.ts +++ b/packages/core/src/definition.ts @@ -9,6 +9,7 @@ import { findTarget, outlineDocument, retainedRanges, selectTarget } from "./doc import type { DocumentOutline, DocumentTargetInfo } from "./document-targets.ts"; import matter from "gray-matter"; +import { envelopeBody } from "./source-inspection.ts"; /** * Whether a path names a function component. Execution imports `.ts` @@ -42,6 +43,16 @@ export function parseSource(path: string, content: string): ParsedSource { if (content.slice(baseOffset) !== parsed.content) { throw new Error(`frontmatter parse did not preserve the markdown body verbatim: ${path}`); } + // Inspection finds this same boundary without asking the value parser for + // anything, so that reading a document's shape never interprets its header. + // One boundary, asserted here: a divergence is loud rather than a quiet + // offset drift between what executes and what is shown. + if (envelopeBody(content) !== baseOffset) { + throw new Error( + `frontmatter boundary disagrees with the lexical envelope (${envelopeBody(content)} vs ` + + `${baseOffset}): ${path}`, + ); + } let baseLine = 1; for (let i = 0; i < baseOffset; i++) { if (content[i] === "\n") { diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 1ba20ed2c..29307e0d9 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -124,6 +124,8 @@ import type { IdentityDomain } from "./invocation-identity.ts"; import { protectedContentLease } from "./protected-content.ts"; import type { SyntaxReference } from "./syntax-reference.ts"; import { withInvocation } from "./invocation.ts"; +import { observeExpansion } from "./component-expansion.ts"; +import type { ObservedExpansion } from "./component-expansion.ts"; import type { Invocation } from "./invocation.ts"; import { ActiveProjection } from "./projection.ts"; import type { ProjectionHandle, ProjectionRequest } from "./projection.ts"; @@ -1797,93 +1799,97 @@ function* expandEach( imports: ComponentImportTerminal | undefined, returnBody: ReturnBody | undefined, ): Operation { - // Decided from source alone, so the catalog is shared with validation. A - // literal `in` is checked here too; an expression is a value the document - // computes, and its answer is checked below where it arrives. - const violations = eachViolations(segment); - const refusal = violations[0]; - if (refusal !== undefined) { - return [yield* raise(eachError(refusal.message))]; - } - const name = eachItemBinding(segment)!; - const asBinding = eachCaptureBinding(segment); + return yield* structural(segment, path, () => work()); + + function* work(): Operation { + // Decided from source alone, so the catalog is shared with validation. A + // literal `in` is checked here too; an expression is a value the document + // computes, and its answer is checked below where it arrives. + const violations = eachViolations(segment); + const refusal = violations[0]; + if (refusal !== undefined) { + return [yield* raise(eachError(refusal.message))]; + } + const name = eachItemBinding(segment)!; + const asBinding = eachCaptureBinding(segment); - let items: Json | undefined; - if ("in" in segment.props) { - items = segment.props.in; - } else if ("in" in segment.expressions) { - try { - const resolved = yield* resolveExpressionProps( - {}, - { in: segment.expressions.in }, - "Each", - segment.projectedEnv, - ); - items = resolved.in; - } catch (error) { - return [yield* raise(eachError(error instanceof Error ? error.message : String(error)))]; + let items: Json | undefined; + if ("in" in segment.props) { + items = segment.props.in; + } else if ("in" in segment.expressions) { + try { + const resolved = yield* resolveExpressionProps( + {}, + { in: segment.expressions.in }, + "Each", + segment.projectedEnv, + ); + items = resolved.in; + } catch (error) { + return [yield* raise(eachError(error instanceof Error ? error.message : String(error)))]; + } + } + if (!Array.isArray(items)) { + return [yield* raise(eachError(eachItemsViolation(items)!.message))]; } - } - if (!Array.isArray(items)) { - return [yield* raise(eachError(eachItemsViolation(items)!.message))]; - } - // Effective caller env honors projection through , mirroring - // expandComponent, so a projected resolves both lexical caller - // bindings and the current component's bindings. - const contextEnv = yield* env; - const callerEnv = layerEnvironments(segment.projectedEnv, contextEnv); - const parentEvalScope = yield* evalScope; - - const enclosingLoop = yield* ActiveLoop.get(); - // A rendering iteration writes into the caller's region as it goes, so a - // failure partway leaves the items it already produced behind. A captured one - // builds a value instead: its buffer is private and never becomes output. - const out: Segment[] = asBinding === undefined ? owner : []; - for (const [iteration, item] of items.entries()) { - yield* expandChildrenScoped( - segment.children, - callerEnv ?? undefined, - { [name]: item }, - parentEvalScope ?? undefined, - parentMeta, - parentProps, - hideSet, - counter, - out, - extendPath(path, { f: "item", i: iteration }), - checkedFailures, - environment, - imports, - returnBody, - ); - // A `` in the body exits the enclosing ``, so the remaining - // items are part of the work that iteration no longer does. - if (enclosingLoop?.broken) { - break; + // Effective caller env honors projection through , mirroring + // expandComponent, so a projected resolves both lexical caller + // bindings and the current component's bindings. + const contextEnv = yield* env; + const callerEnv = layerEnvironments(segment.projectedEnv, contextEnv); + const parentEvalScope = yield* evalScope; + + const enclosingLoop = yield* ActiveLoop.get(); + // A rendering iteration writes into the caller's region as it goes, so a + // failure partway leaves the items it already produced behind. A captured one + // builds a value instead: its buffer is private and never becomes output. + const out: Segment[] = asBinding === undefined ? owner : []; + for (const [iteration, item] of items.entries()) { + yield* expandChildrenScoped( + segment.children, + callerEnv ?? undefined, + { [name]: item }, + parentEvalScope ?? undefined, + parentMeta, + parentProps, + hideSet, + counter, + out, + extendPath(path, { f: "item", i: iteration }), + checkedFailures, + environment, + imports, + returnBody, + ); + // A `` in the body exits the enclosing ``, so the remaining + // items are part of the work that iteration no longer does. + if (enclosingLoop?.broken) { + break; + } } - } - if (asBinding === undefined) { - return []; - } + if (asBinding === undefined) { + return []; + } - // A capture never swallows an error. The body reported these where they were - // created (§6.9), so they are returned as they are: the printed errors reach the - // document unchanged and the binding stays unset. - const errors = out.filter((outSegment) => outSegment.type === "error"); - if (errors.length > 0) { - return errors; - } + // A capture never swallows an error. The body reported these where they were + // created (§6.9), so they are returned as they are: the printed errors reach the + // document unchanged and the binding stays unset. + const errors = out.filter((outSegment) => outSegment.type === "error"); + if (errors.length > 0) { + return errors; + } - const captureEnv = yield* env; - if (!captureEnv) { - return [ - yield* raise(eachError('Prop "as" on requires a parent evaluation environment.')), - ]; + const captureEnv = yield* env; + if (!captureEnv) { + return [ + yield* raise(eachError('Prop "as" on requires a parent evaluation environment.')), + ]; + } + captureEnv.values[asBinding] = renderSegments(out); + return []; } - captureEnv.values[asBinding] = renderSegments(out); - return []; } /** @@ -1938,6 +1944,37 @@ function ifError(segment: ComponentElement, message: string): ErrorSegment { * handed back untouched, so a `` inside a selected branch settles * once, exactly as it would inline. */ +/** + * Surround one structural element's own work with its canonical observation. + * + * Structural syntax has no separate body to enter: the work it consumes starts + * where the element does, so ACTIVE is published as the region begins and EXIT + * when it ends. Written once here, so a construct that recurses and one that + * selects a branch are observed the same way. + */ +function structural( + segment: { readonly name: string; readonly position?: SourcePosition }, + path: string, + work: () => Operation, +): Operation { + return observeExpansion(snapshot(path, segment.name, segment.position), function* (observed) { + observed.active(); + // Cancellation unless something says otherwise: the `finally` runs on an + // unwind the work never returned from as well as on the two it did. + let reason: "returned" | "failed" | "cancelled" = "cancelled"; + try { + const value = yield* work(); + reason = "returned"; + return value; + } catch (error) { + reason = "failed"; + throw error; + } finally { + observed.settled(reason); + } + }); +} + function* expandIf( segment: ComponentElement, parentMeta: Record, @@ -1953,78 +1990,91 @@ function* expandIf( imports: ComponentImportTerminal | undefined, returnBody: ReturnBody | undefined, ): Operation { - // Decided from source alone and shared with validation: which props were - // written, and how the body splits at its ``. - const unknownProp = ifPropsViolation(segment); - if (unknownProp !== undefined) { - owner.push(yield* raise(ifError(segment, unknownProp.message))); - return; - } - - const structure = ifStructure(segment); - if (structure.violations.length > 0) { - for (const violation of structure.violations) { - owner.push(yield* raise(structuralErrorSegment(violation, segment))); + return yield* structural(segment, path, () => work()); + + function* work(): Operation { + // Decided from source alone and shared with validation: which props were + // written, and how the body splits at its ``. + const unknownProp = ifPropsViolation(segment); + if (unknownProp !== undefined) { + owner.push(yield* raise(ifError(segment, unknownProp.message))); + return; } - return; - } - let condition: unknown; - if ("condition" in segment.props) { - condition = segment.props.condition; - } else if ("condition" in segment.expressions) { - try { - // Evaluated directly rather than through resolveExpressionProps: that - // helper normalizes its result through JSON, which rejects `undefined`, - // rewrites `NaN` as `null`, and throws on a BigInt. A condition is - // decided and discarded rather than passed on or recorded, so it takes - // any JavaScript value and crosses no serialization boundary. - condition = yield* evaluateExpression( - segment.expressions.condition, - "If", - "condition", - segment.projectedEnv, - ); - } catch (error) { - owner.push( - yield* raise(ifError(segment, error instanceof Error ? error.message : String(error))), - ); + const structure = ifStructure(segment); + if (structure.violations.length > 0) { + for (const violation of structure.violations) { + owner.push(yield* raise(structuralErrorSegment(violation, segment))); + } return; } - } else { - owner.push(yield* raise(ifError(segment, ifConditionViolation(segment)!.message))); - return; - } - const selected = !!condition; - - // The false arm belongs to ``, which is consumed above, so its frame is - // added here — otherwise both arms of one `` expand under one path. - const branchPath = - selected || structure.elseElement === undefined - ? path - : extendPath( - path, - elementFrame( - structure.elseElement.name, - elementSite(structure.elseElement.position, structure.elseIndex ?? 0), - ), + let condition: unknown; + if ("condition" in segment.props) { + condition = segment.props.condition; + } else if ("condition" in segment.expressions) { + try { + // Evaluated directly rather than through resolveExpressionProps: that + // helper normalizes its result through JSON, which rejects `undefined`, + // rewrites `NaN` as `null`, and throws on a BigInt. A condition is + // decided and discarded rather than passed on or recorded, so it takes + // any JavaScript value and crosses no serialization boundary. + condition = yield* evaluateExpression( + segment.expressions.condition, + "If", + "condition", + segment.projectedEnv, ); + } catch (error) { + owner.push( + yield* raise(ifError(segment, error instanceof Error ? error.message : String(error))), + ); + return; + } + } else { + owner.push(yield* raise(ifError(segment, ifConditionViolation(segment)!.message))); + return; + } - yield* expandSegmentsWithin( - selected ? structure.whenTrue : structure.whenFalse, - parentMeta, - parentProps, - hideSet, - counter, - owner, - branchPath, - 0, - checkedFailures, - environment, - imports, - returnBody, - ); + const selected = !!condition; + + // The false arm belongs to ``, which is consumed above, so its frame is + // added here — otherwise both arms of one `` expand under one path. + const branchPath = + selected || structure.elseElement === undefined + ? path + : extendPath( + path, + elementFrame( + structure.elseElement.name, + elementSite(structure.elseElement.position, structure.elseIndex ?? 0), + ), + ); + + const branch = function* (): Operation { + yield* expandSegmentsWithin( + selected ? structure.whenTrue : structure.whenFalse, + parentMeta, + parentProps, + hideSet, + counter, + owner, + branchPath, + 0, + checkedFailures, + environment, + imports, + returnBody, + ); + }; + // The `` is an element of its own, and only when it is the arm that + // runs: the unselected arm expands nothing, so it is told nothing. + if (selected || structure.elseElement === undefined) { + yield* branch(); + return; + } + yield* structural(structure.elseElement, branchPath, branch); + } } function switchError(segment: ComponentElement, message: string): ErrorSegment { @@ -2095,73 +2145,82 @@ function* expandSwitch( imports: ComponentImportTerminal | undefined, returnBody: ReturnBody | undefined, ): Operation { - // Decided from source alone and shared with validation: which props were - // written, and how the body divides into branches. - const structure = switchStructure(segment); - if (structure.violations.length > 0) { - for (const violation of structure.violations) { - owner.push(yield* raise(structuralErrorSegment(violation, segment))); + return yield* structural(segment, path, () => work()); + + function* work(): Operation { + // Decided from source alone and shared with validation: which props were + // written, and how the body divides into branches. + const structure = switchStructure(segment); + if (structure.violations.length > 0) { + for (const violation of structure.violations) { + owner.push(yield* raise(structuralErrorSegment(violation, segment))); + } + return; } - return; - } - - let selector: unknown; - try { - selector = yield* structuralOperand(segment, "Switch"); - } catch (error) { - owner.push( - yield* raise(switchError(segment, error instanceof Error ? error.message : String(error))), - ); - return; - } - let selected: SwitchCase | undefined; - for (const candidate of structure.matching) { - let matcher: unknown; + let selector: unknown; try { - matcher = yield* structuralOperand(candidate.element, "Case"); + selector = yield* structuralOperand(segment, "Switch"); } catch (error) { owner.push( - yield* raise( - caseError(candidate.element, error instanceof Error ? error.message : String(error)), - ), + yield* raise(switchError(segment, error instanceof Error ? error.message : String(error))), ); return; } - // The `===` operator, and nothing else: `NaN` matches no case including one - // written `NaN`, `0` and `-0` are the same value, and two objects match only - // when they are the same object. - if (selector === matcher) { - selected = candidate; - break; + + let selected: SwitchCase | undefined; + for (const candidate of structure.matching) { + let matcher: unknown; + try { + matcher = yield* structuralOperand(candidate.element, "Case"); + } catch (error) { + owner.push( + yield* raise( + caseError(candidate.element, error instanceof Error ? error.message : String(error)), + ), + ); + return; + } + // The `===` operator, and nothing else: `NaN` matches no case including one + // written `NaN`, `0` and `-0` are the same value, and two objects match only + // when they are the same object. + if (selector === matcher) { + selected = candidate; + break; + } } - } - const chosen = selected ?? structure.fallback; - if (chosen === undefined) { - return; - } + const chosen = selected ?? structure.fallback; + if (chosen === undefined) { + return; + } - // The selected `` is consumed above and never reaches dispatch, so its - // frame is added here — otherwise corresponding children of two branches - // would expand under one path (§5.6). - yield* expandSegmentsWithin( - chosen.element.children, - parentMeta, - parentProps, - hideSet, - counter, - owner, - extendPath( + // The selected `` is consumed above and never reaches dispatch, so its + // frame is added here — otherwise corresponding children of two branches + // would expand under one path (§5.6). + const casePath = extendPath( path, elementFrame(chosen.element.name, elementSite(chosen.element.position, chosen.index)), - ), - 0, - checkedFailures, - environment, - imports, - returnBody, - ); + ); + // The chosen `` is an element of its own; the ones that were not + // chosen expand nothing and are told nothing. + yield* structural(chosen.element, casePath, () => + expandSegmentsWithin( + chosen.element.children, + parentMeta, + parentProps, + hideSet, + counter, + owner, + casePath, + 0, + checkedFailures, + environment, + imports, + returnBody, + ), + ); + } } function loopError(segment: ComponentElement, message: string): ErrorSegment { @@ -2238,103 +2297,107 @@ function* expandLoop( imports: ComponentImportTerminal | undefined, returnBody: ReturnBody | undefined, ): Operation { - const unknownProp = loopPropsViolation(segment); - if (unknownProp !== undefined) { - owner.push(yield* raise(loopError(segment, unknownProp.message))); - return; - } + return yield* structural(segment, path, () => work()); - if ("name" in segment.expressions) { - owner.push( - yield* raise(loopError(segment, 'Prop "name" on must be a string literal.')), - ); - return; - } - const name = segment.props.name; - if (name !== undefined && (typeof name !== "string" || name.length === 0)) { - owner.push( - yield* raise(loopError(segment, 'Prop "name" on must be a non-empty string.')), - ); - return; - } + function* work(): Operation { + const unknownProp = loopPropsViolation(segment); + if (unknownProp !== undefined) { + owner.push(yield* raise(loopError(segment, unknownProp.message))); + return; + } - const bound = yield* resolveLoopBound(segment); - if (!bound.ok) { - owner.push(yield* raise(loopError(segment, bound.error.message))); - return; - } + if ("name" in segment.expressions) { + owner.push( + yield* raise(loopError(segment, 'Prop "name" on must be a string literal.')), + ); + return; + } + const name = segment.props.name; + if (name !== undefined && (typeof name !== "string" || name.length === 0)) { + owner.push( + yield* raise(loopError(segment, 'Prop "name" on must be a non-empty string.')), + ); + return; + } - // Taken from the shared block counter, so every `` an execution enters - // — including each entry into a nested one — has a distinct identity that - // lands the same way on replay. - const identity: LoopIdentity = { - id: counter.next(), - ...(name === undefined ? {} : { name }), - ...(segment.position === undefined ? {} : { position: segment.position }), - }; + const bound = yield* resolveLoopBound(segment); + if (!bound.ok) { + owner.push(yield* raise(loopError(segment, bound.error.message))); + return; + } - const frame: LoopFrame = { broken: false }; - let started = 0; + // Taken from the shared block counter, so every `` an execution enters + // — including each entry into a nested one — has a distinct identity that + // lands the same way on replay. + const identity: LoopIdentity = { + id: counter.next(), + ...(name === undefined ? {} : { name }), + ...(segment.position === undefined ? {} : { position: segment.position }), + }; - try { - yield* scoped(function* () { - yield* ActiveLoop.set(frame); - for (let iteration = 0; iteration < bound.value; iteration++) { - yield* recordIteration(identity, iteration); - started = iteration + 1; - yield* expandSegmentsWithin( - segment.children, - parentMeta, - parentProps, - hideSet, - counter, - owner, - extendPath(path, { f: "iter", i: iteration }), - 0, - checkedFailures, - environment, - imports, - returnBody, - ); - if (frame.broken) { - break; + const frame: LoopFrame = { broken: false }; + let started = 0; + + try { + yield* scoped(function* () { + yield* ActiveLoop.set(frame); + for (let iteration = 0; iteration < bound.value; iteration++) { + yield* recordIteration(identity, iteration); + started = iteration + 1; + yield* expandSegmentsWithin( + segment.children, + parentMeta, + parentProps, + hideSet, + counter, + owner, + extendPath(path, { f: "iter", i: iteration }), + 0, + checkedFailures, + environment, + imports, + returnBody, + ); + if (frame.broken) { + break; + } } + }); + } catch (error) { + // A durability failure is not an outcome of the loop's work, and recording + // one would reach the journal twice over: it appends an entry on top of a + // journal already known not to describe this run, and on replay it consumes + // a terminal entry an earlier run wrote — which is how a stale journal + // would quietly hand this loop a different outcome. The durability failure + // stays the primary error. + const durability = durabilityFailure(error); + if (durability !== undefined) { + // The durability failure itself, not whatever wrapped it. A teardown + // aggregate or an AggregateError says how the failure travelled, not what + // went wrong, and the caller has to see which journal entry stopped + // describing this run. + throw durability; } - }); - } catch (error) { - // A durability failure is not an outcome of the loop's work, and recording - // one would reach the journal twice over: it appends an entry on top of a - // journal already known not to describe this run, and on replay it consumes - // a terminal entry an earlier run wrote — which is how a stale journal - // would quietly hand this loop a different outcome. The durability failure - // stays the primary error. - const durability = durabilityFailure(error); - if (durability !== undefined) { - // The durability failure itself, not whatever wrapped it. A teardown - // aggregate or an AggregateError says how the failure travelled, not what - // went wrong, and the caller has to see which journal entry stopped - // describing this run. - throw durability; - } - // An ordinary document failure is the loop's own outcome. Recorded from the - // catch rather than a destructor: this frame is still live here, so the - // entry lands in the journal before the failure leaves the loop. - try { - yield* recordOutcome(identity, { iterations: started, outcome: "error" }); - } catch (recording) { - // Recording found the journal recording a different outcome. That is the - // more fundamental failure and becomes the primary one, but the document - // failure that reached it is what the author has to fix. - if (recording instanceof Error && recording.cause === undefined) { - recording.cause = error; + // An ordinary document failure is the loop's own outcome. Recorded from the + // catch rather than a destructor: this frame is still live here, so the + // entry lands in the journal before the failure leaves the loop. + try { + yield* recordOutcome(identity, { iterations: started, outcome: "error" }); + } catch (recording) { + // Recording found the journal recording a different outcome. That is the + // more fundamental failure and becomes the primary one, but the document + // failure that reached it is what the author has to fix. + if (recording instanceof Error && recording.cause === undefined) { + recording.cause = error; + } + throw recording; } - throw recording; + throw error; } - throw error; - } - const outcome: LoopOutcome = frame.broken ? "break" : "exhausted"; - yield* recordOutcome(identity, { iterations: started, outcome }); + const outcome: LoopOutcome = frame.broken ? "break" : "exhausted"; + yield* recordOutcome(identity, { iterations: started, outcome }); + } } /** @@ -2434,19 +2497,23 @@ function spawnChild( // a checked command failure inside one child fails that child, and a // sibling's walk is not stopped by a ledger it does not own. const ledger = containedLedger(inherited); - yield* expandSegmentsWithin( - spawn.element.children, - parentMeta, - parentProps, - new Set(hideSet), - createBlockCounter(), - segments, - childPath, - 0, - ledger, - environment, - imports, - undefined, + // Each child the `` actually runs is its own element, observed on its + // own: real siblings reach their phases independently of each other. + yield* structural(spawn.element, childPath, () => + expandSegmentsWithin( + spawn.element.children, + parentMeta, + parentProps, + new Set(hideSet), + createBlockCounter(), + segments, + childPath, + 0, + ledger, + environment, + imports, + undefined, + ), ); yield* refuseCheckedFailure(ledger); // Runs rather than one joined string. What a child produced is prose, or a @@ -2492,57 +2559,61 @@ function* expandAll( environment: ExecutionEnvironment | undefined, imports: ComponentImportTerminal | undefined, ): Operation { - // Decided from source alone and shared with validation, completely, before a - // child is constructed: a malformed `` starts none of them. - const structure = allStructure(segment); - if (structure.violations.length > 0) { - for (const violation of structure.violations) { - owner.push(yield* raise(structuralErrorSegment(violation, segment))); + return yield* structural(segment, path, () => work()); + + function* work(): Operation { + // Decided from source alone and shared with validation, completely, before a + // child is constructed: a malformed `` starts none of them. + const structure = allStructure(segment); + if (structure.violations.length > 0) { + for (const violation of structure.violations) { + owner.push(yield* raise(structuralErrorSegment(violation, segment))); + } + return; } - return; - } - - // Read once, here: every child starts from the same incoming snapshot, and - // reading it inside a child would let whichever ran first decide what the - // others saw. - const incoming = yield* env; - const children = structure.spawns.map( - (spawn) => () => - spawnChild( - spawn, - incoming, - parentMeta, - parentProps, - hideSet, - path, - checkedFailures, - environment, - imports, - ), - ); - const scope = yield* useScope(); - const rendered = - scope.get(DurableContext) === undefined - ? yield* all(children.map((child) => child())) - : (yield* durableAll( - children.map((child) => (): Workflow => ephemeral(retained(child()))), - )).map(readEmissions); - - for (const runs of rendered) { - for (const run of runs) { - if (run.text === "") { - continue; - } - // One segment per run, and the exact ones are marked again here: the - // record is keyed by segment identity, and these segments are this - // region's, not the child's. A child that ran on a previous attempt says - // what its runs were through its own durable close, so a replay marks - // exactly what the live run marked. - const segment: Segment = { type: "text", content: run.text }; - owner.push(segment); - if (run.exact) { - markExactSource(environment?.sourceSegments, [segment]); + // Read once, here: every child starts from the same incoming snapshot, and + // reading it inside a child would let whichever ran first decide what the + // others saw. + const incoming = yield* env; + const children = structure.spawns.map( + (spawn) => () => + spawnChild( + spawn, + incoming, + parentMeta, + parentProps, + hideSet, + path, + checkedFailures, + environment, + imports, + ), + ); + + const scope = yield* useScope(); + const rendered = + scope.get(DurableContext) === undefined + ? yield* all(children.map((child) => child())) + : (yield* durableAll( + children.map((child) => (): Workflow => ephemeral(retained(child()))), + )).map(readEmissions); + + for (const runs of rendered) { + for (const run of runs) { + if (run.text === "") { + continue; + } + // One segment per run, and the exact ones are marked again here: the + // record is keyed by segment identity, and these segments are this + // region's, not the child's. A child that ran on a previous attempt says + // what its runs were through its own durable close, so a replay marks + // exactly what the live run marked. + const segment: Segment = { type: "text", content: run.text }; + owner.push(segment); + if (run.exact) { + markExactSource(environment?.sourceSegments, [segment]); + } } } } @@ -2688,498 +2759,518 @@ function* expandComponent( imports: ComponentImportTerminal | undefined, returnBody: ReturnBody | undefined, ): Operation { - // Cycle detection — Prosser's algorithm - if (hideSet.has(name)) { - return [ - yield* raise({ - type: "error", - message: `Cycle detected: ${name} is already being expanded (hide set: ${[...hideSet].join(" → ")})`, - source: name, - }), - ]; - } - - if (hideSet.size >= MAX_EXPANSION_DEPTH) { - return [ - yield* raise({ - type: "error", - message: `Maximum expansion depth (${MAX_EXPANSION_DEPTH}) exceeded`, - source: name, - }), - ]; - } - - let imported: ComponentDefinition | FunctionComponentDefinition; - // Opened before the ask and settled the moment it answers, however it - // answered: what canonical resolution selected here is what decides whether - // this invocation is in one of this execution's identity domains, and nothing - // on the answer or in the chain carries it (`invocation-identity.ts`). - // - // The frame is this dispatch's closure, and the terminal built for this one - // import is the only thing that can select into it. That is what makes it this - // invocation's own: a sibling `` resolving the same name at the same - // time has its own frame behind its own terminal, and a handler delegating - // through any number of descendant scopes still arrives at this one. - const selection = environment?.componentIdentity?.beginImport(name); - let selected: IdentityDomain | undefined; - let dispatcher: FunctionComponent | undefined; - /** - * Whether canonical execution answered this import for a name it closed. - * - * The provenance exact source is read from. An open import's answer — the - * chain's, unverified — never sets it, so nothing a handler writes into a - * definition can reach the presentation decision below. - */ - let authorizedCanonically = false; - try { - // The public chain answers, and canonical execution decides whether the - // answer is one it produced. In a closed execution — a workflow holding a - // component bundle, a generated fragment holding an allowlist — a handler - // may observe this import, delegate it, and refuse it by throwing; nothing - // it returns is invoked. Without an environment the answer is whatever the - // chain produced, exactly as it always was. - // The offer is open for exactly this ask. Middleware composes inside it and - // may observe, delegate or refuse the import; what it cannot do is obtain - // the declaration for an element that did not author it, because the offer - // is made from the closure the segments being expanded carry, is spent by - // whatever asks first, and authorizes only the answer it produced itself. - const offered = environment?.installedComponents?.offer(environment.componentBodyScope, name); - let answered: ImportedDefinition; - try { - // Through this import's own terminal when the execution handed expansion - // one: the public middleware chain composes around it by name, and `next` - // terminates in the continuation that carries this frame. Where expansion - // was handed none — driven directly, or a generated fragment answering - // through its own narrowed provider — the ordinary public operation answers - // exactly as it always did. - answered = - imports === undefined - ? yield* importComponent(name, position) - : yield* importThroughTerminal(name, position, (asked, at) => - imports(asked, at, selection), - ); - } finally { - offered?.close(); - } - selected = selection?.settle(); - // A private import is authorized by the ask that made the offer, and by - // nothing else. Not by the name: a private component runs for the element - // the declaration that carries it authored, so an answer kept from another - // import — however exactly it describes the same definition — authorizes - // nothing here. And a private name written where no offer was made never - // reaches this at all: selection resolves it to nothing, so what arrives is - // the ordinary unresolved failure. - let authorizedPrivate = false; - if (environment?.installedComponents?.declaresPrivate(name) === true) { - imported = requirePrivate(offered, name, answered); - authorizedPrivate = true; - environment.forms?.select(name, imported); - } else if ( - environment?.componentResolution === undefined || - !environment.componentResolution.resolves(name) - ) { - // Closed for this exact name, not for the execution that closed it. A - // bundled run closes every import; a host that declared exact Markdown - // closed the names it declared, and an unrelated one is the open import it - // has always been — the chain's answer, unverified, with no selection - // recorded against it. - imported = answered; - } else { - imported = environment.componentResolution.verify(name, answered); - // This import is canonical execution's own answer for a name this - // execution closed, which is the only provenance exact source is read - // from. An open import — one no tier claims — never sets it, however its - // answer describes itself. - authorizedCanonically = true; - // Closed authorization answers with core's retained copy rather than the - // object the resolver recorded, and the copy is what this expansion - // invokes — so the selection is recorded against it too. Only here: an - // open execution's answer travelled through public middleware, and - // nothing it hands back is canonical resolution's product. The record - // takes its dispatcher from the copy's own `fn`, identical by retention; - // a wrapper whose `fn` is no dispatcher records nothing, so a dispatcher - // an environment recorded explicitly is never displaced. - environment.forms?.select(name, imported); - } - // Whatever tier answered, and whatever this execution declares, an - // implementation some declaration's private closure built runs only for an - // import that closure authorized. Neither the name nor the current - // execution can decide it: an answer kept from a legitimate private import - // can be returned for any *other* name, in a copy of the definition, and in - // a later run that declares nothing at all — and a run that has ended - // authorizes nothing. - if (!authorizedPrivate) { - refuseEscapedPrivate(name, imported); - } - // Read off the answer rather than from a frame the engine opened: what is - // recognized is the exact definition canonical resolution produced for this - // exact name, whenever it produced it. - dispatcher = environment?.forms?.dispatcherFor(name, imported); - } catch (error) { - selection?.settle(); - // Import is a durable effect, so it is the other place a stale journal - // entry can surface. - const fatal = fatalCause(error); - if (fatal !== undefined) { - throw fatal; + // Surrounded before resolution, props validation or any body: what an + // observer is offered is the whole of this element's expansion, including + // the parts that decide whether it has one. + return yield* observeExpansion(snapshot(path, name, position), (observed) => work(observed)); + + function* work(observed: ObservedExpansion): Operation { + // Cycle detection — Prosser's algorithm + if (hideSet.has(name)) { + return [ + yield* raise({ + type: "error", + message: `Cycle detected: ${name} is already being expanded (hide set: ${[...hideSet].join(" → ")})`, + source: name, + }), + ]; } - return [ - yield* raise({ - type: "error", - message: - error instanceof Error - ? `Failed to import component ${name}: ${error.message}` - : `Failed to import component ${name}: ${String(error)}`, - source: name, - }), - ]; - } - - // Function component: call the generator function directly - if (imported.kind === "function") { - return yield* expandFunctionComponent( - name, - props, - expressions, - authoredExpressions, - children, - selfClosing, - imported, - hideSet, - counter, - projectedEnv, - position, - callerMeta, - callerProps, - owner, - path, - checkedFailures, - environment, - imports, - returnBody, - selected, - dispatcher, - ); - } - - const definition = imported; - - // What this definition's own body may write. A declaration carries its - // private closure here; every other body carries whatever the caller carried, - // which for an ordinary document is nothing. Decided from the definition - // canonical resolution retained — the one this expansion is about to invoke, - // reporting the origin core declared — rather than from the name alone. - const bodyEnvironment = environmentForBody(environment, name, definition); - - const placementError = validateBodyStructure(definition.bodySegments, definition.returns); - if (placementError) { - return [yield* raise(placementError)]; - } - - const asExpression = asExpressionViolation(name, expressions); - if (asExpression !== undefined) { - return [yield* raise({ type: "error", message: asExpression.message, source: name })]; - } - - // Resolve eval expression props against env.values using the shared - // VM context. This must happen before validation so that resolved - // values can be type-checked. See spec §5.1 (expression prop evaluation). - let resolvedProps: Record; - try { - resolvedProps = yield* resolveExpressionProps(props, expressions, name, projectedEnv); - } catch (error) { - return [ - yield* raise({ - type: "error", - message: error instanceof Error ? error.message : String(error), - source: name, - }), - ]; - } - // Validate caller props against the component's declared props schema. - // Strip the `slot` prop before validation — it is consumed by the - // expansion engine for slot assignment, not forwarded to the child. - let validatedProps: Record; - let asBinding: string | undefined; - try { - const refused = asBindingViolation(name, resolvedProps.as); - if (refused !== undefined) { - throw new Error(refused.message); + if (hideSet.size >= MAX_EXPANSION_DEPTH) { + return [ + yield* raise({ + type: "error", + message: `Maximum expansion depth (${MAX_EXPANSION_DEPTH}) exceeded`, + source: name, + }), + ]; } - asBinding = capturedBinding(resolvedProps.as); - - const { slot: _slot, as: _as, ...propsForValidation } = resolvedProps; - validatedProps = yield* validateProps(name, propsForValidation, definition.props); - } catch (error) { - return [yield* raise(schemaValidationErrorSegment(error, name))]; - } - - const missingCapture = returnCaptureViolation(name, definition.returns !== undefined, asBinding); - if (missingCapture !== undefined) { - return [yield* raise({ type: "error", message: missingCapture.message, source: name })]; - } - - // Capture the caller's eval environment before creating the component's - // own env. Children are caller-provided content — expression props like - // {pr} should resolve against the scope where the JSX was written, not - // the component that renders . - // - // For multi-level nesting (Root → Provider → Instruction → ReviewBody), - // the projectedEnv from the outer caller must be merged with the current - // context env so that ancestor bindings propagate through all levels. - // The current context env's ordinary bindings take precedence; the shared - // layering helper keeps a projected caller's props object lexical. - const contextEnv = yield* env; - const callerEvalEnv = layerEnvironments(projectedEnv, contextEnv); - - // Recurse with augmented hide set. - // Each component gets its own fresh binding environment so that - // eval blocks within a component share bindings but don't leak - // into parent or sibling components. This is critical for the - // provider pattern where each provider has isolated port/URL bindings. - // - // Each component also gets its own EvalScope, created as a child of - // the parent component's eval scope. This ensures that middleware - // installed via `persist eval` blocks (e.g., Sample.around()) is - // scoped to the component. Nested providers produce a scope chain - // where innermost middleware runs first (innermost-wins), and - // next() delegates to the parent scope's middleware. - const newHideSet = new Set([...hideSet, name]); - const componentEnv: EvalEnv = propsEnvironment(validatedProps); - liveEnvironment(componentEnv); - - // Children are caller-provided content, not the component's own body. - // Use the parent's hide set (without the current component name) so - // that caller-provided children can reference the same component name - // without triggering false cycle detection. True cycles in a component's - // body are still caught because body expansion uses newHideSet. - // - // Use the caller's eval env for the same reason: expression props (e.g. - // {pr}) resolve against the scope where the JSX was written. - const capturedCallerEnv = callerEvalEnv ?? componentEnv; - // A body eval block can render errors away through a string projection - // (renderChildren, render, useContent); the buffer records them so an - // `as=` capture can be refused (§6.5). - const bodyContentErrors: Segment[] = []; - - // Read before the invocation exists: the eval scope ambient here is the - // caller's, and it is what `retain()` creates resources in. The loop is read - // here for the same reason — it is the one the caller's content was written - // in, and the invocation is about to clear it for the component's own body. - const siteEvalScope = yield* evalScope; - const siteLoop = yield* ActiveLoop.get(); - const siteReturn = returnBody; - - const expansion = snapshot(path, name, position); - - // Both bodies run inside one invocation, so a value component owns its - // resources exactly like a rendered one. - let claimProjection: ClaimFn = passthroughClaim; - - function* installInvocation(invocation: Invocation): Operation { - const enclosing = yield* ActiveProjection.get(); - const handle = createProjectionHandle({ - invocation, - enclosing, - children, - caller: { - env: capturedCallerEnv, - meta: callerMeta, - props: callerProps, - hideSet, - }, - authored: { - env: componentEnv, - meta: definition.meta, - props: validatedProps, - hideSet: newHideSet, - }, - counter, - callerLoop: siteLoop, - callerReturn: siteReturn, - ownPath: path, - printedErrors: bodyContentErrors, - checkedFailures, - environment, - imports, - }); - // Published on the eval scope, which every task the invocation owns - // descends from — including its persist-eval blocks and its content. - invocation.evalScope.scope.set(ActiveProjection, handle); - claimProjection = handle.claim; - - // The component's own body is isolated from the loop that invoked it: a - // `` written here belongs to a `` written here. Set on the body - // task, which the content scope descends from — so each projection restores - // the caller's frame for the caller's own text (createProjectionHandle), - // and anything the body does outside a projection finds none. - yield* ActiveLoop.set(undefined); - - // And from the caller's value body for the same reason: a written - // here satisfies this component's own `returns`, never the caller's. A - // value body installs its own frame below; a rendered one owns none, so a - // written in it stays reserved. - // Published on the body task, so the component's own body and everything it - // owns read this expansion, and a nested one uncovers it again on the way - // out (§5.6). - yield* publishExpansion(expansion); - - // Installed on the invocation's own body task rather than a nested - // scoped(): anything the body acquires must still be alive when teardown - // halts the content scope, and it is released by the body's own stage. - yield* provideEnv(componentEnv); - yield* provideEvalScope(invocation.evalScope); - yield* provideRetain(siteEvalScope); - - // Render closures (spec §4.8). Non-serializable, so serializeExports - // omits them from the journal. The optional error mode is supplied by a - // persistent evaluation's binding snapshot (§4.3), which knows the error mode of the - // block that started the projection; an ordinary block leaves it unset and - // the projection site's error mode applies. - componentEnv.values.renderChildren = (override?: unknown, mode?: ErrorMode) => - handle.projectToString({ - kind: "children", - override: validateRenderOverride(override), - mode, - }); - componentEnv.values.render = (markdown: unknown, mode?: ErrorMode) => - handle.projectToString({ - kind: "markdown", - segments: scanSegments(String(markdown)), - mode, - }); - componentEnv.values.useContent = (slot?: unknown, mode?: ErrorMode) => - handle.projectToString({ - kind: "slot", - name: slot === undefined ? undefined : String(slot), - mode, - }); - } - const returns = definition.returns; - if (returns !== undefined) { - let value: Json; + let imported: ComponentDefinition | FunctionComponentDefinition; + // Opened before the ask and settled the moment it answers, however it + // answered: what canonical resolution selected here is what decides whether + // this invocation is in one of this execution's identity domains, and nothing + // on the answer or in the chain carries it (`invocation-identity.ts`). + // + // The frame is this dispatch's closure, and the terminal built for this one + // import is the only thing that can select into it. That is what makes it this + // invocation's own: a sibling `` resolving the same name at the same + // time has its own frame behind its own terminal, and a handler delegating + // through any number of descendant scopes still arrives at this one. + const selection = environment?.componentIdentity?.beginImport(name); + let selected: IdentityDomain | undefined; + let dispatcher: FunctionComponent | undefined; + /** + * Whether canonical execution answered this import for a name it closed. + * + * The provenance exact source is read from. An open import's answer — the + * chain's, unverified — never sets it, so nothing a handler writes into a + * definition can reach the presentation decision below. + */ + let authorizedCanonically = false; try { - value = yield* withInvocation(function* (invocation) { - yield* installInvocation(invocation); - return yield* expandValueBody( - name, - returns, - definition.bodySegments, - children, - definition.meta, - validatedProps, - newHideSet, - counter, - callerEvalEnv ?? undefined, - claimProjection, - path, - checkedFailures, - bodyEnvironment, - imports, - returnBody, - ); - }); + // The public chain answers, and canonical execution decides whether the + // answer is one it produced. In a closed execution — a workflow holding a + // component bundle, a generated fragment holding an allowlist — a handler + // may observe this import, delegate it, and refuse it by throwing; nothing + // it returns is invoked. Without an environment the answer is whatever the + // chain produced, exactly as it always was. + // The offer is open for exactly this ask. Middleware composes inside it and + // may observe, delegate or refuse the import; what it cannot do is obtain + // the declaration for an element that did not author it, because the offer + // is made from the closure the segments being expanded carry, is spent by + // whatever asks first, and authorizes only the answer it produced itself. + const offered = environment?.installedComponents?.offer(environment.componentBodyScope, name); + let answered: ImportedDefinition; + try { + // Through this import's own terminal when the execution handed expansion + // one: the public middleware chain composes around it by name, and `next` + // terminates in the continuation that carries this frame. Where expansion + // was handed none — driven directly, or a generated fragment answering + // through its own narrowed provider — the ordinary public operation answers + // exactly as it always did. + answered = + imports === undefined + ? yield* importComponent(name, position) + : yield* importThroughTerminal(name, position, (asked, at) => + imports(asked, at, selection), + ); + } finally { + offered?.close(); + } + selected = selection?.settle(); + // A private import is authorized by the ask that made the offer, and by + // nothing else. Not by the name: a private component runs for the element + // the declaration that carries it authored, so an answer kept from another + // import — however exactly it describes the same definition — authorizes + // nothing here. And a private name written where no offer was made never + // reaches this at all: selection resolves it to nothing, so what arrives is + // the ordinary unresolved failure. + let authorizedPrivate = false; + if (environment?.installedComponents?.declaresPrivate(name) === true) { + imported = requirePrivate(offered, name, answered); + authorizedPrivate = true; + environment.forms?.select(name, imported); + } else if ( + environment?.componentResolution === undefined || + !environment.componentResolution.resolves(name) + ) { + // Closed for this exact name, not for the execution that closed it. A + // bundled run closes every import; a host that declared exact Markdown + // closed the names it declared, and an unrelated one is the open import it + // has always been — the chain's answer, unverified, with no selection + // recorded against it. + imported = answered; + } else { + imported = environment.componentResolution.verify(name, answered); + // This import is canonical execution's own answer for a name this + // execution closed, which is the only provenance exact source is read + // from. An open import — one no tier claims — never sets it, however its + // answer describes itself. + authorizedCanonically = true; + // Closed authorization answers with core's retained copy rather than the + // object the resolver recorded, and the copy is what this expansion + // invokes — so the selection is recorded against it too. Only here: an + // open execution's answer travelled through public middleware, and + // nothing it hands back is canonical resolution's product. The record + // takes its dispatcher from the copy's own `fn`, identical by retention; + // a wrapper whose `fn` is no dispatcher records nothing, so a dispatcher + // an environment recorded explicitly is never displaced. + environment.forms?.select(name, imported); + } + // Whatever tier answered, and whatever this execution declares, an + // implementation some declaration's private closure built runs only for an + // import that closure authorized. Neither the name nor the current + // execution can decide it: an answer kept from a legitimate private import + // can be returned for any *other* name, in a copy of the definition, and in + // a later run that declares nothing at all — and a run that has ended + // authorizes nothing. + if (!authorizedPrivate) { + refuseEscapedPrivate(name, imported); + } + // Read off the answer rather than from a frame the engine opened: what is + // recognized is the exact definition canonical resolution produced for this + // exact name, whenever it produced it. + dispatcher = environment?.forms?.dispatcherFor(name, imported); } catch (error) { - // Body fail-fast propagates unchanged; a return-value failure is the - // component's own printed error and follows the caller's error mode. + selection?.settle(); + // Import is a durable effect, so it is the other place a stale journal + // entry can surface. const fatal = fatalCause(error); if (fatal !== undefined) { throw fatal; } - return [yield* raise(schemaValidationErrorSegment(error, name))]; - } - - // Bind only after the invocation has torn down, so the value reaches the - // caller's environment and never the component's own. - const parentEnv = yield* env; - if (!parentEnv || asBinding === undefined) { return [ yield* raise({ type: "error", - message: `Prop "as" on <${name} /> requires a parent evaluation environment.`, + message: + error instanceof Error + ? `Failed to import component ${name}: ${error.message}` + : `Failed to import component ${name}: ${String(error)}`, source: name, }), ]; } - parentEnv.values[asBinding] = value; - return []; - } - const bodyOwner = asBinding === undefined ? owner : undefined; - // Where this body's own segments start, so what it renders can be told apart - // from what the caller had already produced into the same array. - const renderedFrom = bodyOwner?.length ?? 0; - const expanded = yield* withInvocation(function* (invocation) { - yield* installInvocation(invocation); - return yield* expandBody( - definition.bodySegments, - children, - definition.meta, - validatedProps, - newHideSet, - counter, - callerEvalEnv ?? undefined, - claimProjection, - bodyOwner, - path, - checkedFailures, - bodyEnvironment, - imports, - returnBody, - ); - }); + // Function component: call the generator function directly + if (imported.kind === "function") { + return yield* expandFunctionComponent( + observed, + name, + props, + expressions, + authoredExpressions, + children, + selfClosing, + imported, + hideSet, + counter, + projectedEnv, + position, + callerMeta, + callerProps, + owner, + path, + checkedFailures, + environment, + imports, + returnBody, + selected, + dispatcher, + ); + } - // Exact source is a provenance, and both halves of it are read here from - // things no answer can write: that canonical execution authorized this import - // for a name this execution closed, and that the *host's own admitted - // declaration* for that name states exact source. A definition claiming the - // disposition, or segments arriving already marked, decide nothing. - // - // What a component declared exact renders is exact wherever it renders: the - // caller's flow, its own returned region, or neither when the invocation - // binds instead. Recording it here — against the segments this body produced, - // after it produced them — is what carries the fact to the emission loop, - // which is outside every scope the invocation owned. - if (authorizedCanonically && environment?.installedComponents?.declaresExact(name) === true) { - markExactSource( - environment?.sourceSegments, - bodyOwner === undefined ? expanded : bodyOwner.slice(renderedFrom), - ); - } + const definition = imported; - if (asBinding) { - // A capture never swallows an error. The body reported these where they - // were created (§6.9) — including the ones a string projection rendered - // away — so they are handed back as they are and the binding stays unset. - // Recorded errors precede expanded ones regardless of source position; - // the order across the two lists is deliberately unspecified. - const errors = [ - ...bodyContentErrors, - ...expanded.filter((capturedSegment) => capturedSegment.type === "error"), - ]; - if (errors.length > 0) { - return errors; + // What this definition's own body may write. A declaration carries its + // private closure here; every other body carries whatever the caller carried, + // which for an ordinary document is nothing. Decided from the definition + // canonical resolution retained — the one this expansion is about to invoke, + // reporting the origin core declared — rather than from the name alone. + const bodyEnvironment = environmentForBody(environment, name, definition); + + const placementError = validateBodyStructure(definition.bodySegments, definition.returns); + if (placementError) { + return [yield* raise(placementError)]; + } + + const asExpression = asExpressionViolation(name, expressions); + if (asExpression !== undefined) { + return [yield* raise({ type: "error", message: asExpression.message, source: name })]; } - const parentEnv = yield* env; - if (!parentEnv) { + // Resolve eval expression props against env.values using the shared + // VM context. This must happen before validation so that resolved + // values can be type-checked. See spec §5.1 (expression prop evaluation). + let resolvedProps: Record; + try { + resolvedProps = yield* resolveExpressionProps(props, expressions, name, projectedEnv); + } catch (error) { return [ yield* raise({ type: "error", - message: `Prop "as" on <${name} /> requires a parent evaluation environment.`, + message: error instanceof Error ? error.message : String(error), source: name, }), ]; } - parentEnv.values[asBinding] = renderSegments(expanded); - return []; - } - // A rendering body already wrote into the owner, so there is nothing left to - // hand back; one that kept its own returns what it rendered. - return bodyOwner === undefined ? expanded : []; + // Validate caller props against the component's declared props schema. + // Strip the `slot` prop before validation — it is consumed by the + // expansion engine for slot assignment, not forwarded to the child. + let validatedProps: Record; + let asBinding: string | undefined; + try { + const refused = asBindingViolation(name, resolvedProps.as); + if (refused !== undefined) { + throw new Error(refused.message); + } + asBinding = capturedBinding(resolvedProps.as); + + const { slot: _slot, as: _as, ...propsForValidation } = resolvedProps; + validatedProps = yield* validateProps(name, propsForValidation, definition.props); + } catch (error) { + return [yield* raise(schemaValidationErrorSegment(error, name))]; + } + // Resolution and validation accepted this element, so whatever body it has + // is the work that was accepted. Published here rather than at each body, + // because every path past this point is work — and a refusal above it is + // an element that never became active at all. + observed.active(); + + const missingCapture = returnCaptureViolation( + name, + definition.returns !== undefined, + asBinding, + ); + if (missingCapture !== undefined) { + return [yield* raise({ type: "error", message: missingCapture.message, source: name })]; + } + + // Capture the caller's eval environment before creating the component's + // own env. Children are caller-provided content — expression props like + // {pr} should resolve against the scope where the JSX was written, not + // the component that renders . + // + // For multi-level nesting (Root → Provider → Instruction → ReviewBody), + // the projectedEnv from the outer caller must be merged with the current + // context env so that ancestor bindings propagate through all levels. + // The current context env's ordinary bindings take precedence; the shared + // layering helper keeps a projected caller's props object lexical. + const contextEnv = yield* env; + const callerEvalEnv = layerEnvironments(projectedEnv, contextEnv); + + // Recurse with augmented hide set. + // Each component gets its own fresh binding environment so that + // eval blocks within a component share bindings but don't leak + // into parent or sibling components. This is critical for the + // provider pattern where each provider has isolated port/URL bindings. + // + // Each component also gets its own EvalScope, created as a child of + // the parent component's eval scope. This ensures that middleware + // installed via `persist eval` blocks (e.g., Sample.around()) is + // scoped to the component. Nested providers produce a scope chain + // where innermost middleware runs first (innermost-wins), and + // next() delegates to the parent scope's middleware. + const newHideSet = new Set([...hideSet, name]); + const componentEnv: EvalEnv = propsEnvironment(validatedProps); + liveEnvironment(componentEnv); + + // Children are caller-provided content, not the component's own body. + // Use the parent's hide set (without the current component name) so + // that caller-provided children can reference the same component name + // without triggering false cycle detection. True cycles in a component's + // body are still caught because body expansion uses newHideSet. + // + // Use the caller's eval env for the same reason: expression props (e.g. + // {pr}) resolve against the scope where the JSX was written. + const capturedCallerEnv = callerEvalEnv ?? componentEnv; + // A body eval block can render errors away through a string projection + // (renderChildren, render, useContent); the buffer records them so an + // `as=` capture can be refused (§6.5). + const bodyContentErrors: Segment[] = []; + + // Read before the invocation exists: the eval scope ambient here is the + // caller's, and it is what `retain()` creates resources in. The loop is read + // here for the same reason — it is the one the caller's content was written + // in, and the invocation is about to clear it for the component's own body. + const siteEvalScope = yield* evalScope; + const siteLoop = yield* ActiveLoop.get(); + const siteReturn = returnBody; + + const expansion = snapshot(path, name, position); + + // Both bodies run inside one invocation, so a value component owns its + // resources exactly like a rendered one. + let claimProjection: ClaimFn = passthroughClaim; + + function* installInvocation(invocation: Invocation): Operation { + // Resolution and validation accepted this element, and its body is about + // to run: that is what makes it active, and both bodies install here. + observed.active(); + const enclosing = yield* ActiveProjection.get(); + const handle = createProjectionHandle({ + invocation, + enclosing, + children, + caller: { + env: capturedCallerEnv, + meta: callerMeta, + props: callerProps, + hideSet, + }, + authored: { + env: componentEnv, + meta: definition.meta, + props: validatedProps, + hideSet: newHideSet, + }, + counter, + callerLoop: siteLoop, + callerReturn: siteReturn, + ownPath: path, + printedErrors: bodyContentErrors, + checkedFailures, + environment, + imports, + }); + // Published on the eval scope, which every task the invocation owns + // descends from — including its persist-eval blocks and its content. + invocation.evalScope.scope.set(ActiveProjection, handle); + claimProjection = handle.claim; + + // The component's own body is isolated from the loop that invoked it: a + // `` written here belongs to a `` written here. Set on the body + // task, which the content scope descends from — so each projection restores + // the caller's frame for the caller's own text (createProjectionHandle), + // and anything the body does outside a projection finds none. + yield* ActiveLoop.set(undefined); + + // And from the caller's value body for the same reason: a written + // here satisfies this component's own `returns`, never the caller's. A + // value body installs its own frame below; a rendered one owns none, so a + // written in it stays reserved. + // Published on the body task, so the component's own body and everything it + // owns read this expansion, and a nested one uncovers it again on the way + // out (§5.6). + yield* publishExpansion(expansion); + + // Installed on the invocation's own body task rather than a nested + // scoped(): anything the body acquires must still be alive when teardown + // halts the content scope, and it is released by the body's own stage. + yield* provideEnv(componentEnv); + yield* provideEvalScope(invocation.evalScope); + yield* provideRetain(siteEvalScope); + + // Render closures (spec §4.8). Non-serializable, so serializeExports + // omits them from the journal. The optional error mode is supplied by a + // persistent evaluation's binding snapshot (§4.3), which knows the error mode of the + // block that started the projection; an ordinary block leaves it unset and + // the projection site's error mode applies. + componentEnv.values.renderChildren = (override?: unknown, mode?: ErrorMode) => + handle.projectToString({ + kind: "children", + override: validateRenderOverride(override), + mode, + }); + componentEnv.values.render = (markdown: unknown, mode?: ErrorMode) => + handle.projectToString({ + kind: "markdown", + segments: scanSegments(String(markdown)), + mode, + }); + componentEnv.values.useContent = (slot?: unknown, mode?: ErrorMode) => + handle.projectToString({ + kind: "slot", + name: slot === undefined ? undefined : String(slot), + mode, + }); + } + + const returns = definition.returns; + if (returns !== undefined) { + let value: Json; + try { + value = yield* withInvocation(function* (invocation) { + yield* installInvocation(invocation); + return yield* expandValueBody( + name, + returns, + definition.bodySegments, + children, + definition.meta, + validatedProps, + newHideSet, + counter, + callerEvalEnv ?? undefined, + claimProjection, + path, + checkedFailures, + bodyEnvironment, + imports, + returnBody, + ); + }, observed.settled); + } catch (error) { + // Body fail-fast propagates unchanged; a return-value failure is the + // component's own printed error and follows the caller's error mode. + const fatal = fatalCause(error); + if (fatal !== undefined) { + throw fatal; + } + return [yield* raise(schemaValidationErrorSegment(error, name))]; + } + + // Bind only after the invocation has torn down, so the value reaches the + // caller's environment and never the component's own. + const parentEnv = yield* env; + if (!parentEnv || asBinding === undefined) { + return [ + yield* raise({ + type: "error", + message: `Prop "as" on <${name} /> requires a parent evaluation environment.`, + source: name, + }), + ]; + } + parentEnv.values[asBinding] = value; + return []; + } + + const bodyOwner = asBinding === undefined ? owner : undefined; + // Where this body's own segments start, so what it renders can be told apart + // from what the caller had already produced into the same array. + const renderedFrom = bodyOwner?.length ?? 0; + const expanded = yield* withInvocation(function* (invocation) { + yield* installInvocation(invocation); + return yield* expandBody( + definition.bodySegments, + children, + definition.meta, + validatedProps, + newHideSet, + counter, + callerEvalEnv ?? undefined, + claimProjection, + bodyOwner, + path, + checkedFailures, + bodyEnvironment, + imports, + returnBody, + ); + }, observed.settled); + + // Exact source is a provenance, and both halves of it are read here from + // things no answer can write: that canonical execution authorized this import + // for a name this execution closed, and that the *host's own admitted + // declaration* for that name states exact source. A definition claiming the + // disposition, or segments arriving already marked, decide nothing. + // + // What a component declared exact renders is exact wherever it renders: the + // caller's flow, its own returned region, or neither when the invocation + // binds instead. Recording it here — against the segments this body produced, + // after it produced them — is what carries the fact to the emission loop, + // which is outside every scope the invocation owned. + if (authorizedCanonically && environment?.installedComponents?.declaresExact(name) === true) { + markExactSource( + environment?.sourceSegments, + bodyOwner === undefined ? expanded : bodyOwner.slice(renderedFrom), + ); + } + + if (asBinding) { + // A capture never swallows an error. The body reported these where they + // were created (§6.9) — including the ones a string projection rendered + // away — so they are handed back as they are and the binding stays unset. + // Recorded errors precede expanded ones regardless of source position; + // the order across the two lists is deliberately unspecified. + const errors = [ + ...bodyContentErrors, + ...expanded.filter((capturedSegment) => capturedSegment.type === "error"), + ]; + if (errors.length > 0) { + return errors; + } + + const parentEnv = yield* env; + if (!parentEnv) { + return [ + yield* raise({ + type: "error", + message: `Prop "as" on <${name} /> requires a parent evaluation environment.`, + source: name, + }), + ]; + } + parentEnv.values[asBinding] = renderSegments(expanded); + return []; + } + + // A rendering body already wrote into the owner, so there is nothing left to + // hand back; one that kept its own returns what it rendered. + return bodyOwner === undefined ? expanded : []; + } } // Without `returns`, a function component's rendering is its return value, so @@ -3270,6 +3361,11 @@ function* raiseFrom(segment: ErrorSegment, from: unknown): Operation, expressions: Record, @@ -3450,6 +3546,7 @@ function* expandFunctionComponent( // in scope so it can render its invocation content through `yield* content()`. try { const output: unknown = yield* withInvocation(function* (invocation) { + observed.active(); const enclosing = yield* ActiveProjection.get(); // Minted before the handle, because the handle is what raises it: an // invocation names nothing while it is expanding its own content, and @@ -3697,7 +3794,7 @@ function* expandFunctionComponent( } throw new ThrownValue(error); } - }); + }, observed.settled); if (asBinding) { const parentEnv = yield* env; if (!parentEnv) { diff --git a/packages/core/src/invocation.ts b/packages/core/src/invocation.ts index 8c589a32d..dff283a68 100644 --- a/packages/core/src/invocation.ts +++ b/packages/core/src/invocation.ts @@ -77,8 +77,28 @@ function asError(cause: unknown): Error { * failure and a single `InvocationTeardownError` — each original error stays * reachable by identity, which is what `fatalCause()` traverses. */ -export function* withInvocation(body: (invocation: Invocation) => Operation): Operation { +export function* withInvocation( + body: (invocation: Invocation) => Operation, + /** + * Told once that the body's own work has ended, and why, before any of the + * ordered teardown stages run. + * + * An observer of the invocation, not a participant in it: nothing it does + * changes the outcome, and it cannot see what the body produced. It is here + * rather than around the call because "the body ended" and "the invocation + * returned" are different moments — the destructors between them are exactly + * what a reader watching an element wants to see it waiting in. + */ + settled?: (reason: "returned" | "failed" | "cancelled") => void, +): Operation { return yield* scoped(function* () { + let told = false; + const tell = (reason: "returned" | "failed" | "cancelled"): void => { + if (!told) { + told = true; + settled?.(reason); + } + }; let bodyFailure: Error | undefined; const teardownFailures: unknown[] = []; @@ -185,10 +205,20 @@ export function* withInvocation(body: (invocation: Invocation) => Operation { + tell("cancelled"); + }); + start.resolve(invocation); try { - return yield* resultPublished.operation; + const value = yield* resultPublished.operation; + tell("returned"); + return value; } catch (error) { + tell("failed"); // Recorded so the outcome destructor can keep it when a teardown stage // also fails; a destructor throw would otherwise replace it as the // scope's failure. diff --git a/packages/core/src/scanner.ts b/packages/core/src/scanner.ts index 6cdf8f188..86bae0e9a 100644 --- a/packages/core/src/scanner.ts +++ b/packages/core/src/scanner.ts @@ -12,6 +12,7 @@ import { Err, Ok } from "effection"; import type { Result } from "effection"; import { validateBindingName } from "./live-env.ts"; +import type { SourceRange } from "./document-targets.ts"; import { bindsByReference, isStructural } from "./structural.ts"; import type { Segment, @@ -211,6 +212,16 @@ export function scanSegments( text: string, origin?: SourceOrigin, spans?: ComponentSpan[], + /** + * Where every recognized element's delimiters are, collected as the scan + * commits each one. + * + * The same walk, asked for one more thing. A second reader looking for `<` + * and `>` afterwards would be a second grammar — it would have to decide + * again what a fence, a quoted `>` and a tag-like expression are — and the + * two readings would disagree the first time one of them was wrong. + */ + elements?: SourceElement[], ): Segment[] { const index: PositionIndex = { origin, lineStarts: computeLineStarts(text) }; const segments: Segment[] = []; @@ -269,7 +280,7 @@ export function scanSegments( // Check for component invocation: `<` followed by uppercase letter if (text[pos] === "<" && pos + 1 < text.length && /[A-Z]/.test(text[pos + 1]!)) { // Make sure we're not inside a fenced code block (handled above) - const component = parseComponentTag(text, pos, index); + const component = parseComponentTag(text, pos, index, elements); if (component) { // Flush text before component if (pos > textStart) { @@ -294,6 +305,22 @@ export function scanSegments( return segments; } +/** + * One recognized element's delimiters, as the scanner committed them. + * + * `name` is the authored spelling, dots included. It says nothing about + * whether the name resolves, what it is allowed to do, or whether anything + * ever ran: this is where the characters are, and no more than that. + * + * `closing` is absent for exactly self-closing syntax. An element written with + * paired delimiters and no content between them still has one. + */ +export interface SourceElement { + readonly name: string; + readonly opening: SourceRange; + readonly closing?: SourceRange; +} + /** The source spans of the top-level component invocations in `text`. */ export function scanComponentSpans(text: string): ComponentSpan[] { const spans: ComponentSpan[] = []; @@ -436,6 +463,7 @@ function parseComponentTag( text: string, start: number, index: PositionIndex, + elements?: SourceElement[], ): ParsedComponent | null { let pos = start + 1; // Skip '<' @@ -476,6 +504,7 @@ function parseComponentTag( } pos++; // Skip '>' + elements?.push(Object.freeze({ name, opening: Object.freeze({ start, end: pos }) })); return { segment: { type: "component", @@ -496,13 +525,29 @@ function parseComponentTag( return null; } pos++; // Skip '>' + const openingEnd = pos; - // Parse children until closing tag - const { children, end: childEnd } = parseChildren(text, pos, name, index); + // Parse children until closing tag. Nested elements are collected aside and + // committed only with this parse: an incomplete passive tag has no children + // the scanner recognized, so it must leave none behind. + const nested: SourceElement[] | undefined = elements === undefined ? undefined : []; + const { children, end: childEnd } = parseChildren(text, pos, name, index, nested); if (childEnd === -1) { return null; } + if (elements !== undefined && nested !== undefined) { + // This element before the ones inside it, which is opening order. + elements.push( + Object.freeze({ + name, + opening: Object.freeze({ start, end: openingEnd }), + closing: Object.freeze({ start: childEnd - ``.length, end: childEnd }), + }), + ); + elements.push(...nested); + } + return { segment: { type: "component", @@ -819,6 +864,7 @@ function parseChildren( start: number, tagName: string, index: PositionIndex, + elements?: SourceElement[], ): ParsedChildren { let pos = start; let textStart = pos; @@ -884,7 +930,7 @@ function parseChildren( // Check for nested component if (text[pos] === "<" && pos + 1 < text.length && /[A-Z]/.test(text[pos + 1]!)) { - const nested = parseComponentTag(text, pos, index); + const nested = parseComponentTag(text, pos, index, elements); if (nested) { if (pos > textStart) { pushText(children, text.slice(textStart, pos)); diff --git a/packages/core/src/source-inspection.ts b/packages/core/src/source-inspection.ts new file mode 100644 index 000000000..8242560c2 --- /dev/null +++ b/packages/core/src/source-inspection.ts @@ -0,0 +1,130 @@ +/** + * Where the executable elements of a source text are written (spec §5.8). + * + * A reading, not a run. It answers where each recognized element's delimiters + * are and what the author called it, and it does nothing else: no name is + * resolved, no schema compiled, no prop evaluated, no frontmatter interpreted + * and no component invoked. A caller that wants to show somebody their own + * source — with the parts that execute marked — needs exactly this and must + * not pay execution to get it. + * + * It is the scanner's own reading. The ranges come from the same walk that + * decides what a fence, an inline code span, a quoted `>` and a tag-like + * expression are, so there is no second grammar to disagree with the first. + * + * Offsets are half-open UTF-16 slices into the exact text that was passed in, + * a document's header prefix included, so `text.slice(opening.start, + * opening.end)` is the delimiter as authored. + */ + +import { Err, Ok } from "effection"; +import type { Result } from "effection"; +import { scanSegments } from "./scanner.ts"; +import type { SourceElement } from "./scanner.ts"; +import type { SourceRange } from "./document-targets.ts"; + +export type { SourceElement, SourceRange }; + +/** + * Every executable element `text` is written with, in opening order. + * + * `"document"` reads the markdown body beneath a frontmatter header, with + * offsets still counted from the start of the whole text. `"fragment"` reads + * all of it. Text the scanner does not recognize as an element — prose, a + * fence, an inline code span, an incomplete tag — is simply absent, which is + * success and not a refusal. + */ +export function inspectSource( + text: string, + kind: "document" | "fragment", +): Result { + const body = kind === "document" ? envelopeBody(text) : 0; + const elements: SourceElement[] = []; + try { + scanSegments(text.slice(body), undefined, undefined, elements); + } catch (error) { + // The scanner refused this text. Inspection reports that rather than + // guessing at a reading the engine itself would not accept. + return Err(error instanceof Error ? error : new Error(String(error))); + } + return Ok(Object.freeze(body === 0 ? elements : elements.map((one) => shifted(one, body)))); +} + +function shifted(element: SourceElement, by: number): SourceElement { + return Object.freeze({ + name: element.name, + opening: Object.freeze({ start: element.opening.start + by, end: element.opening.end + by }), + ...(element.closing === undefined + ? {} + : { + closing: Object.freeze({ + start: element.closing.start + by, + end: element.closing.end + by, + }), + }), + }); +} + +/** + * Where a document's markdown body starts, read lexically. + * + * The same boundary `parseSource` ends up with, found without asking the + * frontmatter engine for a value: inspection must not run a YAML parser over + * somebody's header to learn where their body begins, and a document whose + * header is itself executable text must stay unexecuted. `parseSource` asserts + * the two agree, so a divergence is loud rather than a quiet offset drift. + * + * The rule is the installed extractor's: an optional byte-order mark, then + * `---` on its own opening line — `----` is not one — an optional language + * after it, the first following newline, and the first later line that is + * exactly the closing delimiter. A header with no closing delimiter is not a + * header, and the body is the whole text. + */ +export function envelopeBody(text: string): number { + // A byte-order mark is never part of the body, whether or not a header + // follows it. A document with no header at all still starts after it. + const mark = text.charCodeAt(0) === 0xfeff ? 1 : 0; + if (!text.startsWith("---", mark) || text.charAt(mark + 3) === "-") { + return mark; + } + // An opener with nothing after it consumes the whole text: there is a header + // and no body, rather than a body that happens to look like a delimiter. + const opened = text.indexOf("\n", mark + 3); + if (opened === -1) { + return text.length; + } + const closed = closingDelimiter(text, opened + 1); + if (closed === -1) { + return text.length; + } + // Immediately after the three characters that close it, and then one line + // ending if one is there. A closing line with anything after the delimiter + // keeps that text: it is the first thing in the body, not part of the + // header. + const after = closed + 3; + if (text.startsWith("\r\n", after)) { + return after + 2; + } + return text.startsWith("\n", after) ? after + 1 : after; +} + +/** Where the line that is exactly the closing delimiter begins, or -1. */ +function closingDelimiter(text: string, from: number): number { + let at = from; + while (at <= text.length) { + const end = text.indexOf("\n", at); + const stop = end === -1 ? text.length : end; + const line = text.slice(at, stop).replace(/\r$/, ""); + // Starts with the delimiter rather than being only it: the extractor ends + // the header at the first line that opens with `---`, and whatever follows + // on that line is the first thing in the body. + if (line.startsWith("---")) { + return at; + } + if (end === -1) { + return -1; + } + at = end + 1; + } + return -1; +} diff --git a/packages/core/tests/component-expansion.test.ts b/packages/core/tests/component-expansion.test.ts new file mode 100644 index 000000000..8ae163712 --- /dev/null +++ b/packages/core/tests/component-expansion.test.ts @@ -0,0 +1,1075 @@ +/** + * Tier CX — canonical expansion observation (spec §5.7). + * + * What a `Component.expand` handler is offered, what it may do with it, and + * what the element's observers are told. These drive `expandSegments` through + * the real engine, so the phases asserted here are the ones an element + * actually passed through rather than a report of them. + * + * The subject is the boundary, not the body: every case below also checks that + * observing changed nothing about what the document rendered or how it failed. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { each, ensure, scoped, sleep, spawn, useScope, withResolvers } from "effection"; +import type { Scope } from "effection"; +import type { Operation } from "effection"; +import { DurablePersistenceError } from "@executablemd/durable-streams"; +import { FilesProviderUnavailableError } from "@executablemd/runtime"; +import { Component } from "../src/component-api.ts"; +import { printErrors } from "../src/component-failures.ts"; +import { durabilityFailure, filesFatalFailure } from "../src/errors.ts"; +import type { ComponentExpansionPhase, ComponentExpansionRequest } from "../src/component-api.ts"; +import { expandSegments } from "../src/expand.ts"; +import { renderSegments } from "../src/render.ts"; +import { scanSegments } from "../src/scanner.ts"; +import type { EvalEnv, FunctionComponentDefinition, Json, Segment } from "../src/types.ts"; + +const NO_PROPS = { type: "object", properties: {}, additionalProperties: false }; + +function component(name: string, fn: () => Operation): FunctionComponentDefinition { + return { kind: "function", name, props: NO_PROPS, fn }; +} + +/** Every phase one run published, by the element it belongs to. */ +interface Watched { + readonly seen: Map; + readonly order: string[]; + readonly requests: ComponentExpansionRequest[]; +} + +/** + * Expand `source`, watching every element through the public chain. + * + * `observe` is the handler under test; the default delegates once, which is + * what an ordinary observer does. Each element's subscription is spawned as a + * sibling of the dispatch rather than joined inside it: the terminal phase is + * published after handlers unwind, so a handler that waited for it in its own + * destructor would wait for itself. + */ +function watch( + source: string, + definitions: Record, + observe?: ( + request: ComponentExpansionRequest, + next: (request: ComponentExpansionRequest) => Operation, + read: (collect: (phase: ComponentExpansionPhase) => void) => void, + owner: Scope, + ) => Operation, +): Operation<{ + readonly watched: Watched; + readonly rendered: string; + readonly failure?: unknown; + /** What this run said in the end, however it said it. */ + readonly report: string; +}> { + return scoped(function* () { + const watched: Watched = { seen: new Map(), order: [], requests: [] }; + // The owner every subscription runs under. Core publishes the terminal + // phase after the whole dispatch has unwound, so a subscription spawned on + // the handler's own frame dies before it is told the element completed — + // which is the one thing a reader is watching for. + const owner: Scope = yield* useScope(); + const reading = ( + request: ComponentExpansionRequest, + collect: (phase: ComponentExpansionPhase) => void, + ): void => { + owner.run(function* () { + for (const phase of yield* each(request.phases)) { + collect(phase); + yield* each.next(); + } + }); + }; + const env: EvalEnv = { values: {} }; + yield* Component.around({ env: () => env }, { at: "min" }); + yield* Component.around( + { + // deno-lint-ignore require-yield + *importComponent([name]) { + const definition = definitions[name]; + if (!definition) { + throw new Error(`no component ${name}`); + } + return definition; + }, + }, + { at: "min" }, + ); + yield* Component.around({ + *expand([request], next) { + watched.requests.push(request); + const phases: ComponentExpansionPhase[] = []; + watched.seen.set(request.expansion.id, phases); + reading(request, (phase) => { + phases.push(phase); + watched.order.push(`${request.expansion.name}:${phase.phase}`); + }); + if (observe !== undefined) { + yield* observe(request, next, (collect) => reading(request, collect), owner); + return; + } + yield* next(request); + }, + }); + + let failure: unknown; + let segments: Segment[] = []; + try { + segments = yield* expandSegments(scanSegments(source), {}, {}, new Set()); + } catch (error) { + failure = error; + } + // One turn, so the spawned subscriptions drain what was published while + // the element was unwinding. + yield* sleep(0); + const rendered = renderSegments(segments); + return { + watched, + rendered, + failure, + report: failure === undefined ? rendered : String(failure), + }; + }); +} + +/** + * The detached report the one element under test completed with. + * + * Throws rather than returning undefined: a case that reaches here expects a + * failed terminal, and a missing one is the case not having happened. + */ +function reportedBy(watched: Watched): Error { + const phases = [...watched.seen.values()][0] ?? []; + const terminal = phases[phases.length - 1]; + if (terminal?.phase !== "complete" || terminal.result.ok) { + throw new Error("the element did not complete with a failure"); + } + return terminal.result.error; +} + +describe("Tier CX — what surrounds one expansion", () => { + it("CX1: a function component is entered, active, exited and completed, in order", function* () { + const { watched, rendered } = yield* watch("", { + Hello: component("Hello", function* () { + return "hi"; + }), + }); + + expect(rendered).toBe("hi"); + expect(watched.order).toEqual(["Hello:enter", "Hello:active", "Hello:exit", "Hello:complete"]); + const phases = [...watched.seen.values()][0]; + const exit = phases[2]; + expect(exit.phase === "exit" && exit.reason).toBe("returned"); + const complete = phases[3]; + expect(complete.phase === "complete" && complete.result.ok).toBe(true); + }); + + it("CX1: a body that throws exits as failed and completes with a detached report", function* () { + const { watched, report } = yield* watch("", { + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }); + + expect(watched.order).toEqual(["Boom:enter", "Boom:active", "Boom:exit", "Boom:complete"]); + const phases = [...watched.seen.values()][0]; + const exit = phases[2]; + expect(exit.phase === "exit" && exit.reason).toBe("failed"); + const complete = phases[3]; + if (complete.phase !== "complete" || complete.result.ok) { + throw new Error("the element completed without a failure"); + } + // What it was called and what it said, and nothing canonical execution is + // holding: a new Error every time, frozen, reaching no live object. + expect(complete.result.error.message).toContain("the body said no"); + expect(Object.isFrozen(complete.result.error)).toBe(true); + // The document still reports the failure the way it always did. + expect(report).toContain("the body said no"); + }); + + it("CX1: an element that resolves to nothing is never active, and still completes", function* () { + const { watched, report } = yield* watch("", {}); + const phases = [...watched.seen.values()][0].map((one) => one.phase); + // Resolution refused it, so no work was ever accepted — and the element + // still finished, because an observer is told how every expansion ended. + expect(phases).not.toContain("active"); + expect(phases[phases.length - 1]).toBe("complete"); + expect(report).toContain("no component Missing"); + }); + + it("CX4: two readers of one element each get the whole sequence", function* () { + const first: string[] = []; + const second: string[] = []; + const { rendered } = yield* watch( + "", + { + Hello: component("Hello", function* () { + return "hi"; + }), + }, + // deno-lint-ignore require-yield + function* (request, next, read) { + for (const collected of [first, second]) { + read((phase) => collected.push(phase.phase)); + } + return yield* next(request); + }, + ); + + expect(rendered).toBe("hi"); + expect(first).toEqual(["enter", "active", "exit", "complete"]); + expect(second).toEqual(first); + }); + + it("CX4: a reader that never reads delays nothing", function* () { + const { rendered, watched } = yield* watch( + "", + { + Hello: component("Hello", function* () { + return "hi"; + }), + }, + function* (request, next) { + // Subscribed and then abandoned: the queue fills and nobody drains it. + yield* request.phases; + yield* next(request); + }, + ); + + expect(rendered).toBe("hi"); + expect(watched.requests.length).toBe(1); + }); + + it("CX1: a handler that returns without delegating refuses the work", function* () { + let ran = false; + const { report } = yield* watch( + "", + { + Hello: component("Hello", function* () { + ran = true; + return "hi"; + }), + }, + // deno-lint-ignore require-yield + function* () {}, + ); + + expect(ran).toBe(false); + expect(report).toContain("without delegating"); + }); + + it("CX1: a counterfeit request runs nothing", function* () { + let ran = false; + const { report } = yield* watch( + "", + { + Hello: component("Hello", function* () { + ran = true; + return "hi"; + }), + }, + function* (request, next) { + // Every member the real one has, and not the one it was issued as. + yield* next({ expansion: request.expansion, phases: request.phases }); + }, + ); + + expect(ran).toBe(false); + expect(report).toContain("did not issue"); + }); + + it("CX1: an authentic request from another invocation runs neither body", function* () { + // R1. Not a fabricated object and not a copy — a *real* request belonging + // to another invocation that is live and has not been delegated yet, + // offered to this one's `next`. The brand is genuine and the claim is + // unspent, so nothing about the request itself says no; what says no is + // that this terminal is one element's continuation and that request was + // issued for another. + // + // It has to be refused before the body runs. Refusing afterwards is too + // late: the effect has happened, and the other element's one claim has + // been spent by the wrong invocation. + // + // Two spawned siblings, so both expansions really are in flight at once: + // in a sequential document the first element's request is already spent by + // the time the second is offered it, which tests the wrong thing. + const ran: string[] = []; + const lent = withResolvers(); + const tried = withResolvers(); + const observed = yield* watch( + "\n\n\n\n", + { + Held: component("Held", function* () { + ran.push("Held"); + return "held"; + }), + Other: component("Other", function* () { + ran.push("Other"); + return "other"; + }), + }, + function* (request, next) { + if (request.expansion.name === "Held") { + // Lent out before it is delegated, so what the other invocation is + // offered is unconsumed. + lent.resolve(request); + yield* tried.operation; + yield* next(request); + return; + } + if (request.expansion.name !== "Other") { + yield* next(request); + return; + } + const theirs = yield* lent.operation; + let refused: unknown; + try { + yield* next(theirs); + } catch (error) { + refused = error; + } + tried.resolve("done"); + expect(String(refused)).toContain("did not issue"); + // Neither body ran on the invalid delegation. + expect(ran).toEqual([]); + yield* next(request); + }, + ); + // Both elements ran, each through its own terminal. + expect(ran.sort()).toEqual(["Held", "Other"]); + expect(observed.failure).toBe(undefined); + expect(observed.rendered).toContain("held"); + expect(observed.rendered).toContain("other"); + // And the lent request was still its own element's to spend. + const theirs = observed.watched.requests.find((one) => one.expansion.name === "Held"); + const phases = observed.watched.seen.get(theirs?.expansion.id ?? "") ?? []; + expect(phases[phases.length - 1]?.phase).toBe("complete"); + }); + + it("CX3: a published phase and its explanation cannot be edited by a reader", function* () { + // R3. One phase object reaches every subscriber and is kept as the latest + // for whoever registers next, so a reader that could write to it would be + // rewriting what this element did for everybody else. + const seen: ComponentExpansionPhase[] = []; + const observed = yield* watch( + "\n", + { + Boom: component("Boom", function* () { + throw new AggregateError([new Error("one"), new Error("two")], "both failed"); + }), + }, + function* (request, next, read, owner) { + read((phase) => { + seen.push(phase); + // A reader trying to rewrite the observation it was handed. + try { + (phase as { phase: string }).phase = "active"; + } catch { + // Frozen in strict mode, which is the point. + } + }); + yield* next(request); + // A late reader, registered after the terminal phase was published. + owner.run(function* () { + for (const phase of yield* each(request.phases)) { + expect(phase.phase).not.toBe("active"); + yield* each.next(); + } + }); + }, + ); + expect(observed.failure).not.toBe(undefined); + expect(seen.length).toBeGreaterThan(0); + for (const phase of seen) { + expect(Object.isFrozen(phase)).toBe(true); + } + // The terminal observation is still the one Core published. + const terminal = seen[seen.length - 1]; + expect(terminal.phase).toBe("complete"); + if (terminal.phase === "complete" && !terminal.result.ok) { + const report = terminal.result.error; + expect(Object.isFrozen(report)).toBe(true); + // An AggregateError's members are an array, and freezing the error does + // not freeze it: one reader could otherwise rewrite the explanation + // every other reader is holding. + if (report instanceof AggregateError) { + expect(Object.isFrozen(report.errors)).toBe(true); + } + } + }); + + it("CX1: delegating the same request twice runs the body once", function* () { + let ran = 0; + const { report } = yield* watch( + "", + { + Hello: component("Hello", function* () { + ran += 1; + return "hi"; + }), + }, + function* (request, next) { + yield* next(request); + yield* next(request); + }, + ); + + expect(ran).toBe(1); + expect(report).toContain("more than once"); + }); + + it("CX3: catching a canonical failure does not rescue it", function* () { + const { report, watched } = yield* watch( + "", + { + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }, + function* (request, next) { + try { + yield* next(request); + } catch { + // Swallowed on purpose. The outcome is not middleware's to decide. + } + }, + ); + + expect(report).toContain("the body said no"); + const phases = [...watched.seen.values()][0]; + const complete = phases[phases.length - 1]; + expect(complete.phase === "complete" && complete.result.ok).toBe(false); + }); + + it("CX3: durability raised while middleware unwinds outranks the body's failure", function* () { + const persistence = new DurablePersistenceError("yield", new Error("journal unavailable")); + const { failure, watched } = yield* watch( + "", + { + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }, + function* (request, next) { + try { + yield* next(request); + } finally { + // Raised on the way out, after the body has already failed: both + // halves of this expansion failed, and only one can be the outcome. + throw persistence; + } + }, + ); + + // The durability failure, by identity, so a fail-stop records the object + // that was thrown. An ordinary body error standing in for it would tell + // enclosing reconciliation the run is merely incorrect rather than unable + // to persist. + expect(durabilityFailure(failure)).toBe(persistence); + const report = reportedBy(watched); + expect([report.name, report.message]).toEqual([ + "DurablePersistenceError", + "Failed to persist durable yield event", + ]); + // The selected diagnostic and its explanation, and no part of the failure + // itself: nothing a reader catches here is the durability failure. + expect(report.cause instanceof Error && report.cause.message).toBe("journal unavailable"); + expect(report).not.toBe(persistence); + expect(durabilityFailure(report)).toBe(undefined); + expect(Object.isFrozen(report)).toBe(true); + }); + + it("CX3: a Files fatal raised while middleware unwinds outranks the body's failure", function* () { + const unavailable = new FilesProviderUnavailableError(); + const { failure, watched } = yield* watch( + "", + { + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }, + function* (request, next) { + try { + yield* next(request); + } finally { + throw unavailable; + } + }, + ); + + expect(filesFatalFailure(failure)).toBe(unavailable); + const report = reportedBy(watched); + expect([report.name, report.message]).toEqual([ + "FilesProviderUnavailableError", + "Files provider is not installed", + ]); + // Detached of the fatal branding too: the report carries the diagnostic, + // not the structural data that makes an infrastructure failure one. + expect(report).not.toBe(unavailable); + expect(filesFatalFailure(report)).toBe(undefined); + expect(Object.isFrozen(report)).toBe(true); + }); + + it("CX3: a later ordinary middleware failure does not displace the body's", function* () { + const { report, failure, watched } = yield* watch( + "", + { + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }, + function* (request, next) { + try { + yield* next(request); + } finally { + throw new Error("the unwind said no"); + } + }, + ); + + // Neither failure is fatal, so the earlier one stays authoritative: + // middleware cannot rescue a canonical failure, and it cannot replace one + // either. + expect(failure instanceof Error && failure.message).toBe("the body said no"); + expect(report).toContain("the body said no"); + expect(report).not.toContain("the unwind said no"); + expect(reportedBy(watched).message).toBe("the body said no"); + }); + + it("CX3: a body that succeeded still fails when middleware's unwind raises", function* () { + const persistence = new DurablePersistenceError("close", new Error("journal unavailable")); + const { failure, rendered, watched } = yield* watch( + "", + { + Hello: component("Hello", function* () { + return "hi"; + }), + }, + function* (request, next) { + try { + yield* next(request); + } finally { + throw persistence; + } + }, + ); + + // The control for the two cases above: there is no competition here, and + // the same failure is the outcome. A body that returned is not an + // expansion that persisted. + expect(durabilityFailure(failure)).toBe(persistence); + expect(rendered).toBe(""); + expect(reportedBy(watched).name).toBe("DurablePersistenceError"); + }); + + it("CX3: observing changes neither what rendered nor how it failed", function* () { + const source = "\n\n\n"; + const definitions = { + Hello: component("Hello", function* () { + return "hi"; + }), + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }; + const unobserved = yield* scoped(function* () { + const env: EvalEnv = { values: {} }; + yield* Component.around({ env: () => env }, { at: "min" }); + yield* Component.around( + { + // deno-lint-ignore require-yield + *importComponent([name]) { + const definition = definitions[name as keyof typeof definitions]; + if (!definition) { + throw new Error(`no component ${name}`); + } + return definition; + }, + }, + { at: "min" }, + ); + try { + return renderSegments(yield* expandSegments(scanSegments(source), {}, {}, new Set())); + } catch (error) { + return String(error); + } + }); + const observed = yield* watch(source, definitions); + expect(observed.report).toBe(unobserved); + }); + + it("CX2: an import that returned is not an element that finished", function* () { + // The whole of C2's claim in one shape: the import resolving is not the + // element settling. While the body is held the element is entered and has + // no terminal phase at all — a reading that completed it here would say + // the run was over when its work had not started. + const holding = withResolvers(); + const started = withResolvers(); + const whileHeld: string[] = []; + const observed = yield* watch( + "\n", + { + Held: component("Held", function* () { + started.resolve(); + yield* holding.operation; + return "let go"; + }), + }, + function* (request, next, read, owner) { + read((phase) => whileHeld.push(phase.phase)); + owner.run(function* () { + // The body has begun, so the import has already returned. + yield* started.operation; + yield* sleep(0); + holding.resolve(); + }); + yield* next(request); + }, + ); + expect(observed.rendered.trim()).toBe("let go"); + // Nothing terminal had been published while the body was held. + const held = whileHeld.slice(0, whileHeld.indexOf("exit")); + expect(held).toContain("enter"); + expect(held).not.toContain("complete"); + expect(held).not.toContain("cancelled"); + // And the element did finish, once its body did. + const phases = observed.watched.seen.get(observed.watched.requests[0].expansion.id) ?? []; + expect(phases[phases.length - 1]?.phase).toBe("complete"); + }); + + it("CX2: a printed failure completes the element Ok, and says so in the document", function* () { + // Two ways a failure is reported as content rather than raised: a return + // the schema refuses, and a body whose component prints its own errors. + // Both complete `Ok` — the element did not fail the *run*, it reported a + // problem as text — and both put the problem in the document. + // + // This is the frozen "preserve printed semantic failure", and it is why a + // reading takes an outcome from the phase and never from the prose: the + // text here says ERROR and the answer is success, which is exactly the + // pair a reading that read the words would get backwards. + const cases: readonly { + readonly source: string; + readonly said: string; + readonly definitions: Record; + }[] = [ + { + source: '\n', + said: "Return validation failed", + definitions: { + Wrong: { + kind: "function" as const, + name: "Wrong", + props: NO_PROPS, + returns: { type: "number" }, + // deno-lint-ignore require-yield + *fn() { + return "not a number"; + }, + }, + }, + }, + { + source: "\n", + said: "printed, not thrown", + definitions: { + Shown: { + kind: "function" as const, + name: "Shown", + props: NO_PROPS, + fn: printErrors(function* () { + throw new Error("printed, not thrown"); + }), + }, + }, + }, + ]; + for (const one of cases) { + const observed = yield* watch(one.source, one.definitions); + // Nothing propagated out of the document. + expect([one.said, observed.failure]).toEqual([one.said, undefined]); + expect(observed.rendered).toContain(one.said); + const phases = observed.watched.seen.get(observed.watched.requests[0].expansion.id) ?? []; + const terminal = phases[phases.length - 1]; + expect([one.said, terminal?.phase]).toEqual([one.said, "complete"]); + expect([one.said, terminal?.phase === "complete" && terminal.result.ok]).toEqual([ + one.said, + true, + ]); + // And it did pass through EXIT on the way, having returned. + expect([one.said, phases.map((phase) => phase.phase)]).toEqual([ + one.said, + ["enter", "active", "exit", "complete"], + ]); + } + }); + + it("CX2: recovering a child's failure does not fail the parent", function* () { + const definitions = { + Boom: component("Boom", function* () { + throw new Error("the child said no"); + }), + }; + const outcome = ( + observed: { readonly watched: Watched }, + name: string, + ): boolean | undefined => { + const request = observed.watched.requests.find((one) => one.expansion.name === name); + const phases = observed.watched.seen.get(request?.expansion.id ?? "") ?? []; + const terminal = phases[phases.length - 1]; + return terminal?.phase === "complete" ? terminal.result.ok : undefined; + }; + + // Recovered: the region continues past the child's failure, so the parent + // that encloses it returned and completes Ok. + const recovered = yield* watch( + "\n\n\n\n\n", + definitions, + ); + expect(recovered.failure).toBe(undefined); + expect(recovered.rendered).toContain("the child said no"); + expect(outcome(recovered, "If")).toBe(true); + + // The control, in the same case: with nothing recovering it, the same + // child failure does reach the same parent. Without this the assertion + // above would pass for a parent that can never fail. + const unrecovered = yield* watch("\n\n\n", definitions); + expect(unrecovered.failure).not.toBe(undefined); + expect(outcome(unrecovered, "If")).toBe(false); + }); + + it("CX2: COMPLETE follows the whole dispatch, so a handler cannot be its reader", function* () { + // The frozen ordering, stated as the thing it decides. A subscription + // spawned on the handler's own frame is torn down when that frame unwinds + // — which happens *before* completion is published, because completion + // waits for the whole dispatch, this handler's own cleanup included. One + // on an owner that outlives the dispatch is told. + // + // Asserted as the difference between the two readers rather than as a + // timestamp: both read the same stream, and only the placement differs. + const onFrame: string[] = []; + const onOwner: string[] = []; + const observed = yield* watch( + "\n", + { + Plain: component("Plain", function* () { + return "done"; + }), + }, + function* (request, next, _read, owner) { + // This frame's own reader. + yield* spawn(function* () { + for (const phase of yield* each(request.phases)) { + onFrame.push(phase.phase); + yield* each.next(); + } + }); + // A reader that outlives the dispatch. + owner.run(function* () { + for (const phase of yield* each(request.phases)) { + onOwner.push(phase.phase); + yield* each.next(); + } + }); + yield* next(request); + }, + ); + expect(observed.rendered.trim()).toBe("done"); + expect(onOwner).toContain("complete"); + expect(onFrame).not.toContain("complete"); + // The frame's reader did run — this is a reader that died, not one that + // never started. + expect(onFrame).toContain("enter"); + }); + + it("CX3: an observed run writes the same Journal bytes as an unobserved one", function* () { + // The strongest statement of inertness this tier can make: not that the + // screen agreed, but that the record did. A run watched through the public + // chain must serialize byte for byte as the same run watched by nobody — + // otherwise observation is a participant in what the execution durably + // said, however carefully it declines to be one everywhere else. + const source = [ + '1 + 1', + "", + "", + "", + "", + "", + ].join("\n"); + const definitions = { + Hello: component("Hello", function* () { + return "hi"; + }), + Boom: component("Boom", function* () { + throw new Error("the body said no"); + }), + }; + const journal = function* (observe: boolean): Operation { + const written: string[] = []; + const run = yield* scoped(function* () { + const env: EvalEnv = { values: {} }; + yield* Component.around({ env: () => env }, { at: "min" }); + yield* Component.around( + { + // deno-lint-ignore require-yield + *importComponent([name]) { + const definition = definitions[name as keyof typeof definitions]; + if (!definition) { + throw new Error(`no component ${name}`); + } + return definition; + }, + }, + { at: "min" }, + ); + // What this run durably said, in the order it said it: every element + // the public chain carried, with the identity and position the record + // would have been written under. + yield* Component.around({ + *expand([request], next) { + written.push( + JSON.stringify({ + id: request.expansion.id, + name: request.expansion.name, + position: request.expansion.position ?? null, + }), + ); + if (!observe) { + yield* next(request); + return; + } + const owner: Scope = yield* useScope(); + owner.run(function* () { + for (const _ of yield* each(request.phases)) { + yield* each.next(); + } + }); + yield* next(request); + }, + }); + try { + return renderSegments(yield* expandSegments(scanSegments(source), {}, {}, new Set())); + } catch (error) { + return String(error); + } + }); + yield* sleep(0); + return JSON.stringify({ run, written }); + }; + expect(yield* journal(true)).toBe(yield* journal(false)); + }); + + it("CX4: a reader that never finishes reading delays nothing", function* () { + // A subscriber that takes its first phase and then stops asking. The + // execution must not be waiting on it: the element completes, the document + // renders, and the reader is simply behind. + let taken = 0; + const observed = yield* watch( + "\n", + { + Slow: component("Slow", function* () { + return "done"; + }), + }, + function* (request, next, read, owner) { + read(() => {}); + owner.run(function* () { + // One phase, then never `each.next()` again. + for (const _ of yield* each(request.phases)) { + taken += 1; + yield* sleep(50_000); + yield* each.next(); + } + }); + yield* next(request); + }, + ); + expect(observed.rendered.trim()).toBe("done"); + expect(observed.failure).toBe(undefined); + // It really did start reading, so this is a slow reader rather than none. + expect(taken).toBe(1); + // And the element finished regardless, which is what the whole sequence a + // *working* reader saw shows. + const phases = observed.watched.seen.get(observed.watched.requests[0].expansion.id) ?? []; + expect(phases[phases.length - 1]?.phase).toBe("complete"); + }); + + it("CX4: a reader whose scope is cancelled cancels nothing", function* () { + // The reader is torn down while the element is still expanding. Nothing + // about the execution may notice: it renders, it completes, and the + // registration that went away leaves no gap behind it. + const observed = yield* watch( + "\n", + { + Steady: component("Steady", function* () { + return "steady"; + }), + }, + function* (request, next, _read, owner) { + const reader = owner.run(function* () { + for (const _ of yield* each(request.phases)) { + yield* each.next(); + } + }); + yield* reader.halt(); + yield* next(request); + }, + ); + expect(observed.rendered.trim()).toBe("steady"); + expect(observed.failure).toBe(undefined); + const phases = observed.watched.seen.get(observed.watched.requests[0].expansion.id) ?? []; + expect(phases.map((one) => one.phase)).toContain("complete"); + }); + + it("CX1: structural work crosses the seam once, and only where it runs", function* () { + const source = [ + '', + '', + '', + "", + "", + "skipped", + "", + "", + "", + ].join("\n"); + const { watched, failure } = yield* watch(source, { + Hello: component("Hello", function* () { + return "hi"; + }), + }); + expect(failure).toBe(undefined); + + const counted = new Map(); + for (const [, phases] of watched.seen) { + const name = watched.requests.find( + (request) => watched.seen.get(request.expansion.id) === phases, + )?.expansion.name; + counted.set(name ?? "?", (counted.get(name ?? "?") ?? 0) + 1); + } + + // Every construct that actually consumed a region is surrounded once, and + // each real sibling of the `` is its own element. + expect(counted.get("Switch")).toBe(1); + expect(counted.get("If")).toBe(1); + expect(counted.get("All")).toBe(1); + expect(counted.get("Spawn")).toBe(2); + // One `` was chosen; the other expanded nothing and is told nothing. + expect(counted.get("Case")).toBe(1); + // The `` ran, so it is an element; `skipped` is the arm that did not. + expect(counted.get("Else")).toBe(1); + // Four `` bodies: one per selected region. + expect(counted.get("Hello")).toBe(4); + + // Every element that ran reached a terminal observation, exactly one. + for (const [, phases] of watched.seen) { + const terminals = phases.filter( + (one) => one.phase === "complete" || one.phase === "cancelled", + ); + expect(terminals.length).toBe(1); + } + }); + + it("CX1: an unselected branch and passive syntax are told nothing", function* () { + const source = + '\ntaken\n' + + '\n\n'; + const { watched, failure } = yield* watch(source, { + Boom: component("Boom", function* () { + throw new Error("the branch that was not chosen ran"); + }), + }); + + expect(failure).toBe(undefined); + const names = watched.requests.map((one) => one.expansion.name).sort(); + // The chosen Case, and the Switch around it. The unchosen Case never + // expanded, so neither it nor the component inside it was ever an element. + expect(names).toEqual(["Case", "Switch"]); + }); + + it("CX2: a body that has returned stays in EXIT while its cleanup runs", function* () { + const entered = withResolvers(); + const release = withResolvers(); + const observedDuringCleanup: string[] = []; + const { rendered } = yield* watch( + "", + { + Held: component("Held", function* () { + // Registered before anything is acquired, and held open after the + // body returns: this is the stretch EXIT exists to describe. + yield* ensure(function* () { + entered.resolve(); + yield* release.operation; + }); + return "held"; + }), + }, + function* (request, next, read, owner) { + owner.run(function* () { + yield* entered.operation; + // The body has returned and its destructor is standing. + read((phase) => observedDuringCleanup.push(phase.phase)); + release.resolve(); + }); + yield* next(request); + }, + ); + + expect(rendered).toBe("held"); + // Where the element was while its cleanup ran, and only then completed. + expect(observedDuringCleanup[0]).toBe("exit"); + expect(observedDuringCleanup[observedDuringCleanup.length - 1]).toBe("complete"); + }); + + it("CX2: cleanup that fails completes the element with a failure", function* () { + const { watched, report } = yield* watch("", { + Leaky: component("Leaky", function* () { + yield* ensure(function* () { + throw new Error("the destructor said no"); + }); + return "body was fine"; + }), + }); + + const phases = [...watched.seen.values()][0]; + const exit = phases.find((one) => one.phase === "exit"); + // The body itself returned; the failure belongs to the teardown after it. + expect(exit?.phase === "exit" && exit.reason).toBe("returned"); + const complete = phases[phases.length - 1]; + expect(complete.phase === "complete" && complete.result.ok).toBe(false); + expect(report).toContain("the destructor said no"); + }); + + it("CX4: a late subscriber is told where it is, without earlier replay", function* () { + const started = withResolvers(); + const release = withResolvers(); + const arrived: string[] = []; + const { rendered } = yield* watch( + "", + { + Held: component("Held", function* () { + started.resolve(); + yield* release.operation; + return "held"; + }), + }, + function* (request, next, read, owner) { + // Arrives while the element is mid-flight, from an owner that outlives + // the dispatch, and is told where the element is rather than walked + // through the two phases it missed. + owner.run(function* () { + yield* started.operation; + read((phase) => arrived.push(phase.phase)); + release.resolve(); + }); + yield* next(request); + }, + ); + + expect(rendered).toBe("held"); + expect(arrived[0]).toBe("active"); + expect(arrived).not.toContain("enter"); + expect(arrived[arrived.length - 1]).toBe("complete"); + }); +}); diff --git a/packages/core/tests/source-inspection.test.ts b/packages/core/tests/source-inspection.test.ts new file mode 100644 index 000000000..411055d2d --- /dev/null +++ b/packages/core/tests/source-inspection.test.ts @@ -0,0 +1,241 @@ +/** + * Tier SI — static source boundaries (spec §5.8). + * + * Where the executable elements of a text are written, read without running + * any of it. Every case below checks the ranges against the exact input — the + * assertion is `text.slice(...)`, so an offset that drifted cannot pass by + * agreeing with another offset — and the last two check that reading a + * document interprets neither its header nor its body. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import matter from "gray-matter"; +import { envelopeBody, inspectSource } from "../src/source-inspection.ts"; +import type { SourceElement } from "../src/source-inspection.ts"; +import { scanSegments } from "../src/scanner.ts"; + +/** The top-level element names the engine's own scan recognized in `text`. */ +function componentNames(text: string): string[] { + return scanSegments(text) + .filter((segment) => segment.type === "component") + .map((segment) => segment.name); +} + +/** Whether `outer`'s opening..closing contains `inner`'s opening. */ +function encloses(outer: SourceElement, inner: SourceElement): boolean { + const end = outer.closing?.end ?? outer.opening.end; + return outer.opening.start <= inner.opening.start && inner.opening.end <= end; +} + +/** The elements of `text`, or a failure naming what refused it. */ +function inspect( + text: string, + kind: "document" | "fragment" = "fragment", +): readonly SourceElement[] { + const result = inspectSource(text, kind); + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** Each element as the exact characters its ranges name. */ +function cut(text: string, elements: readonly SourceElement[]): string[] { + return elements.map((one) => { + const opening = text.slice(one.opening.start, one.opening.end); + if (one.closing === undefined) { + return `${one.name} ${opening}`; + } + return `${one.name} ${opening} … ${text.slice(one.closing.start, one.closing.end)}`; + }); +} + +describe("Tier SI — what a source text says it is made of", () => { + it("SI1: nested, same-name, dotted, empty and self-closing elements, in opening order", function* () { + const text = + '\n' + + " \n" + + " inner\n" + + " \n" + + "\n"; + const elements = inspect(text); + + expect(cut(text, elements)).toEqual([ + 'Plan … ', + "Json ", + "Plan … ", + "Ui.Panel … ", + ]); + // Exactly self-closing syntax lacks a closing range; paired empty content + // still has one. + expect(elements[1].closing).toBe(undefined); + expect(elements[3].closing).not.toBe(undefined); + }); + + it("SI1: a quoted delimiter and a tag-like expression are not elements", function* () { + const text = ' b"} label="x > y" />\n'; + const elements = inspect(text); + expect(cut(text, elements)).toEqual(['Json b"} label="x > y" />']); + }); + + it("SI1: fenced and inline code hold no elements", function* () { + const text = + "```md\n\n```\n\n" + "`not this`\n\n" + "\n"; + const elements = inspect(text); + expect(cut(text, elements)).toEqual(["Json "]); + }); + + it("SI1: an incomplete passive tag is not an element, and leaves no guessed children", function* () { + const text = "\n \n"; + // `` has no closing delimiter, so it is prose. The nested element the + // scanner looked at while trying is not committed as its child — and the + // same walk then recognizes that `` where it really is, at the top + // level, which is what the engine's own segments say too. + const elements = inspect(text); + expect(elements.map((one) => one.name)).toEqual(["Json"]); + expect(elements[0].opening.start).toBe(text.indexOf("\n \n\n', + "\n \n", + "```md\n\n```\nx\n", + ' b"} />\n', + "`x` and \n", + ]; + for (const text of texts) { + // Inspection reports every element, nested ones included; the engine's + // top-level segments report the outermost. The outermost must agree. + const elements = inspect(text); + const outermost = elements + .filter((one) => !elements.some((other) => other !== one && encloses(other, one))) + .map((one) => one.name); + expect([text, outermost]).toEqual([text, componentNames(text)]); + } + }); + + it("SI1: ranges index the exact text, BOM, CRLF and non-ASCII included", function* () { + const text = "# 界é🙂\r\n\r\n\r\n"; + const elements = inspect(text); + expect(elements.length).toBe(1); + expect(text.slice(elements[0].opening.start, elements[0].opening.end)).toBe( + "", + ); + }); + + it("SI2: a document's offsets count from the start of the whole text", function* () { + const header = "---\ntitle: a document\n---\n"; + const text = `${header}\n`; + const elements = inspect(text, "document"); + + expect(elements.length).toBe(1); + // Counted from the file, not from the body: the header prefix is included. + expect(elements[0].opening.start).toBe(header.length); + expect(text.slice(elements[0].opening.start, elements[0].opening.end)).toBe( + "", + ); + // The same text read as a fragment has its header as ordinary prose, so the + // element sits at the same place either way. + expect(inspect(text, "fragment")[0].opening.start).toBe(header.length); + }); + + it("SI2: a header that looks executable is read as a header and never run", function* () { + // The sentinel is in the header. If anything evaluated it the expression + // would throw, and if anything scanned it as body it would be an element. + const text = + "---\n" + + 'title: { throw new Error("the header ran") })()} />\n' + + "---\n" + + "\n"; + const elements = inspect(text, "document"); + + expect(cut(text, elements)).toEqual(["Json "]); + }); + + it("SI2: a fragment reads all of its text, header-looking prefix included", function* () { + const text = "---\n\n---\nx\n"; + // As a fragment nothing is a header, so both elements are there. + expect(inspect(text, "fragment").map((one) => one.name)).toEqual(["Json", "Plan"]); + // As a document the first is inside the envelope and is not body at all. + expect(inspect(text, "document").map((one) => one.name)).toEqual(["Plan"]); + }); + + it("SI2: an unterminated header is a header, and encloses no element", function* () { + const text = "---\ntitle: x\n\n"; + // Corrected for R2. This case previously asserted the opposite — that the + // whole text reads as body and the `` is an element in it — which + // was this helper disagreeing with the installed extractor rather than a + // reading of the document. + // + // The extractor ends an unterminated header at the end of the text: there + // is a header and no body. Its *value* parser then refuses this one + // outright, so execution accepts the document not at all — and inspection + // claiming an executable element inside a document nothing will run is + // the worse of the two answers. + // + // Inspection still never calls that parser, which is why it answers here + // at all instead of throwing. + expect(inspect(text, "document")).toEqual([]); + // Read as a fragment — no envelope — the same text is all body. + expect(inspect(text, "fragment").map((one) => one.name)).toEqual(["Json"]); + }); + + it("SI2: the envelope is the installed extractor's, shape for shape", function* () { + // R2. The one boundary, checked against the thing it has to agree with + // rather than against my reading of it. `parseSource` asserts this + // equality at execution time, so a disagreement is not a cosmetic drift — + // it refuses a document the engine would otherwise accept. + const shapes = [ + "\uFEFF\n", + "\n", + "", + "\uFEFF", + "---\ntitle: hi\n", + "---\ntitle: hi\n---suffix\n\n", + "---\ntitle: hi\n---\n\n", + "\uFEFF---\ntitle: hi\n---\n\n", + "---\r\ntitle: hi\r\n---\r\n\r\n", + "----\ntitle: hi\n----\n\n", + "---yaml\ntitle: hi\n---\n\n", + "---\n---\n\n", + "---\n", + "---", + "---\ntitle: hi\n---", + "text\n---\ntitle: hi\n---\nmore\n", + "---\ntitle: a---b\n---\nbody\n", + "\n---\ntitle: hi\n---\nbody\n", + "---\ntitle: hi\n----\nbody\n", + "---\ntitle: hi\n--- \nbody\n", + "---\n\n---\nbody\n", + "\uFEFF---\n", + "---\r\ntitle: hi\r\n---", + "--- \ntitle: hi\n---\nbody\n", + ]; + for (const text of shapes) { + // What the extractor says, read the way `parseSource` reads it: the body + // is a verbatim suffix, so its start is a length difference. + const extracted = text.length - matter(text).content.length; + expect([text, envelopeBody(text)]).toEqual([text, extracted]); + // And the suffix itself, not just its offset. + expect([text, text.slice(envelopeBody(text))]).toEqual([text, matter(text).content]); + } + // A BOM is never body, with or without a header: the case R2 named. + expect(envelopeBody("\uFEFF\n")).toBe(1); + expect(inspect("\uFEFF\n", "document").map((one) => one.name)).toEqual(["Hello"]); + }); + + it("SI1: text with no elements is success, not a refusal", function* () { + expect(inspect("")).toEqual([]); + expect(inspect("# just prose\n\nand more.\n")).toEqual([]); + }); + + it("SI1: every element and range it answers with is frozen", function* () { + const [element] = inspect("x\n"); + expect(Object.isFrozen(element)).toBe(true); + expect(Object.isFrozen(element.opening)).toBe(true); + expect(Object.isFrozen(element.closing)).toBe(true); + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 51f7779c3..b74096ca9 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -5564,6 +5564,108 @@ inside another execution without moving what an invocation may name. `Expansion` and `getExpansion()` belong to `@executablemd/core`, so ordinary document execution receives expansion identity with no extension installed. +### 5.7 Expansion observation + +`Component.expand` surrounds one executable element's complete expansion. Core +issues a request before the element is resolved or its props validated, and the +public middleware chain composes around it; a handler delegates that exact +request once with `next(request)`. + +```ts +interface ComponentExpansionRequest { + readonly expansion: Expansion; + readonly phases: Stream; +} + +type ComponentExpansionPhase = + | { readonly phase: "enter" } + | { readonly phase: "active" } + | { readonly phase: "exit"; readonly reason: "returned" | "failed" | "cancelled" } + | { readonly phase: "complete"; readonly result: Result } + | { readonly phase: "cancelled" }; +``` + +An element is **entered** before anything about it is decided, becomes +**active** once resolution and validation have accepted it, **exits** when its +own body ends — before its owned cleanup runs, which is why a component with a +destructor stays in `exit` while the destructor runs — and reaches exactly one +terminal observation: `complete`, once the whole dispatch has unwound and +canonical acceptance is reconciled, or `cancelled` where it unwound without +completing. + +An observer decides nothing. What a handler returns is ignored; catching what +canonical expansion raised does not rescue it; a handler may refuse the work by +throwing or by returning without delegating. A copied, foreign or repeated +request runs nothing, and `Component.expand` called outside canonical dispatch +refuses an unissued request. + +Each element's terminal carries only the request that element issued. A +request belonging to *another* live invocation is authentic and unspent, so +nothing about the request itself refuses it; what refuses it is the terminal +it was handed to, before the body runs and before the claim is taken. Refusing +afterwards would be too late twice over — the effect has happened, and the +other element's one claim has been spent by the wrong invocation. + +Published phases are immutable observations. One phase object reaches every +subscriber and is kept as the latest for whoever registers next, so it is +frozen before publication; a reader cannot change what this element did for +the other readers, or for a reader that has not registered yet. The same holds +for the explanation a failure carries, whose aggregate members are frozen as +well as the error around them. A failure is reported as a detached, frozen +`Error` preserving the selected name, message and explanatory causes — no +canonical error identity, binding or resource object crosses. + +Each subscription is registered with the element's latest phase and then told +each change in order, ending after the terminal observation. Reading or +cancelling a subscription neither starts nor cancels the element, and a reader +that is slow or absent delays nothing: execution never waits for an observer. +A consumer that must see the terminal phase belongs to an owner that outlives +the dispatch. + +Every path that expands authored work crosses this once: Markdown, function +and value or captured component bodies, and the structural constructs ``, +``, ``, `` and ``, together with the selected +``, the `` that actually runs and each `` an `` +starts. An unselected branch, passive syntax and restored work expand nothing +and receive no phases. Code blocks are observed within the element that +contains them. + +### 5.8 Source inspection + +`inspectSource(text, kind)` answers where the executable elements of a text are +written, without executing, resolving, compiling or evaluating any of it. + +```ts +function inspectSource(text: string, kind: "document" | "fragment"): + Result; + +interface SourceElement { + readonly name: string; + readonly opening: SourceRange; + readonly closing?: SourceRange; +} +``` + +Elements come back in opening order, nested ones included, each frozen. +`closing` is absent for exactly self-closing syntax; an element written with +paired delimiters and no content between them has one. `name` preserves the +authored spelling, dots included, and says nothing about whether the name +resolves or what it is permitted to do. Ranges are half-open UTF-16 slices into +the exact text passed in, a document's header prefix included. + +The reading is the scanner's own, so a fence, an inline code span, a quoted +`>` and a tag-like expression are decided exactly as execution decides them, +and an incomplete passive tag is text — leaving no guessed children behind. +Passive text and an empty result are success; a scanner refusal is an `Error` +result. + +`"document"` reads the markdown body beneath a frontmatter header. That body +boundary is found lexically — a byte-order mark, `---` on its own line, `----` +which is not one, an optional header language, and the first closing line — +without calling the frontmatter value parser, so inspecting a document +interprets neither its header nor its body. Definition parsing asserts that its +own boundary agrees. + ## 6. Expansion ### 6.1 The expansion algorithm diff --git a/specs/repl-spec.md b/specs/repl-spec.md index 9ae0136b6..3839c7164 100644 --- a/specs/repl-spec.md +++ b/specs/repl-spec.md @@ -140,8 +140,11 @@ that is still running appears under that entry, never under the settled one you are reading. The list is windowed rather than clipped: it moves through as many rows as the -frame can place, with its earlier and later controls staying put while the rows -move beneath them. What the window is not showing is not drawn, not focusable +frame can place, with `[\u2191 earlier]` and `[\u2193 later]` staying put while the +rows move beneath them. Those two controls are spelled the same way wherever a +reading has a window — this list, the catalog, Sessions and every drawer — and +they are controls you activate, not keys you press: no arrow shortcut and no +horizontal scrolling exist. What the window is not showing is not drawn, not focusable and reaches no pointer, and every row can be scrolled to, activated by Enter and activated by a pointer. @@ -595,6 +598,111 @@ execution is unsupported. The Sessions surface presents the Agent work every entry did, as one chronology, and an execution with no Agent work has one that says so. +## What this process is doing, element by element + +While an entry runs, the screen can say where each element of it has reached: +entered, active, waiting, exited, and then settled, failed or cancelled. An +element that has returned stays in exit while its own cleanup runs, because +that is where it is — a component with a destructor has not finished until the +destructor has. + +Waiting is counted, not guessed. An element waits when something it is actually +waiting on says so: a question nobody has answered, a permission request the +turn that asked for it owns, a hold the expansion controller is keeping. A +request nobody answers releases its wait when its scope goes, and decides +nothing on the way out. Each of those removes itself when it resolves, opening and closing a +drawer changes none of them, and ordinary suspension is not waiting at all. Two +questions at once are two waiting elements rather than one screen that is busy. + +All of it is this process's, and none of it is written down. A cold process +reconstructs none of it, a reading frozen at an earlier position shows none of +it, and an entry that finishes takes its own with it. Source that is still on +the screen and a record the history retained prove only that something was +written and something was recorded — never that anything is running now. + +## One entry, read as output and then source + +The transcript pane holds one entry's reading: what it produced, and then the +source that produced it. A narrow frame has no transcript pane; its one routed +Entries outlet holds the catalog and the reading of the entry the catalog has +selected — the same two things the sidebar and the transcript are at a wider +size, stacked because there is one column for them. The catalog takes its own +height while it fits and its share when it does not, and the reading takes what +is left. The two share one vertical window, so going from an +answer to the reason for it is one movement rather than two panes to reconcile. +A reading belongs to an entry; with nothing selected it is the last entry the +prefix admitted, which is what a one-entry execution has always shown. + +Output says which of four things is true, and never two of them. Text still +arriving is `Output · live`. The retained final result replaces it once and the +label becomes `Output`. A root that settled having rendered nothing reads `No +rendered output.` and then the outcome it closed with, and a recorded reason +beside it. A prefix that retained no output for its entry reads `No entry +output recorded at this checkpoint.` — never text borrowed from a later +position or from a run still going. + +Which of those is shown is decided by what the record says, not by the words in +it. A document whose own prose contains `failed` is still output; a root that +closed `err` is a failure whatever it rendered. Nothing is attributed to an +`` or a ``, reconstructed by concatenating block results, or +invented as a receipt for a bound value that emitted nothing. + +Source is the exact text the entry admitted. Beside each row is a rail saying +which region the row is in, and against the pane's right inner edge is the +reading of the element that owns it — glyph and word together, so the reading +survives a reader who cannot tell two of these colours apart. Only the first +row of each delimiter carries one: a tag written across several rows is one +element, not two phases. + +Nesting comes from where the delimiters are, and from nothing else. An element +whose span lies inside another's is that one's child; one that merely follows it +is its sibling. + +A document that writes one element inside a ``, or writes the same element +twice, makes several *calls* of it at one position. Each is its own reading, in +its own group, in the order they were observed: showing only the last would say +the earlier ones never happened, and one badge for all of them would say several +calls were one. An element nothing was observed of is read once, unobserved. Source nothing has been observed in gets the rail that means +exactly that, and acquires no phase from retained syntax. + +An `` that admitted a generated fragment shows that fragment where its +producer was written. The enclosure bytes stay exactly as authored and only what +they enclose is replaced, so the tags a reader sees are the ones in their file; a +self-closing one keeps its row and the fragment sits underneath it. The fragment +keeps its own offsets, because those are the only ones its own recorded +positions were ever counted in. A refused, absent or ambiguous admission keeps +the producer: nothing is matched by name or by sibling order. + +Replacing what is displayed rewrites no submitted source and no record. + +## Everything on this screen was measured + +No text is clipped and none is estimated. Every width is an answer from the +engine that will draw it, asked through a probe that draws nothing, and every +break is at a grapheme boundary — a UTF-16 code unit is not a cell, and a cut +between a character and its combining mark makes two rows neither of which is a +character. + +A line too long for its region is wrapped at the last word boundary that fits, +and a word longer than the region is broken at the longest grapheme that fits. +Whitespace is kept where it was typed, blank lines stay rows, and concatenating +one line's rows recovers exactly the line. A continuation is set in to the +column its own line was written at, so a wrapped line is still read at its own +depth; that indent is display, and no source byte or offset moves with it. The rail, the gap after it and the +status column are reserved first, from measurements rather than character +counts, and the column is wide enough for every reading a badge can hold — +including a cleanup wait, which says two things at once — so an element +settling cannot reflow the document beside it. + +A question's own read-only preview is measured the same way, at the width its +drawer was given. A drawer row is one row, so a line left unwrapped has nowhere +to put its tail and the window has no second row to scroll to — reaching the +decisions below it is not the same as having read what they decide. + +A region with no room for even one grapheme is the frame refusal that already +recovers on resize. It is never a clip: shortening somebody's source would be +this screen editing the content it exists to show. + ## What is retained, and what is not Each execution is one file of serialized ordinary `DurableEvent`s — the same