Skip to content

feat(harness): add the scenario language, runner, and tessera run - #66

Merged
robfeldmann merged 52 commits into
mainfrom
phase-4/05-scenarios
Oct 25, 2026
Merged

robfeldmann merged 52 commits into
mainfrom
phase-4/05-scenarios

Conversation

@robfeldmann

@robfeldmann robfeldmann commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Plan 036 Slice 5, stacked on #65 (phase-4/04-driver).

What this adds

  • Examples/Scenarios/README.md, the scenario grammar's single source: app, size, and term headers; resize, key, type, paste, and bytes input; shot, tree, expect text, expect screen <<EOF, and expect node SEL frame; shell quoting, # comments, \xHH escapes; key names, sizes, locators, the term header, goldens, the report and exit codes, a "Not yet expressible" table, and the stability rules. A test holds the table to the parser's list.
  • A header-binding, fail-fast parser (FILE:LINE: error:; an unknown statement is a parse error; CRLF files and a leading BOM are read), a runner that continues after a failed expectation, and one ScenarioReport with a human form and a JSON form. --record, --repeat N, and --timing.
  • Public in TesseraHarness, and nothing else new: ScenarioRunner (init(registry:record:), run(_ paths: String...) async -> [ScenarioReport]), ScenarioReport (scenario, passed, failures, description, json), and ScenarioFailure (line, statement, verb, message).
  • HarnessCLI gains run PATH..., shot NAME [SIZE], tree NAME [SIZE], check, and replay, with real help COMMAND text, one JSON object per command when stdout is not a terminal, and exit codes 0 pass, 1 an expectation failed, 2 a usage or parse error, 3 a harness error. mise run dev:run, dev:shot, dev:tree, dev:check, dev:replay, and dev:help mirror them, each with #USAGE.
  • The headless device answers a session's colour queries (OSC 10, 11, 4, and DA1) from the term header (colors=, theme=light|dark), so a golden never depends on Ghostty's defaults (Decision 5).
  • The corpus: the four passing spike scenarios moved to Examples/Scenarios with git mv, a new counter-basics.scn, their text goldens, and the stack and workspace demos they run against. CI runs tessera run --pretty Examples/Scenarios on every leg of the test matrix (macOS, Linux, Windows, and the Swift 6.4 lanes), and .gitattributes checks goldens out with LF.
  • The testing-posture spec, the Developer tooling page, the layout spec, README, CONTRIBUTING, CHANGELOG, and the tessera-devtools skill describe what is built.

What it refuses, on purpose

The slice text lists dump, expect cell, expect cost, and check in the grammar and check and replay in the CLI, while Slice 6 builds their engines (the projection, cost, invariants, the trace). They are named and refuse: each is a parse error that says what it waits for (wait, settle, focus, mouse, hit, dump, check, expect cell|cost|focused|wireframe, expect node … visible|clipped|count, shot NAME SEL, tree SEL, term glyphs=|keyboard=|theme=probe), and tessera check and tessera replay exit 2 with "not yet available", because a check over no rules would pass without checking anything. Slice 6 replaces the refusals.

Verification

All on the baseline toolchain (Xcode 26.6) unless noted; the diff touches no Package.resolved.

  • mise run quality:format, mise run core:test (838 tests; 750 on the base), swift test --package-path Examples (25; 23 on the base), mise run quality:lint, mise run docs:lint: green.
  • Xcode 27.1 (--scratch-path, deleted after): the root suite (838), the Examples tests (25), and tessera run Examples/Scenarios pass.
  • CI on ef4592b (run 40074331235) and on 5b4dabf (run 40075060791): all 14 jobs green. Every test leg ran tessera run --pretty Examples/Scenarios and printed 5 scenarios, 5 passed, 0 failed, test (windows-latest) included, with its goldens checked out by Git for Windows' defaults. The first run (3faa2f4, 40073722211) failed only on Windows, in the README table test: a CRLF checkout of README.md hides \n from a Foundation search, fixed in ef4592b.
  • Two runs of tessera run Examples/Scenarios give identical bytes (the same shasum twice, 6,996 bytes of JSON); --repeat 3 passes the corpus.
  • A parse error under JSON is a failure with verb parse and exit 2; a drifted golden fails without --record with the first differing row and exit 1, and --record rewrites it and lists (updated).
  • The CI probe, locally: with counter-basics.scn's expectation broken, tessera run Examples/Scenarios exits 1 with a failure at line 19; reverted with git checkout, it exits 0. The reports are in the plan's Discoveries.
  • ScenarioCorpusTests runs the corpus through the public ScenarioRunner; the harness tests cover the parser, the player, discovery, goldens, the report shapes, repeat and timing, the answerer, and every command.

Critic review fixes

The seven findings of the review are fixed on top of 5b4dabf, each with a test where it is behavioural; the head's run is on the PR.

  • F1: a scenario ends when its app finishes, as a live run does. The statement that finishes the app is the last that runs and drops the rest of its input; the next statement fails with its own verb and a message naming the line that finished the app. An app already finished at the start fails its first statement. Documented in the README.
  • F2: shot names are unique ignoring case (a parse error naming both lines), because goldens are files and APFS and NTFS fold case.
  • F3: a golden with a carriage return is a CRLF failure before any row is compared, drifted or not.
  • F4: run --repeat PATH reports the --repeat value error.
  • F5: the README and the run help name every promised JSON field: recorded[].change, line 0 with an empty statement for file- and command-level failures, --repeat N as a repeat failure's statement, and the missing or CRLF golden shape (no expected, actual, or screen).
  • F6: the expect screen closing line may carry trailing spaces or tabs.
  • F7: check (and replay) validate their app name before they refuse.

Decisions to weigh

Each is recorded in the plan with its reasoning. In short: the refusals above; .scn files with goldens in a directory named like the file (counter-basics/reset.txt); exit codes derived from failure verbs and the JSON shapes as the 1.0 promise; term builds colors= and theme=light|dark only, and the headless session still detects passively, so nothing asks for colours until Slice 1 (#60) merges and Slice 8 reads them; the spike's stack and workspace became Tessera demos.

Not in this PR

Slice 6 (the projection, locators, invariants, cost, trace, grid shots) and Slice 1 (#60, the terminal's colour round).

…ort slice, and record the catalog capability audit

Record the owner's 2026-10-23 decisions in plan 036: Flow and ViewThatFits get their own slice before styling, the terminal colour-report probe becomes Slice 1 and runs in parallel, catalog requirements gain Focus, Pointer, Scroll, and Clock subsections that the capability slices work through, and the coverage walk reports levels, key-and-count routes, and stop reasons. Renumber the slices, add unattended-execution rules, update the developer tooling page, and add research record 043 with the audit's placements and drafted gap requirements for owner review.
Add the package-level seams Slice 2 of plan 036 lists, adapted from the seams
spike patch to today's sources, with no new public API:

- InMemoryTerminalDevice: package init(...keepsInputOpen:), deliverResize(to:),
  and closeInput(). An open device registers its byte and size-change streams
  when terminalDevice is read, so a resize delivered afterwards is never lost
  and late readers see no replayed history.
- InMemoryTerminalSession: package init(device:) to wrap an open device.
- TerminalSession: lastDrawn (buffer, cursor, sequence; set after a successful
  flush), size, and outputTap (teed before the draw's io.write);
  withApplicationTerminal(configuration:io:...) forwards probeImageID and
  probeSleep.
- ViewGraph: a package init taking the clock the passes read.

The seam tests live in TerminalSessionTests and ViewGraphClockTests.
Make TerminalSession.draw a nonisolated(nonsending) entry point, per the
runtime isolation design in the Phase 5 spec proposal: the frame body runs in
the caller's isolation, so a @mainactor host renders a non-Sendable graph
where it lives, and the session serializes draws and commits the finished
buffer on itself. sending is dropped from the body and the result.

The draw splits into beginFrame (size and rendering policy, fixed once),
the body in the caller's isolation, and commitFrame (encode, tee outputTap,
write, flush, then set lastDrawn after a successful flush). The seam
semantics are unchanged and nothing new is public.

A @mainactor test host draws through an InMemoryTerminalDevice session; the
same code did not compile against the sending signature.
The draw body now runs in the caller's isolation, so the scope that owns the
session can close between beginFrame and the commit. commitFrame checks that
the session is open before it encodes, taps, or writes, so a draw caught in
that window throws TerminalSessionError.closed instead of painting a frame onto
the restored terminal.

Document that deliverResize(to:) also emits where the public resize(to:) only
changes the reported size, amend the Phase 5 spec's draw transaction and plan
036's Slice 2 text to match, and narrow the Delivers line to the main-actor
host the test uses.
TesseraRuntime holds the app contract: Application refines only AnyObject, with body, apply(_:), and a defaulted isFinished, and its documentation carries the four adoption shapes. Tessera re-exports it, and TesseraRuntimeTests compiles each shape against import Tessera.

TesseraHarness is a library product beside TesseraTestSupport. It carries the HeadlessRunner spike's driver, scenario parser, runner, report, and registry with its scenario corpus and tests, every public declaration now package. The spike stays under spikes/ as frozen evidence.

Both targets get boundary allowlist rows and the shared swiftSettings, docs:targets and docs:merge know TesseraRuntime, and .gitattributes pins scenario files to LF.
Each view component ships its own demo instead of one monolithic app. Delete Examples/Sources/TesseraShowcase, its tests and Xcode schemes, the core:showcase task, and the manifest entries only the Showcase used, and point README.md, CONTRIBUTING.md, and docs/project-status.md at core:example.
…ement

Check the Slice 3 box with its evidence, move Next: to Slice 4, record the copy-not-move, access, dependency, shim, and .gitattributes decisions the owner may want to weigh, add the discoveries, and update the changelog.
# Conflicts:
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
…the unused CLI

Qualify the Application docs and the spec: leaving @mainactor off an extension conformance is a compile error from another module and in plain Swift 6 mode, but a module with default main-actor isolation or InferIsolatedConformances infers the isolated conformance for its own types.

Add Examples/Sources/ApplicationConsumer, a one-file target that imports only Tessera and declares an Application, so the Examples build fails if Tessera stops re-exporting TesseraRuntime. Remove the harness's CLI, which had no caller and no test; Slice 4 writes HarnessCLI fresh and the spike keeps the original.

Record in plan Slice 4 that its driver snapshot test restores the graph-to-session-to-bytes coverage the Showcase test gave, and note the three findings under Discoveries.
A scratch two-module SwiftPM package on Swift 6.3.3 and 6.4 settled when a bare extension conformance on a main-actor type compiles: it is an error in plain Swift 6 mode, builds with default main-actor isolation in the extension's module, and builds under InferIsolatedConformances (on 6.3.3 only for a type declared @mainactor, not one isolated through its module's default isolation). Rewrite the Application docs, the spec paragraph, and the plan's Discoveries and Slice 4 Verify to say so, run Slice 4's second fixture in plain Swift 6 mode, and describe ShotArtifact without the removed CLI.
…nd tessera launch

ApplicationRegistry and Viewport are the first public names in TesseraHarness. The registry
selects an app by name through a main-actor factory opened once per run, and a duplicate name
makes every read of it fail. `mobile` is 40 columns at the app's natural height, measured with a
package nil-height proposal on ViewGraph and two tall probe heights; an app whose root grows
with the proposal fails `size mobile` naming `mobileScreen`.

TesseraRuntime gains the package Step.run(_:graph:event:isolation:), and the harness gains the
@mainactor Driver over it, shared by a headless host (a session over a headless device into a
Ghostty virtual terminal), a live host, and a per-test actor host. HarnessCLI.main(registry:)
runs apps, launch NAME, and help. DemoSupport adds the Demo protocol, isQuitKey, and a counter
demo; Examples/Sources/TesseraDev builds it as the tessera executable, with mise run dev:apps
and dev:launch.

Two compile-fail fixtures with positive controls pin the Application isolation rules, run by
quality:architecture. ApplicationConsumer is deleted, since DemoSupport imports Tessera from
outside the root package. The tessera-devtools skill documents what is usable.
apps --json writes minimum: null (and a failure writes command: null) instead of omitting the
key. launch checks that stdin is a terminal before the lifecycle starts, so it exits 3 with a
clear message instead of failing inside raw mode. The registry's headless path is tested for
every adoption shape, including the main-actor app.

CI and ci:build-test now run swift test --package-path Examples. The step's doc and the spec
say a synchronous nonisolated caller cannot compile the call, and the spec's runtime loop
delivers the starting size as a resize, as the driver does. The docs call the driver and its
hosts internal, as the code has them.
Examples/Scenarios/README.md is the scenario grammar's single source: app, size,
and term headers; resize, key, type, paste, and bytes input; shot, tree, expect
text, expect screen, and expect node frame; shell quoting and # comments; and a
table of the forms it names but cannot honour yet, each a parse error that says
what it waits for. The parser is header-binding and fail-fast (FILE:LINE: error:),
reads CRLF files, and treats an unknown statement as an error. The runner
continues after a failed expectation; shot compares a text golden in a directory
named like the scenario and fails on drift unless --record writes it.

TesseraHarness gains ScenarioRunner, ScenarioReport, and ScenarioFailure as its
public run entry point, and HarnessCLI gains run (--record, --repeat N,
--timing), shot, tree, check, and replay, with real help text, one JSON object
per command off a terminal, and exit codes 0-3. check and replay refuse as not
yet available until the projection and the trace exist. The headless device
answers a session's colour queries from the term header. mise run dev:run,
dev:shot, dev:tree, dev:check, dev:replay, and dev:help mirror the commands.

The spike corpus moves to Examples/Scenarios with a counter scenario, the stack
and workspace demos it runs against, and its goldens, and CI runs tessera run on
every leg of the test matrix, Windows included, with goldens checked out as LF.
The testing-posture spec, the Developer tooling page, and the tessera-devtools
skill describe what is built.
…ut parses

A Windows checkout of README.md has CRLF line endings, and a "\r\n" is one
Character that a Foundation search for "\n" does not see inside, so the
table test found no section on the Windows leg. It now splits at the \n scalar,
as the parser does, and the Examples corpus test finds its directory by
splitting its own path at either separator.
A scenario ends when its app finishes, as a live run does: the statement that
finishes the app is the last that runs, and the next statement fails with its own
verb and a message naming the line that finished it. Shot names are unique
ignoring case, because goldens are files and APFS and NTFS fold case. A golden
with a carriage return fails as CRLF before any row is compared. run --repeat PATH
reports the --repeat value error. The expect screen closing line tolerates
trailing blanks. check and replay validate their app name before they refuse.
The README and the run help name every promised JSON field.
…ntime

# Conflicts:
#	CHANGELOG.md
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
Sort the root and Examples manifests alphabetically, state enduring facts in comments and DocC instead of plan numbers, conform Locator and FrameExpectation to CustomStringConvertible, say what Application's class-only contract means, express the scenario tests as snapshots, and record the owner's approval of the Slice 3 decisions.
… phase-4/04-driver

# Conflicts:
#	CHANGELOG.md
#	Package.swift
#	Sources/TesseraHarness/Driver.swift
#	Tests/TesseraHarnessTests/Fixtures.swift
#	Tests/TesseraHarnessTests/ScenarioTests.swift
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
Sort the conformance lists of RegistryError and StillLabel, express the driver, CLI, registry, viewport, and demo tests as inline snapshots and expectNoDifference where they check structured state, and give the DemoSupport tests the snapshot libraries they need.
…05-scenarios

# Conflicts:
#	Examples/Tests/DemoSupportTests/CounterDemoTests.swift
#	Sources/TesseraHarness/Scenarios/Scenario.swift
#	Tests/TesseraHarnessTests/HarnessCLITests.swift
#	Tests/TesseraHarnessTests/RegistryTests.swift
#	Tests/TesseraHarnessTests/ScenarioTests.swift
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
Express the scenario parser, player, runner, CLI, and corpus tests as inline snapshots and expectNoDifference where they check structured state, redact the scratch directory from file-path output so those snapshots are the same in every run, and cite the architecture page instead of a decision number in the CI comment.
A backslash in a dumped string makes the dump quote it differently, so a Windows scratch path redacted to text that no other platform produced; forward slashes keep the snapshots the same everywhere.
…arser

TesseraHarness alone depends on ArgumentParser, which parses the arguments, writes the help,
and completes app names for fish, zsh, and bash. HarnessCLI keeps the 1.0 promises it would
not: a usage error exits 2 where ArgumentParser exits 64, the JSON object and the text form
are chosen from the raw --json and --pretty so a failed parse is reported in the form asked
for, and a usage error lists what is accepted.
With no name, dev:launch prints the command's help and the registered apps and exits 0
instead of mise's missing-argument error, and dev:launch completes the app names from the
tessera binary a previous run built, printing nothing rather than building when there is none.
…05-scenarios

# Conflicts:
#	.agents/skills/tessera-contributing/SKILL.md
#	.agents/skills/tessera/SKILL.md
#	Sources/TesseraHarness/HarnessCLI.swift
#	Tests/TesseraHarnessTests/HarnessCLITests.swift
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
#	docs/spec/module-and-file-layout.md
…rser

The hand-rolled argument splitter and the help text tables are gone: ArgumentParser parses
the three commands' arguments and writes their help, and the run, shot, and tree results are
unchanged. A usage error still exits 2 and says what is accepted where it happened. A
repeated --repeat now takes the last value instead of failing, because ArgumentParser's
options have no exclusive mode.
A dev task ran `swift run` for each tessera call, so one that calls it twice printed two
SwiftPM build blocks. It now builds once with `swift build --quiet`, which prints nothing
when it succeeds and still prints the compiler's errors when it fails, and runs the built
binary for every call.
…05-scenarios

# Conflicts:
#	.agents/skills/tessera-contributing/SKILL.md
#	CHANGELOG.md
#	docs/architecture/developer-tooling.md
…pletion as the scripts ask

help takes one command: `help launch extra` is a usage error that lists the commands again,
and `help help` is pinned. The completion tests send the request the shipped scripts send
(positional@0 and the option's flag) instead of the legacy argument name, for every command
that takes an app name.
… terminal

A task run without its argument, or with an empty one, printed help and exited 0 even in a
script, where an unset variable then passed for success. It exits 0 at a terminal and 2 in a
pipe or a script, as tessera does.
…05-scenarios

# Conflicts:
#	.agents/skills/tessera-contributing/SKILL.md
#	CHANGELOG.md
#	Tests/TesseraHarnessTests/HarnessCLITests.swift
#	docs/architecture/developer-tooling.md
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
… completion

ArgumentParser takes the last value of a repeated option, so --repeat is declared as an array
and validate() rejects a second one with the message the command line always gave. The
completion tests also send the request the shipped scripts send for shot and tree.
…ts off a terminal, and complete dev:run with scenarios only
…rser parses

--repeat is a plain single-value option again, so usage reads `--repeat <n>` and an agent
does not take it for a repeatable one. HarnessCLI checks the raw arguments for each option a
command lists in singleValueOptions, as `--option value` or `--option=value` up to a `--`,
and rejects a second one with the usage error the command line always gave. A test compares
each command's declared options with ArgumentParser's own dump, so a new option that the
list misses fails it.
# Conflicts:
#	CHANGELOG.md
#	CONTRIBUTING.md
#	Examples/Package.swift
#	Package.swift
#	README.md
#	Sources/TesseraHarness/Driver.swift
#	Sources/TesseraHarness/Scenarios/Runner.swift
#	Sources/TesseraHarness/Scenarios/Scenario.swift
#	Sources/TesseraRuntime/Application.swift
#	Tests/TesseraHarnessTests/ScenarioTests.swift
#	docs/architecture/developer-tooling.md
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
#	scripts/check-package-boundaries.py
#	scripts/test_check_package_boundaries.py
Slice 4 deletes ApplicationConsumer and moves the harness's demos and registry; merging the squashed #64 re-added them.
Base automatically changed from phase-4/04-driver to main October 24, 2026 21:12
# Conflicts:
#	.agents/skills/tessera-contributing/SKILL.md
#	.agents/skills/tessera/SKILL.md
#	.github/workflows/ci.yml
#	CHANGELOG.md
#	CONTRIBUTING.md
#	Examples/Sources/DemoSupport/Demos.swift
#	Examples/Sources/DemoSupport/Workspace.swift
#	Examples/Tests/DemoSupportTests/CounterDemoTests.swift
#	README.md
#	Sources/TesseraHarness/ApplicationRegistry.swift
#	Sources/TesseraHarness/Demos.swift
#	Sources/TesseraHarness/HarnessCLI+Commands.swift
#	Sources/TesseraHarness/HarnessCLI.swift
#	Sources/TesseraHarness/HeadlessHost.swift
#	Sources/TesseraHarness/HeadlessTerminalDevice.swift
#	Sources/TesseraHarness/Scenarios/Runner.swift
#	Sources/TesseraHarness/Scenarios/Scenario.swift
#	Tests/TesseraHarnessTests/Fixtures.swift
#	Tests/TesseraHarnessTests/HarnessCLITests.swift
#	Tests/TesseraHarnessTests/RegistryTests.swift
#	Tests/TesseraHarnessTests/ScenarioTests.swift
#	docs/architecture/developer-tooling.md
#	docs/plans/phase-4-view-layer-and-developer-tooling.md
#	docs/spec/module-and-file-layout.md
#	mise-tasks/ci/build-test
@robfeldmann
robfeldmann marked this pull request as ready for review October 25, 2026 02:22
@robfeldmann
robfeldmann merged commit 3226c3b into main Oct 25, 2026
16 checks passed
@robfeldmann
robfeldmann deleted the phase-4/05-scenarios branch October 25, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant