diff --git a/architecture.md b/architecture.md index dde145584..680d8ecaa 100644 --- a/architecture.md +++ b/architecture.md @@ -1181,6 +1181,17 @@ capability, and the workflow package calls none of it. Ordinary `execute()` remains Git-independent, and so does starting, resuming, forking or exporting a run. +The same Api answers four **selected-checkout reads** — `files`, `changes`, +`diff` and `file` — which are what ``, ``, `` +and `` ask. They differ from the four questions above in what decides +which repository answers: each carries the `GitInvocationPlace` the component +observed, so the installed provider authenticates a Repository selection and +routes to a checkout rather than reading whatever directory the process is in. +Each returns a bounded page and a `next` holding the complete arguments of the +read that continues it. There is no host-less default: a host that installs no +selected-repository provider refuses, and nothing falls back to the caller's +own filesystem. + **The workflow package imports no Git feature, and a retained run reaches no repository either.** The CLI establishes a candidate from filesystem bytes, and the retained lifecycle re-derives the source from the run's own store, so a run @@ -1246,7 +1257,7 @@ it is doing rather than of the machinery behind it: | `Repository` | `executablemd.git.repository` | `select`, `worktree`, `ambient` | | `RepositoryContext` | `executablemd.git.repository.current` | `current` | | `Git` | `executablemd.git` | `switch`, `add`, `commit`, `push` | -| `GitQuery` | `executablemd.git.query` | `resolve`, `root`, `format`, `read` | +| `GitQuery` | `executablemd.git.query` | `resolve`, `root`, `format`, `read`, `files`, `changes`, `diff`, `file` | | `PullRequestAPI` | `executablemd.git.pull-request` | `read`, `upsert` | | `IssueApi` | `executablemd.git.issue` | `read`, `upsert` | | `IssueTrackerContext` | `executablemd.git.issue-tracker.current` | `current` | @@ -1255,9 +1266,13 @@ it is doing rather than of the machinery behind it: `Git` and `GitQuery` are two seams and never one. `Git` performs the four authored durable transitions a document writes as ``, ``, `` and ``, and what lifecycle they have is -the installed provider's. `GitQuery` asks read-only questions of the checkout -containing the contextual working directory and mutates nothing, so replacing -it answers questions rather than becoming a route to a transition. +the installed provider's. `GitQuery` asks read-only questions and mutates +nothing — four of the checkout containing the contextual working directory, and +four of the checkout a Repository selected — so replacing it answers questions +rather than becoming a route to a transition. A selected-checkout read reaches +the same provider instance the transitions do, at the same implementation slot, +and confers none of their evidence: nothing a read returns authorizes a +mutation, a publication or a pull request. These names and identities are a break from `@executablemd/workflow@0.12.1`, taken deliberately and without compatibility aliases. A provider compiled @@ -5161,9 +5176,10 @@ makes a run child of `xmd test` the run profile while the test root is not. #### Ordinary repository vocabulary -The bundled Git Plugin declares twelve ordinary shadowable defaults: +The bundled Git Plugin declares sixteen ordinary shadowable defaults: `Repository`, `Worktree`, `Git.Switch`, `Git.Add`, `Git.Commit`, `Git.Push`, -`PullRequest`, `PullRequest.Reviews`, `PullRequest.Comments`, +`Git.Files`, `Git.Changes`, `Git.Diff`, `Git.File`, `PullRequest`, +`PullRequest.Reviews`, `PullRequest.Comments`, `PullRequest.Checks`, `IssueTracker` and `Issue`. Dir is core-protected and remains available without this Plugin. One declaration array serves execution, syntax inspection, Plan validation/generation and executing workflow actions. @@ -8492,7 +8508,7 @@ Status is measured against main. | transactional Git effects (`Git.Switch` / `Git.Add` / `Git.Commit`) under a workflow run | publish local Git mutations with their journal result; the enclosing Repository and the contextual working directory select which retained checkout one runs in, and neither observation admits anything — the observed record is compared with the retained row and the directory with the checkouts that row holds, so a failure of admission, of retained state or of an unrecognized native condition fails the run instead of publishing a result [Invariant](#workspace-local-transactions); [Interface and evidence](specs/workflow-workspace-spec.md#7-git-operations). | built on the #294 stack, Deno provider only | | `Git.Push` under a workflow run | Publishes or adopts the selected named branch through authenticated non-force reconciliation. [Authored contract](specs/workflow-workspace-spec.md#74-push) and [effect mechanics](#replay-and-effect-transactions). | built on the #370 stack, Deno provider only | | `` under a workflow run | Upserts the exact published branch and returns stable evidence only after this run proves matching Push history. [Props, results and refusals](specs/workflow-workspace-spec.md#75-pull-request); [positive-integer number and reconciliation](#issue-effects). | built on the #295 stack, Deno provider only | -| repository composition vocabulary | Declares twelve ordinary shadowable defaults without installing an operational provider. [Names and profile ownership](#ordinary-repository-vocabulary). | built on the #643 stack | +| repository composition vocabulary | Declares sixteen ordinary shadowable defaults without installing an operational provider. [Names and profile ownership](#ordinary-repository-vocabulary). | built on the #643 stack | | `` / `` composition under an ordinary run | Selects an authenticated persistent checkout under an execution-owned exclusive lock [Full contract](#ordinary-repository-ownership). | built on the #643 stack, Deno and compiled only | | local Git operations (`Git.Switch` / `Git.Add` / `Git.Commit`) under an ordinary run | Performs explicit local transitions against the authenticated checkout without rollback or replay [Full contract](#ordinary-repository-ownership). | built on the #643 stack, Deno and compiled only | | `Git.Push` under an ordinary run | Publishes or adopts a non-force branch result and keeps invocation-private evidence [Full contract](#ordinary-repository-ownership). | built on the #643 stack, Deno and compiled only | diff --git a/packages/cli/src/documents/sidekick/entries.md b/packages/cli/src/documents/sidekick/entries.md index c97821c7d..a0812e38a 100644 --- a/packages/cli/src/documents/sidekick/entries.md +++ b/packages/cli/src/documents/sidekick/entries.md @@ -55,6 +55,39 @@ response. `select="code"` extracts the first matching fenced block without its fence, and trims trailing whitespace — pass a direct string value when exact string boundaries matter. +## Reading the repository in an entry + +An entry can ask the checkout it is standing in four questions. Read their exact +forms with `` +before writing one; each requires `as`, renders nothing, and binds JSON. + +````xmd + +```xmd + + +``` + +{code} + + +```` + +A page is bounded — 100 records, or 4096 characters of text — and `next` holds +the complete arguments of the read that continues it, or `null` when there is no +more. Follow it in a later entry by passing the captured value back, which is an +ordinary binding the entry before it published: + +```xmd + +``` + +If the answer changed in between, that read refuses and says to restart without +the cursor — ask for a fresh read at the beginning rather than treating the +refusal as an error to work around. Nothing these reads return lets an entry +change the repository; a change still takes ``, `` or +``. + ## Common mistake Writing the entry source directly into your response, or inside a fence marked diff --git a/packages/cli/tests/repl-git-observations.test.ts b/packages/cli/tests/repl-git-observations.test.ts new file mode 100644 index 000000000..2f06a7826 --- /dev/null +++ b/packages/cli/tests/repl-git-observations.test.ts @@ -0,0 +1,589 @@ +/** + * What a REPL entry's local Git reads retain, inherit and read again. + * + * The four reads are the only thing in this suite that is new; everything + * around them is the production REPL — a journal that is a file on disk, a + * session and a model this process builds from those bytes, the Git Plugin's + * own contribution for the `repl` command, and the per-entry recording core + * initialization establishes. What that combination has to do is answer a + * reopened entry from what it recorded, carry those answers to the entry after + * it, and read the repository again when a new entry asks. + * + * ## Why the counters are on the host + * + * "Answered from the record" is only worth asserting if the alternative was + * reachable. So the host beneath these entries counts every `git` invocation, + * every scratch directory and every authentication session, and the rows that + * expect nothing to have run assert on those rather than on the absence of a + * difference. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, resource, scoped, until } from "effection"; +import type { Operation, Result } from "effection"; +import { readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { appendFile, mkdtemp, open, realpath, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { API, useHostFiles } from "@executablemd/runtime"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { Execution } from "@executablemd/core/api"; +import type { ExecutionInstallation } from "@executablemd/core/host"; +import { denoRepositoryHost } from "../../git/src/deno/composition/host.ts"; +import type { + GitBytes, + GitInvocation, + GitOutcome, + RepositoryHost, +} from "../../git/src/deno/composition/host.ts"; +import { UNAUTHENTICATED } from "../../git/src/deno/composition/authentication.ts"; +import type { GitAuthenticationSession } from "../../git/src/deno/composition/authentication.ts"; +import { gitSession } from "../../git/src/deno/composition/git.ts"; +import { useReplRepositoryRecording } from "../../git/src/deno/retained-repository/provider.ts"; +import { gitPlugin } from "../../git/src/plugin.ts"; +import { REPL_REPOSITORY_OBSERVATION } from "../../git/src/retained-repository/observation-records.ts"; +import { ordinaryGeneratedSelection } from "../src/evaluation-profile.ts"; +import { ReplHost } from "../src/repl/host.ts"; +import { openReplSession, submitReplEntry } from "../src/repl/session.ts"; +import { entryInitialBindings } from "../src/repl/model.ts"; +import { + createExecutionFile, + executionPath, + openExecutionFile, + readRecords, +} from "../src/repl/journal.ts"; +import type { ReplSession } from "../src/repl/session.ts"; + +const ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789"; + +/** A synthetic GitHub token, format-realistic and assembled at run time. */ +const CANARY = ["ghp", "_", ALPHABET].join(""); + +interface Fixture { + readonly root: string; + readonly checkout: string; + readonly journals: string; + /** The first commit, which every pinned read names. */ + readonly first: string; + run(...args: string[]): Operation; +} + +/** + * A committed checkout an entry is opened in. + * + * Committed, because a Repository identity is pinned at the commit HEAD named + * when the entry opened: a checkout with no commit is one no selection can + * name, which is a different story from this one. + */ +function useFixture(): Operation { + return resource(function* (provide) { + const root = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "repl-git-reads-"))))); + yield* ensure(() => rm(root, { recursive: true, force: true })); + const checkout = join(root, "checkout"); + const git = gitSession(denoRepositoryHost(), root); + yield* git.run(["init", "-b", "main", checkout], root); + yield* git.run(["config", "user.name", "Fixture"], checkout); + yield* git.run(["config", "user.email", "fixture@example.invalid"], checkout); + yield* until(writeFile(join(checkout, "seed.md"), "seed\n")); + yield* until(writeFile(join(checkout, "notes.md"), "first notes\n")); + yield* git.run(["add", "--", "seed.md", "notes.md"], checkout); + yield* git.run(["commit", "-m", "first"], checkout); + const head = yield* git.read(["rev-parse", "HEAD"], checkout); + yield* provide({ + root, + checkout, + journals: join(root, "journals"), + first: head ?? "", + *run(...args: string[]): Operation { + const outcome = yield* git.run([...args], checkout); + if (outcome.code !== 0) { + throw new Error(`git ${args.join(" ")} exited ${outcome.code}: ${outcome.stderr}`); + } + return outcome.stdout.trim(); + }, + }); + }); +} + +interface Counted { + /** Every `git` invocation, as its argument list. */ + readonly commands: string[][]; + /** How many scratch directories the reads acquired. */ + directories: number; + /** Every authentication locator a session was opened for. */ + readonly sessions: string[]; +} + +function counters(): Counted { + return { commands: [], directories: 0, sessions: [] }; +} + +/** The native reads these entries performed, by exact subcommand. */ +const READS: readonly string[] = ["ls-files", "diff", "cat-file", "ls-tree"]; + +function reads(counted: Counted): string[] { + return counted.commands + .map((args) => args.find((arg) => READS.includes(arg))) + .filter((name): name is string => name !== undefined); +} + +/** The production host, counted at each leaf a read could reach. */ +function countingHost(counted: Counted): RepositoryHost { + const host = denoRepositoryHost(); + return { + *git(invocation: GitInvocation): Operation { + counted.commands.push([...invocation.args]); + return yield* host.git(invocation); + }, + *gitBytes(invocation: GitInvocation): Operation { + counted.commands.push([...invocation.args]); + if (host.gitBytes === undefined) { + throw new Error("the production host lends no byte capture"); + } + return yield* host.gitBytes(invocation); + }, + *useDirectory(): Operation { + counted.directories += 1; + return yield* host.useDirectory(); + }, + *useAuthentication(locator: string): Operation { + counted.sessions.push(locator); + return host.useAuthentication === undefined + ? UNAUTHENTICATED + : yield* host.useAuthentication(locator); + }, + }; +} + +/** + * What a command contributes to an entry's execution. + * + * Taken from the Git Plugin's own contribution for the `repl` command rather + * than written out here, because that contribution is the production path: the + * admission that reads this feature's records travels with the vocabulary. + */ +function* replInstallations(): Operation { + const install = gitPlugin.install; + if (install === undefined) { + throw new Error("the Git Plugin installs nothing"); + } + const installed = yield* install.call(gitPlugin, { command: "repl", args: [] }); + if (installed === undefined) { + throw new Error("the Git Plugin declared nothing for the repl command"); + } + return [{ admissions: [...(installed.admissions ?? [])] }]; +} + +/** The host half a runtime entrypoint owns, installed here by the suite. */ +function useNodeReplHost(name: string): Operation { + return ReplHost.around( + { + // deno-lint-ignore require-yield + *identify(): Operation { + return name; + }, + *createExclusive([path]: [string]): Operation { + const handle = yield* until(open(path, "wx")); + yield* until(handle.close()); + }, + *appendRecord([path, record]: [string, string]): Operation { + yield* until(appendFile(path, record)); + }, + }, + { at: "min" }, + ); +} + +function useWorkingDirectory(directory: string): Operation { + return API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return directory; + }, + }, + { at: "min" }, + ); +} + +function useCountedRecording(counted: Counted, root: string): Operation { + // The same seam the per-entry factory registers on, because what this suite + // is about is what an entry records rather than which host assembled it. + return Execution.around({ + *init(_args, next): Operation { + yield* useReplRepositoryRecording({ host: countingHost(counted), root }); + yield* next(); + }, + }); +} + +function opened(result: Result): ReplSession { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** Everything a row needs installed around one session. */ +function* prepare(fixture: Fixture, counted: Counted): Operation { + yield* useNodeReplHost("execution-1"); + yield* useHostFiles(); + yield* useWorkingDirectory(fixture.checkout); + yield* useCountedRecording(counted, join(fixture.root, "repositories")); +} + +/** Wait until this session is holding a question. */ +function* waitForQuestion(session: ReplSession): Operation { + const changes = yield* session.elicitation.changes; + if (session.elicitation.pending !== undefined) { + return; + } + yield* changes.next(); +} + +/** The four reads at root, each binding its page. */ +const READS_AT_ROOT = (fixture: Fixture): string => + [ + '', + "", + '', + "", + '', + "", + ``, + "", + "Read {files.paths.length} paths.", + "", + ].join("\n"); + +/** How many observation records this history holds. */ +function observationCount(records: readonly DurableEvent[]): number { + return records.filter( + (record) => record.type === "yield" && record.description.type === REPL_REPOSITORY_OBSERVATION, + ).length; +} + +describe("RGO1 — a reopened entry answers from what it recorded", () => { + it("carries all four pages through a cold reopen and into the entry after it", function* () { + const fixture = yield* useFixture(); + const live = counters(); + yield* writeTextFile(join(fixture.checkout, "notes.md"), "edited notes\n"); + + const first = yield* scoped(function* (): Operation<{ + bindings: Record; + path: string; + output: string; + }> { + yield* prepare(fixture, live); + const created = yield* createExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* submitReplEntry({ + execution: created, + source: READS_AT_ROOT(fixture), + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + const outcome = yield* session.join(); + if (!outcome.ok) { + throw outcome.error; + } + return { + bindings: entryInitialBindings(session.model), + path: executionPath(fixture.journals, "execution-1"), + output: session.model.entries[0]?.terminal?.output ?? "", + }; + }); + + expect(first.output).toContain("Read 2 paths."); + expect(reads(live)).toContain("ls-files"); + const recorded = yield* readRecords(first.path); + expect(observationCount(recorded)).toBe(4); + + // Everything the first entry read is now gone. + yield* rm(fixture.checkout, { recursive: true, force: true }); + + const cold = counters(); + const reopened = yield* scoped(function* (): Operation<{ + bindings: Record; + output: string; + }> { + yield* prepare(fixture, cold); + const file = yield* openExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* openReplSession({ + execution: file, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + // A settled history is read, never resumed. + expect(session.live).toBe(false); + const inherited = entryInitialBindings(session.model); + // Entry 2 inherits all four and renders selected members through Json. + const submitted = yield* session.submit( + [ + "Paths: {files.paths.length}", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"), + ); + if (!submitted.ok) { + throw submitted.error; + } + const outcome = yield* session.join(); + if (!outcome.ok) { + throw outcome.error; + } + return { + bindings: inherited, + output: session.model.entries[1]?.terminal?.output ?? "", + }; + }); + + // The pages a fresh session and a fresh model derived from the bytes on + // disk are the pages the first entry bound. + for (const name of ["files", "changes", "diff", "source"]) { + expect(`${name}: ${JSON.stringify(reopened.bindings[name])}`).toBe( + `${name}: ${JSON.stringify(first.bindings[name])}`, + ); + } + expect(reopened.output).toContain("Paths: 2"); + expect(reopened.output).toContain(`"${fixture.first}"`); + expect(reopened.output).toContain('"compare": "unstaged"'); + // And nothing live happened for any of it, with the checkout unavailable. + expect(reads(cold)).toEqual([]); + expect(cold.directories).toBe(0); + expect(cold.sessions).toEqual([]); + }); +}); + +describe("RGO2 — an unfinished entry restores its prefix and reads today", () => { + it("resumes after a question and sees data the saved pages never held", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + '', + "", + "Before: {before.paths.length}", + "", + 'Continue?', + "", + '', + "", + "After: {after.paths.length}, answered {answer.decision}.", + "", + ].join("\n"); + + yield* scoped(function* () { + yield* prepare(fixture, live); + const created = yield* createExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* submitReplEntry({ + execution: created, + source, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + yield* waitForQuestion(session); + }); + + const prefix = yield* readRecords(executionPath(fixture.journals, "execution-1")); + expect(observationCount(prefix)).toBe(1); + expect(prefix.some((event) => event.type === "close")).toBe(false); + + // New data behind the question: a tracked path the saved page never held. + yield* writeTextFile(join(fixture.checkout, "newer.md"), "newer\n"); + yield* fixture.run("add", "--", "newer.md"); + + const resumed = counters(); + const settled = yield* scoped(function* (): Operation { + yield* prepare(fixture, resumed); + const file = yield* openExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* openReplSession({ + execution: file, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + yield* waitForQuestion(session); + session.elicitation.pending?.submit({ decision: "go" }); + const outcome = yield* session.join(); + if (!outcome.ok) { + throw outcome.error; + } + return session.model.entries[0]?.terminal?.output ?? ""; + }); + + // The prefix answered from its record — one path, as it was — and the + // suffix read live and found two. + expect(settled).toContain("Before: 1"); + expect(settled).toContain("After: 2, answered go."); + // Exactly one fresh read for the suffix. + expect(reads(resumed)).toEqual(["ls-files"]); + const after = yield* readRecords(executionPath(fixture.journals, "execution-1")); + expect(observationCount(after)).toBe(2); + }); +}); + +describe("RGO3 — a recorded refusal is the same refusal next time", () => { + it("replays an ordinary bad-revision refusal with no git", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + "", + '', + "Source: {source.text}", + "", + "", + ].join("\n"); + + const first = yield* scoped(function* (): Operation { + yield* prepare(fixture, live); + const created = yield* createExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* submitReplEntry({ + execution: created, + source, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + const outcome = yield* session.join(); + if (!outcome.ok) { + throw outcome.error; + } + return session.model.entries[0]?.terminal?.output ?? ""; + }); + + expect(first).toContain("names no commit in the selected repository"); + const records = yield* readRecords(executionPath(fixture.journals, "execution-1")); + expect(observationCount(records)).toBe(1); + + const replayed = counters(); + const again = yield* scoped(function* (): Operation { + yield* prepare(fixture, replayed); + const file = yield* openExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* openReplSession({ + execution: file, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + return session.model.entries[0]?.terminal?.output ?? ""; + }); + + expect(again).toBe(first); + expect(reads(replayed)).toEqual([]); + }); +}); + +describe("RGO4 — two identical entries are two questions", () => { + it("reads the repository again rather than answering from the entry before", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + const source = [ + '', + "", + "Paths: {files.paths.length}", + "", + ].join("\n"); + + const outputs = yield* scoped(function* (): Operation { + yield* prepare(fixture, counted); + const created = yield* createExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* submitReplEntry({ + execution: created, + source, + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + const firstOutcome = yield* session.join(); + if (!firstOutcome.ok) { + throw firstOutcome.error; + } + // Between the two identical entries, the repository changes. + yield* writeTextFile(join(fixture.checkout, "newer.md"), "newer\n"); + yield* fixture.run("add", "--", "newer.md"); + const submitted = yield* session.submit(source); + if (!submitted.ok) { + throw submitted.error; + } + const secondOutcome = yield* session.join(); + if (!secondOutcome.ok) { + throw secondOutcome.error; + } + return [ + session.model.entries[0]?.terminal?.output ?? "", + session.model.entries[1]?.terminal?.output ?? "", + ]; + }); + + // The same source twice, and two different answers: each entry executes + // against its own segment, so the second one read rather than replayed. + expect(outputs[0]).toContain("Paths: 1"); + expect(outputs[1]).toContain("Paths: 2"); + expect(reads(counted).filter((name) => name === "ls-files")).toHaveLength(2); + const records = yield* readRecords(executionPath(fixture.journals, "execution-1")); + expect(observationCount(records)).toBe(2); + }); +}); + +describe("RGO5 — the secret gate still decides what reaches the journal", () => { + it("keeps a canary the repository holds out of the journal, fatally", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + // The canary is in the repository, not in the document: what a read carries + // is host data, and the gate has to hold that to the same rule as a + // document's own bytes. + yield* writeTextFile(join(fixture.checkout, "token.md"), `token ${CANARY}\n`); + yield* fixture.run("add", "--", "token.md"); + yield* fixture.run("commit", "-m", "token"); + const tainted = yield* fixture.run("rev-parse", "HEAD"); + + const reported = yield* scoped(function* (): Operation { + yield* prepare(fixture, counted); + const created = yield* createExecutionFile(fixture.journals, "execution-1"); + const session = opened( + yield* submitReplEntry({ + execution: created, + source: [ + ``, + "", + "Read {source.length} characters.", + "", + ].join("\n"), + installations: yield* replInstallations(), + generated: ordinaryGeneratedSelection(), + }), + ); + const outcome = yield* session.join(); + return outcome.ok + ? (session.model.entries[0]?.terminal?.output ?? "") + : String(outcome.error); + }); + + // The read really happened, so the gate had something to refuse. + expect(reads(counted)).toContain("cat-file"); + // And nothing it carried reached the file on disk. + const journal = yield* readTextFile(executionPath(fixture.journals, "execution-1")); + expect(journal).not.toContain(CANARY); + expect(reported).not.toContain(CANARY); + // The existing classification: a detected secret ends the entry rather + // than becoming a value a printing region could carry on past. + expect(reported).toContain("secret"); + const records = yield* readRecords(executionPath(fixture.journals, "execution-1")); + expect(observationCount(records)).toBe(0); + }); +}); diff --git a/packages/cli/tests/run-composition.test.ts b/packages/cli/tests/run-composition.test.ts index 373a182ba..00864c686 100644 --- a/packages/cli/tests/run-composition.test.ts +++ b/packages/cli/tests/run-composition.test.ts @@ -4,11 +4,11 @@ * Three requirements, and they are about opposite things. Describing the vocabulary * must reach nothing; one runtime *operates* it; and one only *describes* it. * All three have to be true at once, so a document written for `xmd run` - * resolves the same twelve names everywhere, and on a runtime that operates + * resolves the same sixteen names everywhere, and on a runtime that operates * none of them every one reports an absent provider before a lock, a * credential, a subprocess or a request exists. `` is canonical core's * rather than the Plugin's, so it is here for the opposite reason: it keeps - * working where every one of those twelve refuses. + * working where every one of those sixteen refuses. * * The declarations are the same array in every case, which is why there is no * fourth thing to keep in agreement. @@ -47,6 +47,10 @@ const OPERATIONS: readonly { readonly name: string; readonly source: string }[] { name: "Git.Add", source: `` }, { name: "Git.Commit", source: `` }, { name: "Git.Push", source: `` }, + { name: "Git.Files", source: `` }, + { name: "Git.Changes", source: `` }, + { name: "Git.Diff", source: `` }, + { name: "Git.File", source: `` }, { name: "PullRequest", source: `` }, { name: "PullRequest.Reviews", @@ -63,7 +67,11 @@ const OPERATIONS: readonly { readonly name: string; readonly source: string }[] { name: "Issue", source: `` }, ]; -/** The twelve names #643 settled that need a provider, as a document writes them. */ +/** + * The sixteen names that need a provider, as a document writes them. + * + * #643 settled twelve; #940 added the four reads beside the four transitions. + */ const COMPOSITION_NAMES = [ "Repository", "Worktree", @@ -71,6 +79,10 @@ const COMPOSITION_NAMES = [ "Git.Add", "Git.Commit", "Git.Push", + "Git.Files", + "Git.Changes", + "Git.Diff", + "Git.File", "PullRequest", "PullRequest.Reviews", "PullRequest.Comments", @@ -193,11 +205,11 @@ describe("ORC1 — describing the vocabulary reaches nothing", () => { }); describe("ORC2 — one language, described everywhere and operated somewhere", () => { - it("registers the same twelve declarations the syntax catalog describes", function* () { + it("registers the same sixteen declarations the syntax catalog describes", function* () { // The array itself, rather than a second list: the Plugin's own // registration and the catalog both consume this one, so there is nothing // for a runtime to disagree about. - expect(COMPOSITION_REGISTRATIONS).toHaveLength(12); + expect(COMPOSITION_REGISTRATIONS).toHaveLength(16); expect([...COMPOSITION_REGISTRATIONS].map((registration) => registration.name).sort()).toEqual( [...COMPOSITION_NAMES].sort(), ); @@ -222,7 +234,7 @@ describe("ORC2 — one language, described everywhere and operated somewhere", ( // refusal below would then be about something else. const failure = yield* raisedValue(ordinaryWithoutProvider(operation.source, ".")); // The element's name travels with the assertion, so a failure says which - // of the twelve reported something else. + // element reported something else. const reported = `${operation.name}: ${String(failure)}`; expect(reported).toMatch( /provider is not installed|no Repository composition provider|no Git composition provider|no issue provider|no Issue provider|no pull-request provider/, diff --git a/packages/cli/tests/syntax-cli.test.ts b/packages/cli/tests/syntax-cli.test.ts index 8717fd31a..f84098074 100644 --- a/packages/cli/tests/syntax-cli.test.ts +++ b/packages/cli/tests/syntax-cli.test.ts @@ -139,7 +139,7 @@ function symbolsWith(props: PropsSchema): SyntaxSymbols { }; } -/** The thirteen names #643 settled, exactly as a document writes them. */ +/** The repository vocabulary, exactly as a document writes it. */ const COMPOSITION_NAMES = [ "Repository", "Worktree", @@ -147,6 +147,10 @@ const COMPOSITION_NAMES = [ "Git.Add", "Git.Commit", "Git.Push", + "Git.Files", + "Git.Changes", + "Git.Diff", + "Git.File", "PullRequest", "PullRequest.Reviews", "PullRequest.Comments", diff --git a/packages/git/api.ts b/packages/git/api.ts index 1ee5802fa..27fb093e7 100644 --- a/packages/git/api.ts +++ b/packages/git/api.ts @@ -18,9 +18,10 @@ * * Each name says what its operations do. `Repository` selects a checkout, * `Git` performs the four authored durable transitions behind ``, and - * `GitQuery` asks read-only questions of the checkout the contextual working - * directory is in. The last two are never the same seam: replacing `GitQuery` - * answers questions and moves no branch. + * `GitQuery` asks read-only questions — four of the checkout the contextual + * working directory is in, and four of the checkout a Repository selected. The + * last two are never the same seam: replacing `GitQuery` answers questions and + * moves no branch. * * ```ts * import { GitQuery, Repository } from "@executablemd/git/api"; @@ -48,17 +49,51 @@ * question from what the question is. */ -// The read-only Git questions, and the four operations pre-bound for callers +// The read-only Git questions, and the eight operations pre-bound for callers // who want the operation rather than the seam. export { gitObjectFormat, GitQuery, gitRoot, + readGitChanges, + readGitDiff, + readGitFile, + readGitFiles, readGitObject, resolveGitRevision, } from "./src/git.ts"; export type { GitObjectFormat, GitQueryApi } from "./src/git.ts"; +// What the four selected-checkout reads ask for and answer with, and the +// continuation that carries one page to the next. +export type { + GitChange, + GitChangesInvocation, + GitChangesNext, + GitChangesRequest, + GitChangesResult, + GitChangeStatus, + GitCompareMode, + GitComparison, + GitComparisonRequest, + GitDiffInvocation, + GitDiffNext, + GitDiffRequest, + GitDiffResult, + GitFileInvocation, + GitFileNext, + GitFileRequest, + GitFileResult, + GitFilesInvocation, + GitFilesNext, + GitFilesRequest, + GitFilesResult, + GitObservationInvocation, + GitObservationKind, + GitObservationRequest, + GitObservationValue, +} from "./src/composition/observation-api.ts"; + // Repository selection: what a `` or `` selects, and the // credential-free selection it hands back. export { Repository } from "./src/composition/api.ts"; diff --git a/packages/git/src/composition/components.md b/packages/git/src/composition/components.md index aff8bdfd1..7800b7533 100644 --- a/packages/git/src/composition/components.md +++ b/packages/git/src/composition/components.md @@ -1,6 +1,6 @@ Long-form documentation for the repository-composition components. -Twelve components for working with repositories, branches, pull requests and +Sixteen components for working with repositories, branches, pull requests and issues from inside a document. They are mostly *regions*: `` and `` establish where the work happens, and the `Git.*`, `PullRequest.*` and issue components act inside whatever those established. @@ -94,6 +94,79 @@ Self-closing. Pushes the branch the working tree is on, to the remote the repository was cloned from — so which branch is decided by `` above it rather than repeated here. +## Git.Files + +Lists the tracked files of the repository in scope. + +```mdx + +``` + +`as` is required: this is a read, and the paths are the result. Only tracked +files are listed — an untracked file, and anything inside a nested or sibling +checkout, is not part of this repository's index and does not appear. `prefix` +is literal text, so `packages/co` lists everything whose path starts that way +and a prefix containing `*` matches a path with a `*` in its name. + +A page holds 100 paths unless you ask for more, and `next` holds the complete +arguments of the read that continues it — pass them straight back. When the +listing has changed in between, the continuation says so and asks you to start +again rather than stitching two different answers together. + +## Git.Changes + +Reports what changed, path by path. + +```mdx + + +``` + +Three comparisons: `unstaged` is the index against the working tree, `staged` is +HEAD against the index, and `commits` compares the two commits `base` and `head` +name directly — not their merge base. Each change carries Git's own letter for +what happened to it, and a rename carries the path it came from, so a document +can tell a move from a delete and an add. + +`paths` narrows the result to literal files and directories. A rename matches on +either side, and `paths={[]}` selects nothing at all — which is different from +leaving it out, and means it. + +## Git.Diff + +Reads the patch one comparison produces. + +```mdx + +``` + +The same three comparisons as ``, answering with unified diff text +instead of a list. A window holds 4096 characters unless you ask for more, and +`length` is how long the whole patch is, so a document can tell how much it has. +Binary changes keep Git's binary marker and a mode change shows as patch +metadata: this is the patch, not a summary of it. + +Use it to show a reviewer what a comparison contains. For deciding *which* +files to look at, `` answers that with a list rather than text to +search. + +## Git.File + +Reads a file as one commit holds it. + +```mdx + +``` + +The commit's own bytes, not the working copy — which is what lets a document +compare what shipped with what is there now, after somebody has already edited +the file. The result names the full commit id the revision resolved to, and a +continuation stays on that exact commit even if the branch moves under it. + +A window holds 4096 characters unless you ask for more, counted so that a +character is never split across two reads. A path the commit does not hold, a +directory, and a file that is not text each refuse and say which it was. + ## PullRequest Opens a pull request, or updates the one already open. diff --git a/packages/git/src/composition/components/GitReads.ts b/packages/git/src/composition/components/GitReads.ts new file mode 100644 index 000000000..61de3ba0e --- /dev/null +++ b/packages/git/src/composition/components/GitReads.ts @@ -0,0 +1,146 @@ +/** + * Read the selected repository: its tracked files, what changed, the patch, and + * one file as a commit holds it (specs/workflow-workspace-spec.md §7.8). + * + * ```mdx + * + * + * + * + * ``` + * + * All four are self-closing value components that bind JSON and render nothing. + * Each observes the two things a document can see — the Repository in scope and + * the contextual working directory — and asks whoever is installed to read. The + * provider decides which checkout, if any, those two select; neither the + * selection nor a `cursor` a document is holding grants access to one. + * + * `as` is required by an explicit `returns` declaration, which is checked before + * the body runs: an invocation that forgot it is refused before a provider is + * asked anything. Props are closed and admitted here, so a zero limit, a + * fractional offset, an absolute path or a revision pair on a live comparison is + * refused before any Git work. + * + * Self-closing form is declared to canonical dispatch, so neither replaceable + * `hasContent()` nor a caller-made invocation object turns a paired element into + * a read. + */ + +import { cwd } from "@executablemd/runtime"; +import type { FormDeclaration, InvocationForm } from "@executablemd/core"; +import type { Operation } from "effection"; +import type { Json } from "@executablemd/durable-streams"; +import { GitQuery } from "../../git.ts"; +import { selectedRepository } from "../context.ts"; +import { GitObservationError, GitOperationAdmissionError } from "../errors.ts"; +import { + admitChangesRequest, + admitDiffRequest, + admitFileRequest, + admitFilesRequest, +} from "../observation-api.ts"; +import type { GitObservationRequest } from "../observation-api.ts"; +import { + gitChangesResultJson, + gitDiffResultJson, + gitFileResultJson, + gitFilesResultJson, +} from "../observation-records.ts"; + +/** The component names, as a document writes them and a refusal names them. */ +export const FILES_ELEMENT = ""; +export const CHANGES_ELEMENT = ""; +export const DIFF_ELEMENT = ""; +export const FILE_ELEMENT = ""; + +function* Files(props: Record): Operation { + const request = admitFilesRequest(FILES_ELEMENT, props); + const repository = yield* selectedRepository(); + if (repository === undefined) { + throw outside(FILES_ELEMENT); + } + return gitFilesResultJson( + yield* GitQuery.operations.files({ + repository, + workingDirectory: yield* cwd(), + request, + }), + ); +} + +function* Changes(props: Record): Operation { + const request = admitChangesRequest(CHANGES_ELEMENT, props); + const repository = yield* selectedRepository(); + if (repository === undefined) { + throw outside(CHANGES_ELEMENT); + } + return gitChangesResultJson( + yield* GitQuery.operations.changes({ + repository, + workingDirectory: yield* cwd(), + request, + }), + ); +} + +function* Diff(props: Record): Operation { + const request = admitDiffRequest(DIFF_ELEMENT, props); + const repository = yield* selectedRepository(); + if (repository === undefined) { + throw outside(DIFF_ELEMENT); + } + return gitDiffResultJson( + yield* GitQuery.operations.diff({ + repository, + workingDirectory: yield* cwd(), + request, + }), + ); +} + +function* File(props: Record): Operation { + const request = admitFileRequest(FILE_ELEMENT, props); + const repository = yield* selectedRepository(); + if (repository === undefined) { + throw outside(FILE_ELEMENT); + } + return gitFileResultJson( + yield* GitQuery.operations.file({ + repository, + workingDirectory: yield* cwd(), + request, + }), + ); +} + +function outside(element: string): Error { + return new GitOperationAdmissionError( + element, + "it is written outside a lexical , so there is no repository in scope to read", + ); +} + +function declare( + element: string, + fn: (props: Record) => Operation, +): FormDeclaration { + return { + forms: "self-closing", + fn, + refuse: (_props: Record, written: InvocationForm | undefined) => + new GitObservationError( + element, + "unexpected-content", + written === "paired" + ? `${element} reads the repository and renders nothing of its own, so it is written ` + + "self-closing." + : `${element} was called without the invocation the engine issued, so which form it ` + + "was written as cannot be established.", + ), + }; +} + +export const filesForm: FormDeclaration = declare(FILES_ELEMENT, Files); +export const changesForm: FormDeclaration = declare(CHANGES_ELEMENT, Changes); +export const diffForm: FormDeclaration = declare(DIFF_ELEMENT, Diff); +export const fileForm: FormDeclaration = declare(FILE_ELEMENT, File); diff --git a/packages/git/src/composition/errors.ts b/packages/git/src/composition/errors.ts index 384853cae..b252b6f26 100644 --- a/packages/git/src/composition/errors.ts +++ b/packages/git/src/composition/errors.ts @@ -344,3 +344,60 @@ export class GitOperationProtocolError extends WorkflowStorageError { ); } } + +/** + * A word from the fixed vocabulary a local Git observation refuses under. + * + * Every one of them is a condition the document can act on by writing + * something else: a revision that names no commit, a path the commit does not + * hold, a blob that is not text, a repository this host could not read, a + * continuation whose observation has since changed. What a read never reports + * is what Git printed — a refusal names the condition and the correction, so a + * document cannot tell which Git version answered. + * + * `invalid-invocation` and `unexpected-content` are decided before a provider + * is reached and are therefore never retained: no observation happened. + */ +export type GitObservationReason = + | "invalid-invocation" + | "unexpected-content" + | "invalid-revision" + | "absent-path" + | "nontext-blob" + | "unusable-repository" + | "stale-continuation"; + +/** + * An observation the document asked for and did not get. + * + * An ordinary refusal rather than a stale-state failure: the read changed + * nothing, so a `` region may print it and carry on. What a + * continuation retains is the word and the sentence, so a replayed refusal is + * the same refusal the live read raised. + */ +export class GitObservationError extends Error { + override name = "GitObservationError"; + + /** The component that was written, as a document writes it. */ + readonly element: string; + readonly reason: GitObservationReason; + + constructor(element: string, reason: GitObservationReason, sentence: string) { + super(sentence); + this.element = element; + this.reason = reason; + } +} + +/** No provider answers local Git observations in this scope. */ +export class GitObservationProviderError extends WorkflowStorageError { + override name = "GitObservationProviderError"; + + constructor(element: string) { + super( + `no provider answers local Git observations, so ${element} cannot read anything. A host ` + + "that selects repositories on a native checkout installs one; a host with no native " + + "checkout to read has none, and nothing here falls back to the caller's own filesystem.", + ); + } +} diff --git a/packages/git/src/composition/installation.ts b/packages/git/src/composition/installation.ts index 8e7b30796..609ca2712 100644 --- a/packages/git/src/composition/installation.ts +++ b/packages/git/src/composition/installation.ts @@ -1,9 +1,10 @@ /** - * The twelve repository-composition components, as one array of ordinary + * The sixteen repository-composition components, as one array of ordinary * declarations. * - * Repository, Worktree, the four Git operations, PullRequest and its three - * evidence reads, IssueTracker and Issue are ordinary registered defaults — not + * Repository, Worktree, the four Git operations, the four Git reads, + * PullRequest and its three evidence reads, IssueTracker and Issue are + * ordinary registered defaults — not * reserved and not structural — so a repository-local component may shadow one * for its own scope. `` is not among them: canonical core reserves that * name (`core/src/components/fixed-filesystem.ts`), and a document reaches it @@ -67,6 +68,17 @@ import { reviewsReturns, } from "./components/PullRequestReads.ts"; import IssueTracker, { props as issueTrackerProps } from "./components/IssueTracker.ts"; +import { changesForm, diffForm, fileForm, filesForm } from "./components/GitReads.ts"; +import { + changesProps, + changesReturns, + diffProps, + diffReturns, + fileProps, + fileReturns, + filesProps, + filesReturns, +} from "./observation-api.ts"; /** * This boundary's long-form documentation, and the components it must cover. @@ -152,6 +164,54 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ fn: GitPush, ...documented(gitPushDocumentation), }, + { + name: "Git.Files", + origin: COMPOSITION_ORIGIN, + props: filesProps, + returns: filesReturns, + fn: formDispatcher(filesForm), + ...documented({ + description: `List tracked files. \`\` lists the selected repository's tracked paths, bounded and continuable.`, + as: "Required. The tracked paths, and the arguments of the next page or null.", + context: null, + }), + }, + { + name: "Git.Changes", + origin: COMPOSITION_ORIGIN, + props: changesProps, + returns: changesReturns, + fn: formDispatcher(changesForm), + ...documented({ + description: `Compare what changed. \`\` reports each changed path and its status for unstaged, staged or two commits.`, + as: "Required. The comparison, each change's path, status and previous path, and the next page or null.", + context: null, + }), + }, + { + name: "Git.Diff", + origin: COMPOSITION_ORIGIN, + props: diffProps, + returns: diffReturns, + fn: formDispatcher(diffForm), + ...documented({ + description: `Read a comparison's patch. \`\` reads a window of the unified diff text.`, + as: "Required. The comparison, the patch window, its offset, the whole length, and the next page or null.", + context: null, + }), + }, + { + name: "Git.File", + origin: COMPOSITION_ORIGIN, + props: fileProps, + returns: fileReturns, + fn: formDispatcher(fileForm), + ...documented({ + description: `Read a committed file. \`\` reads the text that commit holds, not the working copy.`, + as: "Required. The resolved commit, the path, the text window, its offset, the whole length, and the next page or null.", + context: null, + }), + }, { name: "PullRequest", origin: COMPOSITION_ORIGIN, diff --git a/packages/git/src/composition/observation-api.ts b/packages/git/src/composition/observation-api.ts new file mode 100644 index 000000000..bfeb025b1 --- /dev/null +++ b/packages/git/src/composition/observation-api.ts @@ -0,0 +1,766 @@ +/** + * The four local Git observations: what a document asks for, and what comes + * back. + * + * ``, ``, `` and `` are read-only + * questions about one selected checkout, and every one of them binds JSON. The + * types here are the whole contract: a normalized request, a bounded page, and + * a `next` object holding the complete arguments of the follow-up read. There + * is no handle, no live cursor object and no field a provider may add. + * + * ## A page is bounded, and the bound is the caller's + * + * A list answers 100 records and a text window 4096 code points unless the + * caller asks for more. Nothing caps what it may ask for beyond what a number + * can represent, because the default exists to bound what a document binds and + * a journal retains rather than to hide what Git had to read. + * + * Windows count Unicode code points, as the REPL's own text windows do, so a + * character is never split across two pages — and a surrogate pair is one + * character here even though it is two JavaScript code units. + * + * ## Filters are literal + * + * A prefix is string-prefix matching and a path filter is a repository-relative + * file or directory. Neither is a pathspec, a glob or anything a shell would + * expand: `*`, `?` and `[` match those characters. That is what lets a document + * name a path containing one without quoting, and what stops a filter from + * selecting more than it says. + */ + +import type { JsonObject, PropsSchema, ReturnsSchema } from "@executablemd/core"; +import type { Json } from "@executablemd/durable-streams"; +import type { GitInvocationPlace } from "./git-api.ts"; +import { GitObservationError } from "./errors.ts"; +import { wellFormedText } from "./parse.ts"; + +/** Which observation a request, a record and a cursor are about. */ +export type GitObservationKind = "files" | "changes" | "diff" | "file"; + +/** The records a list answers with when the caller names no limit. */ +export const DEFAULT_RECORD_LIMIT = 100; + +/** The code points a text window answers with when the caller names no limit. */ +export const DEFAULT_TEXT_LIMIT = 4096; + +/** The three comparisons `` and `` accept. */ +export type GitCompareMode = "unstaged" | "staged" | "commits"; + +export const GIT_COMPARE_MODES: readonly GitCompareMode[] = ["unstaged", "staged", "commits"]; + +/** + * What a document asked to compare, before anything was resolved. + * + * The revisions are the strings the caller wrote. Resolving one is live work + * against a repository, and what the document asked is not. + */ +export type GitComparisonRequest = + | { readonly compare: "unstaged" } + | { readonly compare: "staged" } + | { readonly compare: "commits"; readonly base: string; readonly head: string }; + +/** + * What one observation actually compared. + * + * `staged` carries the HEAD commit it read, or `null` on an unborn branch, + * which is ordinary Git behavior rather than a refusal. `commits` carries both + * full object ids, so a `next` page cannot be retargeted by a branch that moved. + */ +export type GitComparison = + | { readonly compare: "unstaged" } + | { readonly compare: "staged"; readonly base: string | null } + | { readonly compare: "commits"; readonly base: string; readonly head: string }; + +/** Git's own letter for what happened to one path. */ +export type GitChangeStatus = "A" | "M" | "D" | "R" | "T" | "U"; + +export const GIT_CHANGE_STATUSES: readonly GitChangeStatus[] = ["A", "M", "D", "R", "T", "U"]; + +/** One changed path, with the path it came from when it was renamed. */ +export interface GitChange { + readonly path: string; + readonly status: GitChangeStatus; + /** The path a rename came from, and `null` for every other status. */ + readonly previousPath: string | null; +} + +/** What one `` read asks for, normalized. */ +export interface GitFilesRequest { + readonly kind: "files"; + /** The literal path prefix, or `null` for the whole selected repository. */ + readonly prefix: string | null; + readonly limit: number; + /** The page this read continues, or `null` for a fresh observation. */ + readonly cursor: string | null; +} + +/** What one `` read asks for, normalized. */ +export interface GitChangesRequest { + readonly kind: "changes"; + readonly comparison: GitComparisonRequest; + /** The literal filters, or `null` when the read names none. */ + readonly paths: readonly string[] | null; + readonly limit: number; + readonly cursor: string | null; +} + +/** What one `` read asks for, normalized. */ +export interface GitDiffRequest { + readonly kind: "diff"; + readonly comparison: GitComparisonRequest; + readonly paths: readonly string[] | null; + readonly offset: number; + readonly limit: number; + readonly cursor: string | null; +} + +/** What one `` read asks for, normalized. */ +export interface GitFileRequest { + readonly kind: "file"; + /** The revision the caller wrote, which a fresh read resolves once. */ + readonly commit: string; + readonly path: string; + readonly offset: number; + readonly limit: number; +} + +export type GitObservationRequest = + | GitFilesRequest + | GitChangesRequest + | GitDiffRequest + | GitFileRequest; + +/** The complete arguments of one follow-up `` read. */ +export interface GitFilesNext { + readonly prefix?: string; + readonly limit: number; + readonly cursor: string; +} + +/** The complete arguments of one follow-up `` read. */ +export interface GitChangesNext { + readonly compare: GitCompareMode; + readonly base?: string; + readonly head?: string; + readonly paths?: readonly string[]; + readonly limit: number; + readonly cursor: string; +} + +/** The complete arguments of one follow-up `` read. */ +export interface GitDiffNext { + readonly compare: GitCompareMode; + readonly base?: string; + readonly head?: string; + readonly paths?: readonly string[]; + readonly offset: number; + readonly limit: number; + readonly cursor: string; +} + +/** + * The complete arguments of one follow-up `` read. + * + * No cursor, because the input is already immutable: the commit is resolved and + * the blob it names cannot change. + */ +export interface GitFileNext { + readonly commit: string; + readonly path: string; + readonly offset: number; + readonly limit: number; +} + +export interface GitFilesResult { + readonly paths: readonly string[]; + readonly next: GitFilesNext | null; +} + +export interface GitChangesResult { + readonly comparison: GitComparison; + readonly changes: readonly GitChange[]; + readonly next: GitChangesNext | null; +} + +export interface GitDiffResult { + readonly comparison: GitComparison; + readonly text: string; + readonly offset: number; + /** The whole observation's length in code points, not this window's. */ + readonly length: number; + readonly next: GitDiffNext | null; +} + +export interface GitFileResult { + /** The full object id the requested revision resolved to. */ + readonly commit: string; + /** The repository-relative path exactly as it was admitted. */ + readonly path: string; + readonly text: string; + readonly offset: number; + readonly length: number; + readonly next: GitFileNext | null; +} + +export type GitObservationValue = GitFilesResult | GitChangesResult | GitDiffResult | GitFileResult; + +export interface GitFilesInvocation extends GitInvocationPlace { + readonly request: GitFilesRequest; +} + +export interface GitChangesInvocation extends GitInvocationPlace { + readonly request: GitChangesRequest; +} + +export interface GitDiffInvocation extends GitInvocationPlace { + readonly request: GitDiffRequest; +} + +export interface GitFileInvocation extends GitInvocationPlace { + readonly request: GitFileRequest; +} + +export type GitObservationInvocation = + | GitFilesInvocation + | GitChangesInvocation + | GitDiffInvocation + | GitFileInvocation; + +function closed(properties: JsonObject): JsonObject { + return { + type: "object", + properties, + required: Object.keys(properties), + additionalProperties: false, + }; +} + +const LIMIT: JsonObject = { type: "integer", minimum: 1 }; +const OFFSET: JsonObject = { type: "integer", minimum: 0 }; +const PATH_FILTERS: JsonObject = { type: "array", items: { type: "string" } }; +const CURSOR: JsonObject = { type: "string", minLength: 1 }; + +export const filesProps: PropsSchema = { + type: "object", + properties: { + prefix: { type: "string" }, + limit: LIMIT, + cursor: CURSOR, + }, + required: [], + additionalProperties: false, +}; + +export const changesProps: PropsSchema = { + type: "object", + properties: { + compare: { enum: [...GIT_COMPARE_MODES] }, + base: { type: "string", minLength: 1 }, + head: { type: "string", minLength: 1 }, + paths: PATH_FILTERS, + limit: LIMIT, + cursor: CURSOR, + }, + required: ["compare"], + additionalProperties: false, +}; + +export const diffProps: PropsSchema = { + type: "object", + properties: { + compare: { enum: [...GIT_COMPARE_MODES] }, + base: { type: "string", minLength: 1 }, + head: { type: "string", minLength: 1 }, + paths: PATH_FILTERS, + offset: OFFSET, + limit: LIMIT, + cursor: CURSOR, + }, + required: ["compare"], + additionalProperties: false, +}; + +export const fileProps: PropsSchema = { + type: "object", + properties: { + commit: { type: "string", minLength: 1 }, + path: { type: "string", minLength: 1 }, + offset: OFFSET, + limit: LIMIT, + }, + required: ["commit", "path"], + additionalProperties: false, +}; + +const COMPARISON: JsonObject = { + oneOf: [ + closed({ compare: { const: "unstaged" } }), + closed({ compare: { const: "staged" }, base: { type: ["string", "null"] } }), + closed({ + compare: { const: "commits" }, + base: { type: "string" }, + head: { type: "string" }, + }), + ], +}; + +/** + * The follow-up arguments a `next` object may hold, exactly. + * + * Open-ended on which optional filters are present and closed on which members + * may exist at all, because a `next` repeats only the filters the read it + * follows actually had: an omitted prefix stays omitted rather than arriving as + * a null a later invocation would have to interpret. + */ +function nextSchema(properties: JsonObject, required: readonly string[]): JsonObject { + return { + type: ["object", "null"], + properties, + required: [...required], + additionalProperties: false, + }; +} + +export const filesReturns: ReturnsSchema = closed({ + paths: { type: "array", items: { type: "string" } }, + next: nextSchema({ prefix: { type: "string" }, limit: LIMIT, cursor: CURSOR }, [ + "limit", + "cursor", + ]), +}); + +const CHANGE: JsonObject = closed({ + path: { type: "string" }, + status: { enum: [...GIT_CHANGE_STATUSES] }, + previousPath: { type: ["string", "null"] }, +}); + +const COMPARISON_NEXT: JsonObject = { + compare: { enum: [...GIT_COMPARE_MODES] }, + base: { type: "string" }, + head: { type: "string" }, + paths: PATH_FILTERS, +}; + +export const changesReturns: ReturnsSchema = closed({ + comparison: COMPARISON, + changes: { type: "array", items: CHANGE }, + next: nextSchema({ ...COMPARISON_NEXT, limit: LIMIT, cursor: CURSOR }, [ + "compare", + "limit", + "cursor", + ]), +}); + +export const diffReturns: ReturnsSchema = closed({ + comparison: COMPARISON, + text: { type: "string" }, + offset: OFFSET, + length: OFFSET, + next: nextSchema({ ...COMPARISON_NEXT, offset: OFFSET, limit: LIMIT, cursor: CURSOR }, [ + "compare", + "offset", + "limit", + "cursor", + ]), +}); + +export const fileReturns: ReturnsSchema = closed({ + commit: { type: "string" }, + path: { type: "string" }, + text: { type: "string" }, + offset: OFFSET, + length: OFFSET, + next: nextSchema( + { commit: { type: "string" }, path: { type: "string" }, offset: OFFSET, limit: LIMIT }, + ["commit", "path", "offset", "limit"], + ), +}); + +function invalid(element: string, sentence: string): never { + throw new GitObservationError(element, "invalid-invocation", `${element} ${sentence}`); +} + +/** + * The limit this value is, or the read's own refusal. + * + * A positive safe integer and nothing else. A fraction, a negative, a zero and + * a number no integer arithmetic can represent are all refused here, before a + * provider is reached, because none of them describes a page a read could + * answer with. + */ +function admitLimit(element: string, value: Json | undefined, fallback: number): number { + if (value === undefined) { + return fallback; + } + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1) { + return invalid( + element, + "needs limit to be a whole number of at least 1 that integer arithmetic can represent.", + ); + } + return value; +} + +function admitOffset(element: string, value: Json | undefined): number { + if (value === undefined) { + return 0; + } + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 0) { + return invalid( + element, + "needs offset to be a whole number of at least 0 that integer arithmetic can represent.", + ); + } + return value; +} + +/** Whether this string holds a NUL, which no argument list carries. */ +function holdsNul(value: string): boolean { + for (let index = 0; index < value.length; index += 1) { + if (value.charCodeAt(index) === 0) { + return true; + } + } + return false; +} + +/** + * The repository-relative path this string names, or the read's own refusal. + * + * POSIX spelling, and ordinary segments only: nothing absolute, no NUL, no + * empty interior segment, no `.` and no `..`. Spaces, tabs, newlines and + * Unicode are ordinary filename characters and pass through untrimmed and + * unnormalized — what a repository holds is what a read has to be able to name. + * + * A backslash is an ordinary character too, on a host whose filenames may hold + * one. It is never read as a separator, so no spelling of it reaches outside the + * repository-relative space. + */ +export function admitObservationPath(element: string, value: string, what: string): string { + if (value === "") { + return invalid(element, `needs ${what} to name a path, and the empty string names none.`); + } + if (!wellFormedText(value)) { + return invalid( + element, + `needs ${what} to be well-formed text. It holds an unpaired surrogate, which this host ` + + "cannot hand to Git as written.", + ); + } + if (holdsNul(value)) { + return invalid(element, `needs ${what} to hold no NUL character.`); + } + if (value.startsWith("/")) { + return invalid( + element, + `needs ${what} to be relative to the repository root, and an absolute path is not.`, + ); + } + const trailing = value.endsWith("/"); + for (const segment of (trailing ? value.slice(0, -1) : value).split("/")) { + if (segment === "" || segment === "." || segment === "..") { + return invalid( + element, + `needs every segment of ${what} to be an ordinary name, so no empty segment, "." or ` + + '".." is accepted.', + ); + } + } + return value; +} + +/** + * The literal prefix this value is, or the read's own refusal. + * + * Wider than a path, because a prefix may stop inside a final segment: + * `packages/co` is a prefix of `packages/core/mod.ts`, and that is the whole + * point of a prefix. The empty string is the whole repository, which is also + * what omitting it means. + */ +function admitPrefix(element: string, value: Json | undefined): string | null { + if (value === undefined) { + return null; + } + if (typeof value !== "string") { + return invalid(element, "needs prefix to be a string."); + } + if (value === "") { + return null; + } + if (!wellFormedText(value)) { + return invalid( + element, + "needs prefix to be well-formed text. It holds an unpaired surrogate, which this host " + + "cannot hand to Git as written.", + ); + } + if (holdsNul(value)) { + return invalid(element, "needs prefix to hold no NUL character."); + } + if (value.startsWith("/")) { + return invalid( + element, + "needs prefix to be relative to the repository root, and an absolute path is not.", + ); + } + for (const segment of value.split("/").slice(0, -1)) { + if (segment === "" || segment === "." || segment === "..") { + return invalid( + element, + 'needs every complete segment of prefix to be an ordinary name, so no empty segment, "." ' + + 'or ".." is accepted.', + ); + } + } + return value; +} + +/** + * The path filters this value holds, or `null` when the read names none. + * + * An empty array is not omission. Omitting the prop means every path; writing + * `paths={[]}` names no path at all, and the read answers with nothing — which + * is a thing a document can say and a thing this must not quietly widen. + */ +function admitPathFilters(element: string, value: Json | undefined): readonly string[] | null { + if (value === undefined) { + return null; + } + if (!Array.isArray(value)) { + return invalid(element, "needs paths to be an array of repository-relative paths."); + } + const filters: string[] = []; + for (const entry of value) { + if (typeof entry !== "string") { + return invalid(element, "needs every entry of paths to be a string."); + } + filters.push(admitObservationPath(element, entry, "every entry of paths")); + } + return Object.freeze(filters); +} + +function admitCursor(element: string, value: Json | undefined): string | null { + if (value === undefined) { + return null; + } + if (typeof value !== "string" || value === "") { + return invalid(element, "needs cursor to be the opaque string a previous next supplied."); + } + return value; +} + +/** + * The comparison this invocation names, or the read's own refusal. + * + * `base` and `head` belong to `commits` and to nothing else. Requiring both + * there and forbidding both elsewhere is what stops a read from naming a + * revision the comparison it asked for would silently ignore. + */ +function admitComparison( + element: string, + props: Record, +): GitComparisonRequest { + const compare = props.compare; + const mode = GIT_COMPARE_MODES.find((candidate) => candidate === compare); + if (mode === undefined) { + return invalid(element, 'needs compare to be "unstaged", "staged" or "commits".'); + } + const base = props.base; + const head = props.head; + if (mode !== "commits") { + if (base !== undefined || head !== undefined) { + return invalid( + element, + `compares ${JSON.stringify(mode)}, which names no revisions, so base and head belong ` + + 'only to compare="commits".', + ); + } + return Object.freeze({ compare: mode }); + } + if (typeof base !== "string" || base === "" || typeof head !== "string" || head === "") { + return invalid(element, 'compares "commits", so it needs both base and head to name a commit.'); + } + if (!wellFormedText(base) || !wellFormedText(head) || holdsNul(base) || holdsNul(head)) { + return invalid( + element, + "needs base and head to be well-formed text with no NUL character, which is what this " + + "host can hand to Git as written.", + ); + } + return Object.freeze({ compare: "commits" as const, base, head }); +} + +/** The `` request these props ask for, copied and frozen. */ +export function admitFilesRequest( + element: string, + props: Record, +): GitFilesRequest { + return Object.freeze({ + kind: "files" as const, + prefix: admitPrefix(element, props.prefix), + limit: admitLimit(element, props.limit, DEFAULT_RECORD_LIMIT), + cursor: admitCursor(element, props.cursor), + }); +} + +/** The `` request these props ask for, copied and frozen. */ +export function admitChangesRequest( + element: string, + props: Record, +): GitChangesRequest { + return Object.freeze({ + kind: "changes" as const, + comparison: admitComparison(element, props), + paths: admitPathFilters(element, props.paths), + limit: admitLimit(element, props.limit, DEFAULT_RECORD_LIMIT), + cursor: admitCursor(element, props.cursor), + }); +} + +/** The `` request these props ask for, copied and frozen. */ +export function admitDiffRequest( + element: string, + props: Record, +): GitDiffRequest { + return Object.freeze({ + kind: "diff" as const, + comparison: admitComparison(element, props), + paths: admitPathFilters(element, props.paths), + offset: admitOffset(element, props.offset), + limit: admitLimit(element, props.limit, DEFAULT_TEXT_LIMIT), + cursor: admitCursor(element, props.cursor), + }); +} + +/** The `` request these props ask for, copied and frozen. */ +export function admitFileRequest( + element: string, + props: Record, +): GitFileRequest { + const commit = props.commit; + if (typeof commit !== "string" || commit === "") { + return invalid(element, "needs commit to name the revision to read the file from."); + } + if (!wellFormedText(commit) || holdsNul(commit)) { + return invalid( + element, + "needs commit to be well-formed text with no NUL character, which is what this host can " + + "hand to Git as written.", + ); + } + const path = props.path; + if (typeof path !== "string") { + return invalid(element, "needs path to name the file to read."); + } + return Object.freeze({ + kind: "file" as const, + commit, + path: admitObservationPath(element, path, "path"), + offset: admitOffset(element, props.offset), + limit: admitLimit(element, props.limit, DEFAULT_TEXT_LIMIT), + }); +} + +/** + * Whether this filter selects this path. + * + * Literal, and by whole segments: a selector matches the path it spells, and + * every path beneath it when the boundary is a `/`. A trailing slash selects + * descendants and not the directory entry itself, which is the one shape where + * the two differ. + */ +export function selectsPath(filter: string, path: string): boolean { + if (filter.endsWith("/")) { + return path.startsWith(filter); + } + return path === filter || path.startsWith(`${filter}/`); +} + +/** Whether these filters select this path, with `null` selecting every one. */ +export function selectedByFilters(filters: readonly string[] | null, path: string): boolean { + if (filters === null) { + return true; + } + return filters.some((filter) => selectsPath(filter, path)); +} + +/** Whether a change record is selected, matching both sides of a rename. */ +export function selectedChange(filters: readonly string[] | null, change: GitChange): boolean { + if (selectedByFilters(filters, change.path)) { + return true; + } + return change.previousPath !== null && selectedByFilters(filters, change.previousPath); +} + +/** Change records in code-point order by current path, then previous path. */ +export function sortedChanges(changes: readonly GitChange[]): readonly GitChange[] { + return Object.freeze( + [...changes].sort((left, right) => { + if (left.path !== right.path) { + return left.path < right.path ? -1 : 1; + } + const leftPrevious = left.previousPath ?? ""; + const rightPrevious = right.previousPath ?? ""; + if (leftPrevious === rightPrevious) { + return 0; + } + return leftPrevious < rightPrevious ? -1 : 1; + }), + ); +} + +/** + * The same admission, applied to a request that arrived through the Api. + * + * The four above read what a document wrote; these four read what a caller + * handed the contextual operation. Both routes end at one boundary, because the + * Api is public: a caller reaching it directly is subject to the same limits, + * and the copy is what stops the object it still holds from being the one a + * provider names its record by. + */ +export function admitFilesInvocation(element: string, request: GitFilesRequest): GitFilesRequest { + return admitFilesRequest(element, { + ...(request.prefix === null ? {} : { prefix: request.prefix }), + limit: request.limit, + ...(request.cursor === null ? {} : { cursor: request.cursor }), + }); +} + +export function admitChangesInvocation( + element: string, + request: GitChangesRequest, +): GitChangesRequest { + return admitChangesRequest(element, { + ...comparisonProps(request.comparison), + ...(request.paths === null ? {} : { paths: [...request.paths] }), + limit: request.limit, + ...(request.cursor === null ? {} : { cursor: request.cursor }), + }); +} + +export function admitDiffInvocation(element: string, request: GitDiffRequest): GitDiffRequest { + return admitDiffRequest(element, { + ...comparisonProps(request.comparison), + ...(request.paths === null ? {} : { paths: [...request.paths] }), + offset: request.offset, + limit: request.limit, + ...(request.cursor === null ? {} : { cursor: request.cursor }), + }); +} + +export function admitFileInvocation(element: string, request: GitFileRequest): GitFileRequest { + return admitFileRequest(element, { + commit: request.commit, + path: request.path, + offset: request.offset, + limit: request.limit, + }); +} + +function comparisonProps(comparison: GitComparisonRequest): Record { + if (comparison.compare === "commits") { + return { compare: "commits", base: comparison.base, head: comparison.head }; + } + return { compare: comparison.compare }; +} diff --git a/packages/git/src/composition/observation-records.ts b/packages/git/src/composition/observation-records.ts new file mode 100644 index 000000000..68aee8984 --- /dev/null +++ b/packages/git/src/composition/observation-records.ts @@ -0,0 +1,926 @@ +/** + * What a local Git observation looks like as JSON, and how a continuation is + * carried. + * + * Two jobs, and they share one contract. A result is written here and parsed + * here, so the page a document binds and the page a journal restores are the + * same object; and the opaque `cursor` a `next` carries is built here and + * verified here, so what a follow-up read is held to is what the first read + * described. + * + * ## A cursor is data, not permission + * + * It names the read kind, the normalized query, the authenticated checkout, the + * resolved comparison, a digest of the complete filtered observation, and the + * position the next page starts at. It carries no content, no credential, no + * callable, no live selection identifier and no grant. A provider authenticates + * and routes independently of it, and then verifies that the cursor describes + * the checkout that route reached — the cursor's own root is never a + * destination. + * + * The digest is what makes continuation honest. A follow-up recomputes the + * whole filtered observation and compares; an answer that has changed cannot be + * paged over, so the read refuses and says to start again. A change elsewhere in + * the repository that does not alter this observation changes no digest and + * needs no restart. Changing `limit` is safe for the same reason: a limit is how + * much of an answer to return, not which answer it is. + * + * ## Windows are code points + * + * `offset`, `limit`, `length` and a next offset all count Unicode code points, + * which is what the REPL's own text windows count. A surrogate pair is one code + * point, so no page ever ends half way through a character. + */ + +import { until } from "effection"; +import type { Operation } from "effection"; +import type { Json } from "@executablemd/durable-streams"; +import { canonicalJson } from "@executablemd/workflow"; +import { members, text } from "./parse.ts"; +import { + parseRepositoryIdentity, + repositoryIdentityJson, + sameRepositoryIdentity, + type RepositoryIdentity, +} from "./selection.ts"; +import { + GIT_CHANGE_STATUSES, + GIT_COMPARE_MODES, + type GitChange, + type GitChangeStatus, + type GitChangesNext, + type GitChangesRequest, + type GitChangesResult, + type GitCompareMode, + type GitComparison, + type GitDiffNext, + type GitDiffRequest, + type GitDiffResult, + type GitFileNext, + type GitFileRequest, + type GitFileResult, + type GitFilesNext, + type GitFilesRequest, + type GitFilesResult, + type GitObservationKind, + type GitObservationRequest, + type GitObservationValue, +} from "./observation-api.ts"; + +/** The cursor payload shape this host writes and the only one it reads. */ +export const GIT_OBSERVATION_CURSOR_VERSION = 1; + +/** The SHA-256 digest of one canonical value, lowercase. */ +export function* observationDigest(value: Json): Operation { + const encoded = new TextEncoder().encode(canonicalJson(value)); + const computed = yield* until(crypto.subtle.digest("SHA-256", encoded)); + return [...new Uint8Array(computed)].map((byte) => byte.toString(16).padStart(2, "0")).join(""); +} + +/** How many code points this string holds. */ +export function codePointLength(value: string): number { + let count = 0; + for (let index = 0; index < value.length; index += 1) { + const unit = value.charCodeAt(index); + if (unit >= 0xd800 && unit <= 0xdbff && index + 1 < value.length) { + const following = value.charCodeAt(index + 1); + if (following >= 0xdc00 && following <= 0xdfff) { + index += 1; + } + } + count += 1; + } + return count; +} + +/** + * The code-unit index the given code point starts at. + * + * Walked rather than materialized: a 64-kilocharacter blob is an ordinary input + * here, and an array of its code points would be an allocation per read for an + * answer two indices already give. + */ +function unitIndexOf(value: string, codePoint: number): number { + let seen = 0; + let index = 0; + while (index < value.length && seen < codePoint) { + const unit = value.charCodeAt(index); + index += 1; + if (unit >= 0xd800 && unit <= 0xdbff && index < value.length) { + const following = value.charCodeAt(index); + if (following >= 0xdc00 && following <= 0xdfff) { + index += 1; + } + } + seen += 1; + } + return index; +} + +/** One text window: the characters it holds, the whole length, and what follows. */ +export interface GitTextWindow { + readonly text: string; + /** The whole observation's length in code points. */ + readonly length: number; + /** The absolute code-point offset the next window starts at, or `null`. */ + readonly nextOffset: number | null; +} + +/** + * The window this offset and limit select. + * + * An offset at or past the end is an empty window at the offset that was asked + * for, with the whole length and no next read — reading past the end of a file + * is an answer rather than a refusal. Nothing is trimmed and no newline is + * dropped: the window is the bytes the observation held, decoded. + */ +export function gitTextWindow(value: string, offset: number, limit: number): GitTextWindow { + const length = codePointLength(value); + if (offset >= length) { + return Object.freeze({ text: "", length, nextOffset: null }); + } + // Subtraction rather than `offset + limit`, which a safe-integer limit can + // carry past what integer arithmetic represents. + const taken = Math.min(limit, length - offset); + const start = unitIndexOf(value, offset); + const end = unitIndexOf(value, offset + taken); + const reached = offset + taken; + return Object.freeze({ + text: value.slice(start, end), + length, + nextOffset: reached < length ? reached : null, + }); +} + +/** One page of records: the entries it holds, and the index that follows it. */ +export interface GitRecordWindow { + readonly page: readonly T[]; + readonly nextIndex: number | null; +} + +export function gitRecordWindow( + records: readonly T[], + offset: number, + limit: number, +): GitRecordWindow { + if (offset >= records.length) { + return Object.freeze({ page: Object.freeze([]), nextIndex: null }); + } + const taken = Math.min(limit, records.length - offset); + const reached = offset + taken; + return Object.freeze({ + page: Object.freeze(records.slice(offset, reached)), + nextIndex: reached < records.length ? reached : null, + }); +} + +export function gitComparisonJson(comparison: GitComparison): Json { + if (comparison.compare === "unstaged") { + return { compare: "unstaged" }; + } + if (comparison.compare === "staged") { + return { compare: "staged", base: comparison.base }; + } + return { compare: "commits", base: comparison.base, head: comparison.head }; +} + +/** The comparison this value describes, or `undefined`. */ +export function parseGitComparison(value: unknown): GitComparison | undefined { + const unstaged = members(value, ["compare"]); + if (unstaged !== undefined) { + return unstaged["compare"] === "unstaged" ? Object.freeze({ compare: "unstaged" }) : undefined; + } + const staged = members(value, ["compare", "base"]); + if (staged !== undefined) { + if (staged["compare"] !== "staged") { + return undefined; + } + const base = staged["base"]; + if (base === null) { + return Object.freeze({ compare: "staged" as const, base: null }); + } + const resolved = text(base); + return resolved === undefined + ? undefined + : Object.freeze({ compare: "staged" as const, base: resolved }); + } + const commits = members(value, ["compare", "base", "head"]); + if (commits === undefined || commits["compare"] !== "commits") { + return undefined; + } + const base = text(commits["base"]); + const head = text(commits["head"]); + if (base === undefined || head === undefined) { + return undefined; + } + return Object.freeze({ compare: "commits" as const, base, head }); +} + +export function gitChangeJson(change: GitChange): Json { + return { path: change.path, status: change.status, previousPath: change.previousPath }; +} + +/** The status this value names, or `undefined`. */ +export function parseGitChangeStatus(value: unknown): GitChangeStatus | undefined { + return GIT_CHANGE_STATUSES.find((status) => status === value); +} + +function parseGitChange(value: unknown): GitChange | undefined { + const record = members(value, ["path", "status", "previousPath"]); + if (record === undefined) { + return undefined; + } + const path = text(record["path"]); + const status = parseGitChangeStatus(record["status"]); + const previous = record["previousPath"]; + if (path === undefined || status === undefined) { + return undefined; + } + // A rename is the one status with a previous path, and the only one. A record + // pairing them any other way describes a change this host does not produce. + if (status !== "R") { + return previous === null ? Object.freeze({ path, status, previousPath: null }) : undefined; + } + const previousPath = text(previous); + return previousPath === undefined ? undefined : Object.freeze({ path, status, previousPath }); +} + +function paths(value: unknown): readonly string[] | undefined { + if (!Array.isArray(value)) { + return undefined; + } + const entries: string[] = []; + for (const entry of value) { + const path = text(entry); + if (path === undefined) { + return undefined; + } + entries.push(path); + } + return Object.freeze(entries); +} + +function count(value: unknown): number | undefined { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 0) { + return undefined; + } + return value; +} + +function positive(value: unknown): number | undefined { + const parsed = count(value); + return parsed === undefined || parsed < 1 ? undefined : parsed; +} + +/** + * The follow-up props for a list or text read, with omitted filters omitted. + * + * A `next` is the complete arguments of one more read, so what is present in it + * is exactly what the read it follows had: a prefix nobody wrote stays absent + * rather than arriving as a null the next invocation would have to interpret. + */ +function comparisonNextJson(comparison: GitComparison): Record { + if (comparison.compare === "commits") { + return { compare: "commits", base: comparison.base, head: comparison.head }; + } + return { compare: comparison.compare }; +} + +export function gitFilesNextJson(next: GitFilesNext): Json { + return { + ...(next.prefix === undefined ? {} : { prefix: next.prefix }), + limit: next.limit, + cursor: next.cursor, + }; +} + +export function gitChangesNextJson(next: GitChangesNext): Json { + return { + compare: next.compare, + ...(next.base === undefined ? {} : { base: next.base }), + ...(next.head === undefined ? {} : { head: next.head }), + ...(next.paths === undefined ? {} : { paths: [...next.paths] }), + limit: next.limit, + cursor: next.cursor, + }; +} + +export function gitDiffNextJson(next: GitDiffNext): Json { + return { + compare: next.compare, + ...(next.base === undefined ? {} : { base: next.base }), + ...(next.head === undefined ? {} : { head: next.head }), + ...(next.paths === undefined ? {} : { paths: [...next.paths] }), + offset: next.offset, + limit: next.limit, + cursor: next.cursor, + }; +} + +export function gitFileNextJson(next: GitFileNext): Json { + return { commit: next.commit, path: next.path, offset: next.offset, limit: next.limit }; +} + +export function gitFilesResultJson(result: GitFilesResult): Json { + return { + paths: [...result.paths], + next: result.next === null ? null : gitFilesNextJson(result.next), + }; +} + +export function gitChangesResultJson(result: GitChangesResult): Json { + return { + comparison: gitComparisonJson(result.comparison), + changes: result.changes.map(gitChangeJson), + next: result.next === null ? null : gitChangesNextJson(result.next), + }; +} + +export function gitDiffResultJson(result: GitDiffResult): Json { + return { + comparison: gitComparisonJson(result.comparison), + text: result.text, + offset: result.offset, + length: result.length, + next: result.next === null ? null : gitDiffNextJson(result.next), + }; +} + +export function gitFileResultJson(result: GitFileResult): Json { + return { + commit: result.commit, + path: result.path, + text: result.text, + offset: result.offset, + length: result.length, + next: result.next === null ? null : gitFileNextJson(result.next), + }; +} + +export function gitObservationValueJson( + kind: GitObservationKind, + value: GitObservationValue, +): Json { + if (kind === "files" && "paths" in value) { + return gitFilesResultJson(value); + } + if (kind === "changes" && "changes" in value) { + return gitChangesResultJson(value); + } + if (kind === "diff" && "text" in value && !("commit" in value)) { + return gitDiffResultJson(value); + } + if (kind === "file" && "commit" in value) { + return gitFileResultJson(value); + } + throw new TypeError(`a retained ${kind} observation carries no ${kind} result`); +} + +function parseFilesNext(value: unknown): GitFilesNext | null | undefined { + if (value === null) { + return null; + } + const withPrefix = members(value, ["prefix", "limit", "cursor"]); + const bare = withPrefix === undefined ? members(value, ["limit", "cursor"]) : undefined; + const read = withPrefix ?? bare; + if (read === undefined) { + return undefined; + } + const limit = positive(read["limit"]); + const cursor = text(read["cursor"]); + if (limit === undefined || cursor === undefined) { + return undefined; + } + if (withPrefix === undefined) { + return Object.freeze({ limit, cursor }); + } + const prefix = text(read["prefix"]); + return prefix === undefined ? undefined : Object.freeze({ prefix, limit, cursor }); +} + +/** + * The comparison and filter members a list or text `next` repeats. + * + * Read by presence, because presence is the contract: a `commits` follow-up + * names both revisions, every other mode names none, and a read with no path + * filter repeats none. + */ +function parseComparisonNext( + read: Record, +): + | { compare: GitCompareMode; base?: string; head?: string; paths?: readonly string[] } + | undefined { + const compare = GIT_COMPARE_MODES.find((mode) => mode === read["compare"]); + if (compare === undefined) { + return undefined; + } + const hasBase = Object.hasOwn(read, "base"); + const hasHead = Object.hasOwn(read, "head"); + if (compare === "commits" ? !hasBase || !hasHead : hasBase || hasHead) { + return undefined; + } + const filters = Object.hasOwn(read, "paths") ? paths(read["paths"]) : undefined; + if (Object.hasOwn(read, "paths") && filters === undefined) { + return undefined; + } + if (compare !== "commits") { + return filters === undefined ? { compare } : { compare, paths: filters }; + } + const base = text(read["base"]); + const head = text(read["head"]); + if (base === undefined || head === undefined) { + return undefined; + } + return filters === undefined ? { compare, base, head } : { compare, base, head, paths: filters }; +} + +/** Every member a comparison `next` may hold, as the parser enumerates them. */ +function comparisonNextMembers( + value: unknown, + own: readonly string[], +): Record | undefined { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return undefined; + } + const record: Record = Object.fromEntries(Object.entries(value)); + const permitted = new Set(["compare", "base", "head", "paths", ...own]); + for (const key of Object.keys(record)) { + if (!permitted.has(key)) { + return undefined; + } + } + for (const key of own) { + if (!Object.hasOwn(record, key)) { + return undefined; + } + } + return Object.hasOwn(record, "compare") ? record : undefined; +} + +function parseChangesNext(value: unknown): GitChangesNext | null | undefined { + if (value === null) { + return null; + } + const read = comparisonNextMembers(value, ["limit", "cursor"]); + if (read === undefined) { + return undefined; + } + const shared = parseComparisonNext(read); + const limit = positive(read["limit"]); + const cursor = text(read["cursor"]); + if (shared === undefined || limit === undefined || cursor === undefined) { + return undefined; + } + return Object.freeze({ ...shared, limit, cursor }); +} + +function parseDiffNext(value: unknown): GitDiffNext | null | undefined { + if (value === null) { + return null; + } + const read = comparisonNextMembers(value, ["offset", "limit", "cursor"]); + if (read === undefined) { + return undefined; + } + const shared = parseComparisonNext(read); + const offset = count(read["offset"]); + const limit = positive(read["limit"]); + const cursor = text(read["cursor"]); + if (shared === undefined || offset === undefined || limit === undefined || cursor === undefined) { + return undefined; + } + return Object.freeze({ ...shared, offset, limit, cursor }); +} + +function parseFileNext(value: unknown): GitFileNext | null | undefined { + if (value === null) { + return null; + } + const read = members(value, ["commit", "path", "offset", "limit"]); + if (read === undefined) { + return undefined; + } + const commit = text(read["commit"]); + const path = text(read["path"]); + const offset = count(read["offset"]); + const limit = positive(read["limit"]); + if (commit === undefined || path === undefined || offset === undefined || limit === undefined) { + return undefined; + } + return Object.freeze({ commit, path, offset, limit }); +} + +/** + * The result this saved value describes for this request, or `undefined`. + * + * Total about members and exact about the request: a page describing another + * query is not this read's answer, so a replay that found one refuses rather + * than binding it. The comparison a result carries is the resolved one, which + * the request's authored mode has to agree with. + */ +export function parseGitObservationValue( + value: unknown, + request: GitObservationRequest, +): GitObservationValue | undefined { + if (request.kind === "files") { + return parseFilesResult(value, request); + } + if (request.kind === "changes") { + return parseChangesResult(value, request); + } + if (request.kind === "diff") { + return parseDiffResult(value, request); + } + return parseFileResult(value, request); +} + +function parseFilesResult(value: unknown, request: GitFilesRequest): GitFilesResult | undefined { + const read = members(value, ["paths", "next"]); + if (read === undefined) { + return undefined; + } + const listed = paths(read["paths"]); + const next = parseFilesNext(read["next"]); + if (listed === undefined || next === undefined) { + return undefined; + } + if (listed.length > request.limit) { + return undefined; + } + if (next !== null && (next.prefix ?? null) !== request.prefix) { + return undefined; + } + return Object.freeze({ paths: listed, next }); +} + +function sameFilters( + left: readonly string[] | undefined, + right: readonly string[] | null, +): boolean { + if (left === undefined) { + return right === null; + } + if (right === null) { + return false; + } + return left.length === right.length && left.every((entry, index) => entry === right[index]); +} + +/** Whether a next page's comparison repeats the mode the request asked for. */ +function sameMode( + next: { compare: GitCompareMode }, + request: { comparison: { compare: GitCompareMode } }, +): boolean { + return next.compare === request.comparison.compare; +} + +function parseChangesResult( + value: unknown, + request: GitChangesRequest, +): GitChangesResult | undefined { + const read = members(value, ["comparison", "changes", "next"]); + if (read === undefined) { + return undefined; + } + const comparison = parseGitComparison(read["comparison"]); + const next = parseChangesNext(read["next"]); + if (comparison === undefined || next === undefined) { + return undefined; + } + if (comparison.compare !== request.comparison.compare) { + return undefined; + } + if (!Array.isArray(read["changes"])) { + return undefined; + } + const changes: GitChange[] = []; + for (const entry of read["changes"]) { + const change = parseGitChange(entry); + if (change === undefined) { + return undefined; + } + changes.push(change); + } + if (changes.length > request.limit) { + return undefined; + } + if (next !== null && (!sameMode(next, request) || !sameFilters(next.paths, request.paths))) { + return undefined; + } + return Object.freeze({ comparison, changes: Object.freeze(changes), next }); +} + +function parseDiffResult(value: unknown, request: GitDiffRequest): GitDiffResult | undefined { + const read = members(value, ["comparison", "text", "offset", "length", "next"]); + if (read === undefined) { + return undefined; + } + const comparison = parseGitComparison(read["comparison"]); + const patch = read["text"]; + const offset = count(read["offset"]); + const length = count(read["length"]); + const next = parseDiffNext(read["next"]); + if ( + comparison === undefined || + typeof patch !== "string" || + offset === undefined || + length === undefined || + next === undefined + ) { + return undefined; + } + if (comparison.compare !== request.comparison.compare || offset !== request.offset) { + return undefined; + } + if (codePointLength(patch) > request.limit) { + return undefined; + } + if (next !== null && (!sameMode(next, request) || !sameFilters(next.paths, request.paths))) { + return undefined; + } + return Object.freeze({ comparison, text: patch, offset, length, next }); +} + +function parseFileResult(value: unknown, request: GitFileRequest): GitFileResult | undefined { + const read = members(value, ["commit", "path", "text", "offset", "length", "next"]); + if (read === undefined) { + return undefined; + } + const commit = text(read["commit"]); + const path = text(read["path"]); + const content = read["text"]; + const offset = count(read["offset"]); + const length = count(read["length"]); + const next = parseFileNext(read["next"]); + if ( + commit === undefined || + path === undefined || + typeof content !== "string" || + offset === undefined || + length === undefined || + next === undefined + ) { + return undefined; + } + if (path !== request.path || offset !== request.offset) { + return undefined; + } + if (codePointLength(content) > request.limit) { + return undefined; + } + // The continuation stays on the commit this read resolved. A next page naming + // another commit would read a different file under the same question. + if (next !== null && (next.commit !== commit || next.path !== path)) { + return undefined; + } + return Object.freeze({ commit, path, text: content, offset, length, next }); +} + +/** + * The checkout one cursor was built against, as the provider authenticated it. + * + * The whole Repository identity, the checkout root and the common Git + * directory. A follow-up read authenticates and routes on its own terms and + * then checks that this is the checkout it reached; a cursor naming another one + * describes a different observation and is not a page of this one. + */ +export interface GitObservationCheckout { + readonly repository: RepositoryIdentity; + readonly root: string; + readonly commonDirectory: string; +} + +/** Which reads page, and therefore which reads carry a cursor. */ +export type GitPagedKind = "files" | "changes" | "diff"; + +/** One continuation, as the opaque string a `next` carries decodes to. */ +export interface GitObservationCursor { + readonly version: typeof GIT_OBSERVATION_CURSOR_VERSION; + readonly kind: GitPagedKind; + /** The normalized query, which a follow-up has to ask again exactly. */ + readonly query: Json; + readonly checkout: GitObservationCheckout; + /** The comparison the first page resolved, or `null` for a file listing. */ + readonly comparison: GitComparison | null; + /** SHA-256 of the complete filtered observation this cursor pages over. */ + readonly observation: string; + /** The record index or absolute code-point offset the next page starts at. */ + readonly position: number; +} + +export function gitObservationCheckoutJson(checkout: GitObservationCheckout): Json { + return { + repository: repositoryIdentityJson(checkout.repository), + root: checkout.root, + commonDirectory: checkout.commonDirectory, + }; +} + +function parseObservationCheckout(value: unknown): GitObservationCheckout | undefined { + const read = members(value, ["repository", "root", "commonDirectory"]); + if (read === undefined) { + return undefined; + } + const repository = parseRepositoryIdentity(read["repository"]); + const root = text(read["root"]); + const commonDirectory = text(read["commonDirectory"]); + if (repository === undefined || root === undefined || commonDirectory === undefined) { + return undefined; + } + return Object.freeze({ repository, root, commonDirectory }); +} + +/** Whether two cursors describe the same authenticated checkout. */ +export function sameObservationCheckout( + left: GitObservationCheckout, + right: GitObservationCheckout, +): boolean { + return ( + left.root === right.root && + left.commonDirectory === right.commonDirectory && + sameRepositoryIdentity(left.repository, right.repository) + ); +} + +/** + * The normalized query one observation pages over. + * + * It holds what decides *which* answer this is and nothing about how much of it + * to return, which is why a caller may raise `limit` between pages without + * restarting. A commit comparison contributes the resolved full object ids, so + * the query a follow-up computes from the `next` it was handed is the query the + * first page recorded — and a branch that moved cannot retarget a later page. + */ +export function gitObservationQuery( + request: GitObservationRequest, + comparison: GitComparison | null, +): Json { + if (request.kind === "files") { + return { kind: "files", prefix: request.prefix }; + } + if (request.kind === "file") { + return { kind: "file", commit: request.commit, path: request.path }; + } + const resolved: Json = + comparison !== null && comparison.compare === "commits" + ? { compare: "commits", base: comparison.base, head: comparison.head } + : { compare: request.comparison.compare }; + return { + kind: request.kind, + comparison: resolved, + paths: request.paths === null ? null : [...request.paths], + }; +} + +/** The comparison members a follow-up `next` repeats for this comparison. */ +export function gitComparisonNext(comparison: GitComparison): { + readonly compare: GitCompareMode; + readonly base?: string; + readonly head?: string; +} { + if (comparison.compare === "commits") { + return Object.freeze({ + compare: "commits" as const, + base: comparison.base, + head: comparison.head, + }); + } + return Object.freeze({ compare: comparison.compare }); +} + +const BASE64_URL_UNSAFE: ReadonlyMap = new Map([ + ["+", "-"], + ["/", "_"], +]); + +/** + * The cursor, as the opaque string a result carries. + * + * Base64url of the canonical JSON payload. The encoding is private: a caller + * passes the string back and nothing else is promised about it, which is what + * keeps the payload's shape this module's to change. + */ +export function encodeGitObservationCursor(cursor: GitObservationCursor): string { + const payload = canonicalJson({ + version: cursor.version, + kind: cursor.kind, + query: cursor.query, + checkout: gitObservationCheckoutJson(cursor.checkout), + comparison: cursor.comparison === null ? null : gitComparisonJson(cursor.comparison), + observation: cursor.observation, + position: cursor.position, + }); + const bytes = new TextEncoder().encode(payload); + let binary = ""; + for (const byte of bytes) { + binary += String.fromCharCode(byte); + } + return btoa(binary) + .replace(/[+/]/g, (character) => BASE64_URL_UNSAFE.get(character) ?? character) + .replace(/=+$/, ""); +} + +/** The continuation this opaque string describes, or `undefined`. */ +export function parseGitObservationCursor(encoded: string): GitObservationCursor | undefined { + const padded = encoded.replace(/-/g, "+").replace(/_/g, "/"); + let payload: string; + try { + const binary = atob(padded); + const bytes = new Uint8Array(binary.length); + for (let index = 0; index < binary.length; index += 1) { + bytes[index] = binary.charCodeAt(index); + } + payload = new TextDecoder("utf-8", { fatal: true }).decode(bytes); + } catch { + return undefined; + } + let decoded: unknown; + try { + decoded = JSON.parse(payload); + } catch { + return undefined; + } + const read = members(decoded, [ + "version", + "kind", + "query", + "checkout", + "comparison", + "observation", + "position", + ]); + if (read === undefined || read["version"] !== GIT_OBSERVATION_CURSOR_VERSION) { + return undefined; + } + const kind = PAGED_KINDS.find((candidate) => candidate === read["kind"]); + const checkout = parseObservationCheckout(read["checkout"]); + const observation = text(read["observation"]); + const position = count(read["position"]); + if ( + kind === undefined || + checkout === undefined || + observation === undefined || + !/^[0-9a-f]{64}$/.test(observation) || + position === undefined + ) { + return undefined; + } + const comparison = read["comparison"] === null ? null : parseGitComparison(read["comparison"]); + if (comparison === undefined) { + return undefined; + } + const query = jsonValue(read["query"]); + if (query === undefined) { + return undefined; + } + return Object.freeze({ + version: GIT_OBSERVATION_CURSOR_VERSION, + kind, + query, + checkout, + comparison, + observation, + position, + }); +} + +const PAGED_KINDS: readonly GitPagedKind[] = ["files", "changes", "diff"]; + +/** + * This value, once it is ordinary JSON. + * + * The cursor's query is compared as canonical JSON rather than interpreted, so + * what it has to be is a value this host can canonicalize again — not a shape + * this module decided in advance. + */ +function jsonValue(value: unknown): Json | undefined { + if (value === null || typeof value === "boolean" || typeof value === "number") { + return value; + } + if (typeof value === "string") { + return value; + } + if (Array.isArray(value)) { + const entries: Json[] = []; + for (const entry of value) { + const parsed = jsonValue(entry); + if (parsed === undefined) { + return undefined; + } + entries.push(parsed); + } + return entries; + } + if (typeof value !== "object") { + return undefined; + } + const object: Record = {}; + for (const [key, entry] of Object.entries(value)) { + const parsed = jsonValue(entry); + if (parsed === undefined) { + return undefined; + } + object[key] = parsed; + } + return object; +} diff --git a/packages/git/src/deno/composition/git.ts b/packages/git/src/deno/composition/git.ts index e0e9fc5ae..297112e9c 100644 --- a/packages/git/src/deno/composition/git.ts +++ b/packages/git/src/deno/composition/git.ts @@ -75,6 +75,8 @@ export interface GitCommand { * invocation, and every command that invocation runs attaches the same one. */ readonly attachment?: GitAttachment; + /** Variables this command needs beyond the session's fixed environment. */ + readonly variables?: Readonly>; } export interface GitSession { diff --git a/packages/git/src/deno/composition/host.ts b/packages/git/src/deno/composition/host.ts index a78cc2375..04cf6ee7b 100644 --- a/packages/git/src/deno/composition/host.ts +++ b/packages/git/src/deno/composition/host.ts @@ -59,11 +59,20 @@ import type { GitAuthentication, GitAuthenticationSession, } from "./authentication.ts"; -import { runProcess, type ProcessInvocation, type ProcessOutcome } from "./subprocess.ts"; +import { + runProcess, + runProcessBytes, + type ProcessBytes, + type ProcessInvocation, + type ProcessOutcome, +} from "./subprocess.ts"; /** What one Git invocation reported. A nonzero exit is an answer, not a throw. */ export type GitOutcome = ProcessOutcome; +/** The same, with standard output kept as the bytes Git wrote. */ +export type GitBytes = ProcessBytes; + export interface GitInvocation { readonly args: readonly string[]; /** The directory Git runs in. */ @@ -80,9 +89,9 @@ export interface GitInvocation { /** * The whole Unix second a commit is authored and committed at. * - * Two named variables rather than an environment a caller may fill: what a - * run's Git state is written by is fixed, and the one thing an operation - * decides for itself is when. Absent for every command that writes no object. + * Two named variables rather than a date left to the clock: what a run's Git + * state is written by is fixed, and the one thing an operation decides for + * itself is when. Absent for every command that writes no object. */ readonly committedAt?: number; /** @@ -96,6 +105,17 @@ export interface GitInvocation { * as fixed as it is for a workflow. */ readonly identity?: GitCommitIdentity; + /** + * Variables this command needs beyond the fixed environment above. + * + * The one route by which a variable is added, and it adds rather than + * replaces: every protection the built environment states stays stated. A + * read-only observation is what needs it — a private index file to read, lazy + * fetching to disable so a missing object refuses instead of reaching a + * remote, and optional locks to leave alone. Absent for every command that + * works inside the materialization as it is. + */ + readonly variables?: Readonly>; /** * What this command's provider invocation borrowed from the host. * @@ -119,6 +139,16 @@ export interface GitInvocation { export interface RepositoryHost { git(invocation: GitInvocation): Operation; + /** + * The same invocation, with standard output kept as the bytes Git wrote. + * + * Optional, because a suite that substitutes the whole host has already + * replaced the thing a byte-preserving read would be reading. A read that + * needs one and finds none reports that rather than decoding with + * replacement: a path listing is NUL-framed and a blob may not be text, and + * neither survives a substitution made during capture. + */ + gitBytes?(invocation: GitInvocation): Operation; /** A host directory owned by the acquiring scope, removed when it ends. */ useDirectory(): Operation; /** @@ -240,36 +270,53 @@ export interface RepositoryHostOptions { readonly helper?: HelperAssembly; } +/** + * One Git invocation, as the child process this host spawns. + * + * Both captures build it the same way, so a byte-preserving read runs under + * exactly the fixed configuration and built environment a decoding one does. + * + * A command with no attachment reaches no authentication mechanism. That is + * what makes a completed replay reach none: replay performs no remote + * operation, so no invocation ever opens a session to attach. + */ +function invoked({ + args, + cwd, + home, + input, + committedAt, + identity, + variables, + attachment, + observe, +}: GitInvocation): ProcessInvocation { + return { + command: "git", + args: [...CONFIGURATION, ...(attachment?.configuration ?? []), ...args], + cwd, + env: { + ...environment(home, committedAt, identity), + ...(variables ?? {}), + ...(attachment?.environment ?? {}), + }, + ...(input === undefined ? {} : { input }), + ...(observe === undefined ? {} : { observe }), + }; +} + export function denoRepositoryHost(options: RepositoryHostOptions = {}): RepositoryHost { const directories = options.directories ?? { create: (prefix: string) => mkdtemp(prefix) }; const authentication = options.authentication ?? denoGitAuthentication(options.helper === undefined ? {} : { assembly: options.helper }); return { - git({ - args, - cwd, - home, - input, - committedAt, - identity, - attachment, - observe, - }: GitInvocation): Operation { - // A command with no attachment reaches no authentication mechanism. That - // is what makes a completed replay reach none: replay performs no remote - // operation, so no invocation ever opens a session to attach. - return runProcess({ - command: "git", - args: [...CONFIGURATION, ...(attachment?.configuration ?? []), ...args], - cwd, - env: { - ...environment(home, committedAt, identity), - ...(attachment?.environment ?? {}), - }, - ...(input === undefined ? {} : { input }), - ...(observe === undefined ? {} : { observe }), - }); + git(invocation: GitInvocation): Operation { + return runProcess(invoked(invocation)); + }, + + gitBytes(invocation: GitInvocation): Operation { + return runProcessBytes(invoked(invocation)); }, useAuthentication(locator: string): Operation { diff --git a/packages/git/src/deno/composition/observations.ts b/packages/git/src/deno/composition/observations.ts new file mode 100644 index 000000000..fc64cfa53 --- /dev/null +++ b/packages/git/src/deno/composition/observations.ts @@ -0,0 +1,1005 @@ +/** + * The native side of the four local Git observations: fixed reads, in a view + * this observation owns, that change nothing. + * + * Never the selected checkout's own Git administration. A checkout carries a + * `.git/config` a person — or a document — may have written, and several + * ordinary settings in one name a *program*: `diff.external`, a textconv + * filter, a `clean`/`process` filter, `core.fsmonitor`, a credential helper. + * Running a read there would let repository configuration decide what else + * runs while somebody looks at a diff. + * + * So each observation builds the same contained view the push transport uses: a + * bare control repository this observation created, configured by nothing but + * its own creation, reading the checkout's objects through a read-only + * alternate whose whole graph has been proven to stay inside the object + * database the provider authenticated. The checkout's configuration is not + * consulted, no transport is attached, and lazy fetching is disabled even while + * a revision is being resolved — a missing promisor object refuses rather than + * reaching a remote. + * + * ## The index is copied, never touched + * + * Two of the three comparisons and the file listing are about the index, and + * the index is a file the person at the keyboard is using. It is discovered + * through Git's own path query rather than assumed to be `/index`, + * because a linked worktree has one of its own; its bytes are copied into this + * observation's scratch directory; and every index-dependent read runs against + * that copy through `GIT_INDEX_FILE`. A split index is read the same way, with + * the shared-index bytes copied beside the private copy — which is where Git + * looks for them. Nothing refreshes, expands or repairs the source. + * + * ## Revisions resolve where the refs are + * + * The control repository holds no references, so an authored revision is + * resolved once in the authenticated checkout — with `--verify`, + * `--end-of-options` and `^{commit}`, so a revision that looks like a flag is + * not read as one and a tag pointing at anything else is refused — and only + * full object ids are used in the isolated view afterwards. + */ + +import { scoped, until } from "effection"; +import type { Operation } from "effection"; +import { copyFile, lstat, readdir } from "@effectionx/fs"; +import { utimes } from "node:fs/promises"; +import { GitObservationError, GitOperationInfrastructureError } from "../../composition/errors.ts"; +import type { GitObjectFormat } from "../../composition/records.ts"; +import type { RepositoryIdentity } from "../../composition/selection.ts"; +import { + DEFAULT_RECORD_LIMIT, + selectedChange, + selectedByFilters, + sortedChanges, + type GitChange, + type GitChangeStatus, + type GitChangesRequest, + type GitChangesResult, + type GitComparison, + type GitComparisonRequest, + type GitDiffRequest, + type GitDiffResult, + type GitFileRequest, + type GitFileResult, + type GitFilesRequest, + type GitFilesResult, + type GitObservationRequest, + type GitObservationValue, +} from "../../composition/observation-api.ts"; +import { + encodeGitObservationCursor, + gitComparisonJson, + gitComparisonNext, + gitObservationQuery, + gitRecordWindow, + gitTextWindow, + observationDigest, + parseGitObservationCursor, + sameObservationCheckout, + type GitObservationCheckout, + type GitObservationCursor, + type GitPagedKind, +} from "../../composition/observation-records.ts"; +import { gitSession } from "./git.ts"; +import { objectSourceAt } from "./object-source.ts"; +import type { RepositoryHost } from "./host.ts"; + +/** Where one observation reads, as the provider authenticated it. */ +export interface ObservationPlace { + /** All six members of the authenticated Repository identity. */ + readonly repository: RepositoryIdentity; + /** The canonical root of the checkout the route selected. */ + readonly root: string; + /** That checkout's canonical common Git directory. */ + readonly commonDirectory: string; +} + +/** + * The variables every read states, and the reason each is stated. + * + * Lazy fetching off, so an object the checkout does not hold refuses instead of + * contacting a promisor remote. Optional locks off, so nothing this read does + * writes a lock file beside somebody's index. + */ +const READ_VARIABLES: Readonly> = { + GIT_NO_LAZY_FETCH: "1", + GIT_OPTIONAL_LOCKS: "0", +}; + +/** The pre-command options every read states, whatever it is reading. */ +const READ_OPTIONS: readonly string[] = [ + "--literal-pathspecs", + "--no-optional-locks", + "-c", + "gc.auto=0", + "-c", + "maintenance.auto=false", +]; + +/** The patch options every diff states, so none of them is configuration's. */ +const PATCH_OPTIONS: readonly string[] = [ + "--no-ext-diff", + "--no-textconv", + "--no-color", + "--unified=3", + "--diff-algorithm=myers", + "--src-prefix=a/", + "--dst-prefix=b/", + "--find-renames=50%", +]; + +/** The rename and status options every comparison states. */ +const STATUS_OPTIONS: readonly string[] = [ + "--name-status", + "-z", + "--no-ext-diff", + "--no-textconv", + "--find-renames=50%", +]; + +/** The tree every comparison uses in place of an unborn HEAD. */ +const EMPTY_TREE: Readonly> = { + sha1: "4b825dc642cb6eb9a060e54bf8d69288fbee4904", + sha256: "6ef19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321", +}; + +/** One observation's contained view, and the scratch it owns. */ +interface IsolatedView { + readonly host: RepositoryHost; + readonly element: string; + /** The scratch root this observation owns, which is also Git's `HOME`. */ + readonly scratch: string; + /** The bare control repository the contained objects are read through. */ + readonly control: string; + /** The authenticated checkout root. */ + readonly worktree: string; + /** That checkout's canonical common Git directory, which holds shared state. */ + readonly commonDirectory: string; + readonly format: GitObjectFormat; + /** The private index copy, prepared by the first read that needs one. */ + index(): Operation; +} + +function refuse( + view: IsolatedView, + reason: Parameters[1], + sentence: string, +): never { + throw refusalOf(view.element, reason, sentence); +} + +function refusalOf( + element: string, + reason: + | "invalid-revision" + | "absent-path" + | "nontext-blob" + | "unusable-repository" + | "stale-continuation", + sentence: string, +): GitObservationError { + return new GitObservationError(element, reason, `${element} ${sentence}`); +} + +function unreadable(view: IsolatedView, what: string): never { + return refuse( + view, + "unusable-repository", + `could not read ${what} from the selected repository. Nothing was changed; check that the ` + + "checkout is complete and readable, then read it again.", + ); +} + +/** One isolated read, with standard output decoded as text. */ +function* readText( + view: IsolatedView, + args: readonly string[], + options: { readonly index?: string; readonly worktree?: boolean } = {}, +): Operation<{ readonly code: number; readonly stdout: string }> { + const outcome = yield* view.host.git({ + args: isolated(view, args, options.worktree === true), + cwd: view.worktree, + home: view.scratch, + variables: variablesFor(options.index), + }); + return { code: outcome.code, stdout: outcome.stdout }; +} + +/** One isolated read, with standard output kept as the bytes Git wrote. */ +function* readBytes( + view: IsolatedView, + args: readonly string[], + options: { readonly index?: string; readonly worktree?: boolean } = {}, +): Operation<{ readonly code: number; readonly stdout: Uint8Array }> { + const capture = view.host.gitBytes; + if (capture === undefined) { + throw new GitOperationInfrastructureError( + view.element, + "the installed host lends no byte-preserving capture, and a path listing, a patch and a " + + "blob have to be decoded by this read rather than by the capture", + ); + } + const outcome = yield* capture({ + args: isolated(view, args, options.worktree === true), + cwd: view.worktree, + home: view.scratch, + variables: variablesFor(options.index), + }); + return { code: outcome.code, stdout: outcome.stdout }; +} + +function variablesFor(index: string | undefined): Readonly> { + return index === undefined ? READ_VARIABLES : { ...READ_VARIABLES, GIT_INDEX_FILE: index }; +} + +/** + * One command's complete argument list, stated rather than inherited. + * + * The control repository is the Git directory; the selected root is the working + * tree only where a comparison is about the working tree, which is the one read + * that has one. Everything else runs against objects and an index alone. + */ +function isolated(view: IsolatedView, args: readonly string[], worktree: boolean): string[] { + return [ + ...READ_OPTIONS, + `--git-dir=${view.control}`, + ...(worktree ? [`--work-tree=${view.worktree}`, "-c", "core.bare=false"] : []), + ...args, + ]; +} + +/** + * The full commit id this revision names in the authenticated checkout, or + * `undefined` when it names none. + * + * In the checkout, because that is where the references are; with lazy fetching + * disabled, so resolving one reaches no remote. Nothing a repository's + * configuration can name runs for a `rev-parse`. + */ +function* resolveRevision(view: IsolatedView, revision: string): Operation { + const outcome = yield* view.host.git({ + args: [ + ...READ_OPTIONS, + "rev-parse", + "--verify", + "--quiet", + "--end-of-options", + `${revision}^{commit}`, + ], + cwd: view.worktree, + home: view.scratch, + variables: READ_VARIABLES, + }); + if (outcome.code !== 0) { + return undefined; + } + const resolved = outcome.stdout.trim(); + return resolved === "" ? undefined : resolved; +} + +/** + * The index this checkout actually uses, copied into this observation's scratch. + * + * Git's own path query rather than `/index`: a linked worktree has + * an index of its own, and reading the primary one would answer a question + * about another checkout. A split index needs the shared-index bytes too, and + * they live beside the control repository so Git finds them where it looks — + * the source is read and never refreshed, expanded or repaired. + */ +function* copyIndex(view: IsolatedView): Operation { + const reported = yield* view.host.git({ + args: [...READ_OPTIONS, "rev-parse", "--git-path", "index"], + cwd: view.worktree, + home: view.scratch, + variables: READ_VARIABLES, + }); + const path = reported.stdout.trim(); + if (reported.code !== 0 || path === "") { + return unreadable(view, "the location of its index"); + } + const source = path.startsWith("/") ? path : `${view.worktree}/${path}`; + const copy = `${view.scratch}/index`; + // Beside the copy, not beside the control repository: Git resolves a split + // index's shared data relative to the index file it is reading, so a copy + // whose shared bytes sat anywhere else would be an index it cannot open. Both + // directories are searched because a linked worktree keeps its own shared + // index under its own administration while the primary keeps one in the + // common directory. + const separator = source.lastIndexOf("/"); + const directories = new Set([ + separator > 0 ? source.slice(0, separator) : view.commonDirectory, + view.commonDirectory, + ]); + try { + yield* copyIndexFile(source, copy); + for (const directory of directories) { + for (const name of yield* readdir(directory)) { + if (name.startsWith("sharedindex.")) { + yield* copyIndexFile(`${directory}/${name}`, `${view.scratch}/${name}`); + } + } + } + } catch { + return unreadable(view, "its index"); + } + return copy; +} + +/** + * Copy one index file, keeping the times the original carried. + * + * The times are part of what an index means. Git decides whether a cached + * entry can be trusted by comparing the entry's recorded modification time + * with the *index file's* own — an entry no older than the index it sits in is + * "racily clean", and Git compares the file's content rather than believing the + * cache. A copy stamped with the current time makes every such entry look + * reliably clean, so a working-tree edit made in the same second as the commit + * that recorded it would be reported as no change at all. + * + * `utimes` rather than an `@effectionx/fs` operation, which has none: the + * promise is adapted here, in the adapter that owns the host call. + */ +function* copyIndexFile(source: string, destination: string): Operation { + const info = yield* lstat(source); + yield* copyFile(source, destination); + yield* until(utimes(destination, info.atime, info.mtime)); +} + +/** + * The NUL-framed records this output holds, or the read's own refusal. + * + * A truncated response is not an empty observation. Git's `-z` output ends with + * a separator after the last record, so bytes after it mean the command stopped + * part way through — and answering a document with the prefix would report a + * listing that was never complete. + */ +function framed(view: IsolatedView, output: Uint8Array, what: string): readonly string[] { + if (output.length === 0) { + return Object.freeze([]); + } + if (output[output.length - 1] !== 0) { + return unreadable(view, `a complete ${what}`); + } + const decoder = new TextDecoder("utf-8", { fatal: true }); + const records: string[] = []; + let start = 0; + for (let index = 0; index < output.length; index += 1) { + if (output[index] !== 0) { + continue; + } + try { + records.push(decoder.decode(output.subarray(start, index))); + } catch { + return unreadable(view, `a ${what} this host can decode`); + } + start = index + 1; + } + return Object.freeze(records); +} + +/** What this comparison resolves to, once, for this observation. */ +function* resolveComparison( + view: IsolatedView, + request: GitComparisonRequest, +): Operation { + if (request.compare === "unstaged") { + return Object.freeze({ compare: "unstaged" as const }); + } + if (request.compare === "staged") { + // An unborn branch has no HEAD commit, and that is ordinary Git rather than + // a refusal: the comparison is against the empty tree and every staged path + // is an addition. Nothing invents a commit for it. + const head = yield* resolveRevision(view, "HEAD"); + return Object.freeze({ compare: "staged" as const, base: head ?? null }); + } + const base = yield* resolveRevision(view, request.base); + if (base === undefined) { + return refuse( + view, + "invalid-revision", + `was given base=${JSON.stringify(request.base)}, which names no commit in the selected ` + + "repository. Write a revision that does, then read it again.", + ); + } + const head = yield* resolveRevision(view, request.head); + if (head === undefined) { + return refuse( + view, + "invalid-revision", + `was given head=${JSON.stringify(request.head)}, which names no commit in the selected ` + + "repository. Write a revision that does, then read it again.", + ); + } + return Object.freeze({ compare: "commits" as const, base, head }); +} + +/** The arguments one comparison's `diff` takes, beside its own output options. */ +function comparisonArgs( + view: IsolatedView, + comparison: GitComparison, +): { readonly args: readonly string[]; readonly worktree: boolean } { + if (comparison.compare === "unstaged") { + // The one comparison about the working tree, so the one that names one. + // `--ignore-submodules=dirty` keeps a tracked gitlink's changed HEAD + // pointer visible through Git's own read while leaving the contents of a + // nested checkout — and anything its configuration would run — alone. + return { args: ["diff", "--ignore-submodules=dirty"], worktree: true }; + } + if (comparison.compare === "staged") { + return { + args: ["diff", "--cached", comparison.base ?? EMPTY_TREE[view.format]], + worktree: false, + }; + } + return { args: ["diff", comparison.base, comparison.head], worktree: false }; +} + +/** + * The change records this comparison produced, complete and normalized. + * + * Git's own letters, with a rename's similarity score removed and its previous + * path kept. A status this host has no letter for is not dropped: a read that + * silently omitted one would report a comparison as smaller than it is. + */ +function* observeChanges( + view: IsolatedView, + comparison: GitComparison, +): Operation { + const shape = comparisonArgs(view, comparison); + const index = comparison.compare === "commits" ? undefined : yield* view.index(); + const outcome = yield* readBytes(view, [...shape.args, ...STATUS_OPTIONS, "--"], { + worktree: shape.worktree, + ...(index === undefined ? {} : { index }), + }); + if (outcome.code !== 0) { + return unreadable(view, "what this comparison changed"); + } + const records = framed(view, outcome.stdout, "comparison"); + const changes: GitChange[] = []; + const unmerged = new Set(); + let at = 0; + while (at < records.length) { + const letter = records[at]; + if (letter === undefined || letter === "") { + return unreadable(view, "a complete comparison"); + } + const status = parseStatus(letter); + if (status === undefined) { + return refuse( + view, + "unusable-repository", + `read a change this host has no word for in the selected comparison. Nothing was ` + + "changed, and no part of the comparison is reported as though it were complete.", + ); + } + const renamed = status === "R"; + const previous = renamed ? records[at + 1] : undefined; + const path = renamed ? records[at + 2] : records[at + 1]; + at += renamed ? 3 : 2; + if (path === undefined || (renamed && previous === undefined)) { + return unreadable(view, "a complete comparison"); + } + if (status === "U") { + unmerged.add(path); + } + changes.push( + Object.freeze({ path, status, previousPath: renamed ? (previous ?? null) : null }), + ); + } + // One record per unmerged path. Git reports an unmerged path twice — once as + // unmerged and once comparing a stage of it with the working tree — and a + // document reading two records for one path would read one conflict as two + // facts. The unmerged one is the one that describes the state. + if (unmerged.size === 0) { + return sortedChanges(changes); + } + const collapsed: GitChange[] = []; + const reported = new Set(); + for (const change of changes) { + if (!unmerged.has(change.path)) { + collapsed.push(change); + continue; + } + if (change.status !== "U" || reported.has(change.path)) { + continue; + } + reported.add(change.path); + collapsed.push(change); + } + return sortedChanges(collapsed); +} + +/** + * The status this letter names, with a rename's similarity score removed. + * + * Copy detection and rewrite breaking are both off, so `C` and `B` are letters + * this host does not produce and does not read back as something else. + */ +function parseStatus(letter: string): GitChangeStatus | undefined { + const head = letter[0]; + if (head === "R") { + return /^R[0-9]*$/.test(letter) ? "R" : undefined; + } + if (letter.length !== 1) { + return undefined; + } + if (head === "A" || head === "M" || head === "D" || head === "T" || head === "U") { + return head; + } + return undefined; +} + +/** The complete patch text this comparison and these filters select. */ +function* observePatch( + view: IsolatedView, + comparison: GitComparison, + filters: readonly string[] | null, +): Operation { + const shape = comparisonArgs(view, comparison); + const index = comparison.compare === "commits" ? undefined : yield* view.index(); + // The pathspecs are the literal paths the status collection actually selected, + // both sides of a rename included. Passing the authored filters would hand a + // directory selector to Git as a pathspec; passing none, when some were + // written, would select everything. + let pathspecs: readonly string[] | undefined; + if (filters !== null) { + const selected: string[] = []; + for (const change of yield* observeChanges(view, comparison)) { + if (!selectedChange(filters, change)) { + continue; + } + selected.push(change.path); + if (change.previousPath !== null) { + selected.push(change.previousPath); + } + } + if (selected.length === 0) { + return ""; + } + pathspecs = Object.freeze([...new Set(selected)]); + } + const outcome = yield* readBytes( + view, + [...shape.args, ...PATCH_OPTIONS, "--", ...(pathspecs ?? [])], + { worktree: shape.worktree, ...(index === undefined ? {} : { index }) }, + ); + if (outcome.code !== 0) { + return unreadable(view, "the patch this comparison produces"); + } + try { + return new TextDecoder("utf-8", { fatal: true }).decode(outcome.stdout); + } catch { + return unreadable(view, "a patch this host can decode"); + } +} + +/** The tracked paths this checkout's index holds, sorted and deduplicated. */ +function* observeTrackedPaths(view: IsolatedView): Operation { + const index = yield* view.index(); + const outcome = yield* readBytes(view, ["ls-files", "--cached", "--full-name", "-z", "--"], { + index, + }); + if (outcome.code !== 0) { + return unreadable(view, "its tracked files"); + } + // One path per entry: an unmerged path has an index entry per stage, and a + // listing that repeated it would report a file three times. + const listed = new Set(framed(view, outcome.stdout, "file listing")); + return Object.freeze( + [...listed].sort((left, right) => (left < right ? -1 : left > right ? 1 : 0)), + ); +} + +/** The checkout one cursor is built against and held to. */ +function checkoutOf(place: ObservationPlace): GitObservationCheckout { + return Object.freeze({ + repository: place.repository, + root: place.root, + commonDirectory: place.commonDirectory, + }); +} + +/** + * The refusal a continuation that cannot be followed raises. + * + * One sentence for every reason a cursor stops describing this observation — + * changed data, a malformed or wrong-kind cursor, another checkout — because + * what the caller does about all of them is the same thing: read again from the + * beginning. No partial page is bound and no two observations are combined. + */ +function staleContinuation(element: string): GitObservationError { + const restart = + element === "" + ? `Restart ${element} without cursor and offset={0}.` + : `Restart ${element} without cursor.`; + return new GitObservationError( + element, + "stale-continuation", + `The Git observation changed. ${restart}`, + ); +} + +/** + * The position this read starts at, having held any cursor to this observation. + * + * The cursor is verified against a route and an observation this read computed + * independently: the kind it is for, the query it asks, the checkout the + * provider authenticated, and the digest of the complete filtered answer. None + * of it is used to decide where to read — only whether this is a page of the + * same answer. + */ +function* continuedPosition( + view: IsolatedView, + place: ObservationPlace, + kind: GitPagedKind, + request: GitObservationRequest, + comparison: GitComparison | null, + observation: string, + cursor: string | null, +): Operation { + if (cursor === null) { + return 0; + } + const parsed: GitObservationCursor | undefined = parseGitObservationCursor(cursor); + if (parsed === undefined || parsed.kind !== kind) { + throw staleContinuation(view.element); + } + const query = yield* observationDigest(gitObservationQuery(request, comparison)); + const recorded = yield* observationDigest(parsed.query); + if (query !== recorded) { + throw staleContinuation(view.element); + } + if (!sameObservationCheckout(parsed.checkout, checkoutOf(place))) { + throw staleContinuation(view.element); + } + if (parsed.observation !== observation) { + throw staleContinuation(view.element); + } + return parsed.position; +} + +function* cursorFor( + place: ObservationPlace, + kind: GitPagedKind, + request: GitObservationRequest, + comparison: GitComparison | null, + observation: string, + position: number, +): Operation { + return encodeGitObservationCursor({ + version: 1, + kind, + query: gitObservationQuery(request, comparison), + checkout: checkoutOf(place), + comparison, + observation, + position, + }); +} + +function* observeFiles( + view: IsolatedView, + request: GitFilesRequest, + place: ObservationPlace, +): Operation { + const tracked = yield* observeTrackedPaths(view); + const selected = + request.prefix === null + ? tracked + : Object.freeze(tracked.filter((path) => path.startsWith(request.prefix ?? ""))); + const observation = yield* observationDigest({ kind: "files", paths: [...selected] }); + const start = yield* continuedPosition( + view, + place, + "files", + request, + null, + observation, + request.cursor, + ); + const window = gitRecordWindow(selected, start, request.limit); + if (window.nextIndex === null) { + return Object.freeze({ paths: window.page, next: null }); + } + const cursor = yield* cursorFor(place, "files", request, null, observation, window.nextIndex); + return Object.freeze({ + paths: window.page, + next: Object.freeze({ + ...(request.prefix === null ? {} : { prefix: request.prefix }), + limit: request.limit, + cursor, + }), + }); +} + +function* observeChangesPage( + view: IsolatedView, + request: GitChangesRequest, + place: ObservationPlace, +): Operation { + const comparison = yield* resolveComparison(view, request.comparison); + const observed = yield* observeChanges(view, comparison); + const selected = Object.freeze( + observed.filter((change) => selectedChange(request.paths, change)), + ); + const observation = yield* observationDigest({ + kind: "changes", + comparison: gitComparisonJson(comparison), + changes: selected.map((change) => ({ + path: change.path, + status: change.status, + previousPath: change.previousPath, + })), + }); + const start = yield* continuedPosition( + view, + place, + "changes", + request, + comparison, + observation, + request.cursor, + ); + const window = gitRecordWindow(selected, start, request.limit); + if (window.nextIndex === null) { + return Object.freeze({ comparison, changes: window.page, next: null }); + } + const cursor = yield* cursorFor( + place, + "changes", + request, + comparison, + observation, + window.nextIndex, + ); + return Object.freeze({ + comparison, + changes: window.page, + next: Object.freeze({ + ...gitComparisonNext(comparison), + ...(request.paths === null ? {} : { paths: Object.freeze([...request.paths]) }), + limit: request.limit, + cursor, + }), + }); +} + +function* observeDiffPage( + view: IsolatedView, + request: GitDiffRequest, + place: ObservationPlace, +): Operation { + const comparison = yield* resolveComparison(view, request.comparison); + const patch = yield* observePatch(view, comparison, request.paths); + const observation = yield* observationDigest({ + kind: "diff", + comparison: gitComparisonJson(comparison), + text: patch, + }); + // A cursor names the offset it was issued for, so a follow-up that moved the + // offset by hand is reading a page this continuation never described. + const position = yield* continuedPosition( + view, + place, + "diff", + request, + comparison, + observation, + request.cursor, + ); + if (request.cursor !== null && position !== request.offset) { + throw staleContinuation(view.element); + } + const window = gitTextWindow(patch, request.offset, request.limit); + if (window.nextOffset === null) { + return Object.freeze({ + comparison, + text: window.text, + offset: request.offset, + length: window.length, + next: null, + }); + } + const cursor = yield* cursorFor( + place, + "diff", + request, + comparison, + observation, + window.nextOffset, + ); + return Object.freeze({ + comparison, + text: window.text, + offset: request.offset, + length: window.length, + next: Object.freeze({ + ...gitComparisonNext(comparison), + ...(request.paths === null ? {} : { paths: Object.freeze([...request.paths]) }), + offset: window.nextOffset, + limit: request.limit, + cursor, + }), + }); +} + +function* observeFile( + view: IsolatedView, + request: GitFileRequest, + place: ObservationPlace, +): Operation { + const commit = yield* resolveRevision(view, request.commit); + if (commit === undefined) { + return refuse( + view, + "invalid-revision", + `was given commit=${JSON.stringify(request.commit)}, which names no commit in the ` + + "selected repository. Write a revision that does, then read it again.", + ); + } + // The tree entry before the object, because the two questions are different: + // whether the commit holds this path at all, and whether what it holds is a + // file. A gitlink's commit is not in this repository's object database, so + // asking the object first would report a path the tree does hold as absent. + const listed = yield* readText(view, [ + "ls-tree", + "-z", + "--full-name", + commit, + "--", + request.path, + ]); + if (listed.code !== 0) { + return unreadable(view, `the tree of commit ${commit}`); + } + const entry = treeEntry(listed.stdout, request.path); + if (entry.kind === "absent") { + return refuse( + view, + "absent-path", + `read commit ${commit}, which holds nothing at ${JSON.stringify(request.path)}. Read a ` + + "path that commit holds, or list them with .", + ); + } + if (entry.kind === "directory" || entry.type !== "blob") { + const what = entry.kind === "directory" ? "tree" : entry.type; + return refuse( + view, + "nontext-blob", + `read commit ${commit}, where ${JSON.stringify(request.path)} is a ${what} rather ` + + "than a file. A directory and a nested checkout's recorded commit are not text to read.", + ); + } + const read = yield* readBytes(view, ["cat-file", "blob", entry.object]); + if (read.code !== 0) { + return unreadable(view, `the file ${JSON.stringify(request.path)} at commit ${commit}`); + } + if (read.stdout.includes(0)) { + return refuse( + view, + "nontext-blob", + `read ${JSON.stringify(request.path)} at commit ${commit}, which holds a NUL byte, so it ` + + "is not text this read can window.", + ); + } + let content: string; + try { + content = new TextDecoder("utf-8", { fatal: true }).decode(read.stdout); + } catch { + return refuse( + view, + "nontext-blob", + `read ${JSON.stringify(request.path)} at commit ${commit}, which is not valid UTF-8, so ` + + "it is not text this read can window. Nothing was replaced or repaired.", + ); + } + const window = gitTextWindow(content, request.offset, request.limit); + return Object.freeze({ + commit, + path: request.path, + text: window.text, + offset: request.offset, + length: window.length, + next: + window.nextOffset === null + ? null + : Object.freeze({ + commit, + path: request.path, + offset: window.nextOffset, + limit: request.limit, + }), + }); +} + +/** What the commit holds at the exact path a read asked for. */ +type TreeEntry = + /** The entry at that path, with the object and type Git reported for it. */ + | { readonly kind: "entry"; readonly type: string; readonly object: string } + /** A directory: the path names no entry, and entries beneath it exist. */ + | { readonly kind: "directory" } + | { readonly kind: "absent" }; + +/** + * The entry this commit holds at exactly the path that was asked for. + * + * Each record is ` `, a tab, then the path. The path is + * what decides the answer, because the argument Git received is a *pathspec* + * rather than a lookup: `dir` matches the directory entry itself and `dir/` + * matches the things inside it. Taking the first record would answer a read of + * `dir/` with the blob at `dir/inside.txt` — a file the caller never named, + * presented under a path that is really a directory. + * + * So a path that matched records but is not one of them is a directory, which + * is refused as the tree it is; a path that matched nothing is absent. + */ +function treeEntry(output: string, path: string): TreeEntry { + let matched = false; + for (const record of output.split("\u0000")) { + const tab = record.indexOf("\t"); + if (tab < 0) { + continue; + } + matched = true; + if (record.slice(tab + 1) !== path) { + continue; + } + const fields = record.slice(0, tab).split(" "); + const type = fields[1]; + const object = fields[2]; + if (type === undefined || object === undefined) { + continue; + } + return { kind: "entry", type, object }; + } + return matched ? { kind: "directory" } : { kind: "absent" }; +} + +/** + * Read the selected checkout, once, and answer with the page that was asked for. + * + * Scoped, so the scratch directory, the control repository and every native + * child this read started are torn down before the answer exists. The caller + * publishes afterwards: a result that appeared while a child was still running + * would be a completion for work that had not finished. + */ +export function performGitObservation( + host: RepositoryHost, + element: string, + request: GitObservationRequest, + place: ObservationPlace, +): Operation { + return scoped(function* () { + const format = place.repository.objectFormat; + const scratch = yield* host.useDirectory(); + const source = objectSourceAt(host, scratch, place.commonDirectory, format, element); + // The containment proof runs before any object is read: everything Git may + // traverse has to resolve inside the database the provider authenticated. + const control = yield* source.ready(); + let index: string | undefined; + const view: IsolatedView = { + host, + element, + scratch, + control, + worktree: place.root, + commonDirectory: place.commonDirectory, + format, + *index(): Operation { + if (index === undefined) { + index = yield* copyIndex(view); + } + return index; + }, + }; + if (request.kind === "files") { + return yield* observeFiles(view, request, place); + } + if (request.kind === "changes") { + return yield* observeChangesPage(view, request, place); + } + if (request.kind === "diff") { + return yield* observeDiffPage(view, request, place); + } + return yield* observeFile(view, request, place); + }); +} diff --git a/packages/git/src/deno/composition/subprocess.ts b/packages/git/src/deno/composition/subprocess.ts index 7895602b4..ce526dbc4 100644 --- a/packages/git/src/deno/composition/subprocess.ts +++ b/packages/git/src/deno/composition/subprocess.ts @@ -1,12 +1,23 @@ /** * One child process, run to completion, as an operation. * - * Three callers need the same four things — a built environment rather than an - * inherited one, output read to the end rather than to the exit, a nonzero exit - * treated as an answer rather than a throw, and a cancellation that finishes - * rather than merely starts. Native Git is one of them, the credential broker - * is another and the Git-host login command is the third, so the discipline - * lives here and each of them says only what it is running. + * Several callers need the same four things — a built environment rather than + * an inherited one, output read to the end rather than to the exit, a nonzero + * exit treated as an answer rather than a throw, and a cancellation that + * finishes rather than merely starts. Native Git is one of them, the credential + * broker is another and the Git-host login command is the third, so the + * discipline lives here and each of them says only what it is running. + * + * ## Two captures, one discipline + * + * `runProcess` decodes standard output as UTF-8 with replacement, which is what + * every existing caller is written against. `runProcessBytes` keeps the bytes + * instead, because a path listing is NUL-framed and a blob may not be text at + * all: a replacement character substituted during capture would make a + * malformed file indistinguishable from one holding U+FFFD, and the read has to + * be able to tell those apart. Everything else — the built environment, the + * detached group, the `close` wait, the listener removal and the cancellation + * that joins rather than signals — is the one implementation below. */ import { ensure, type Operation, withResolvers } from "effection"; @@ -50,14 +61,37 @@ export interface ProcessInvocation { ) => void; } -export function* runProcess({ - command, - args, - cwd, - env, - input, - observe, -}: ProcessInvocation): Operation { +/** What one invocation reported, with standard output kept as bytes. */ +export interface ProcessBytes { + readonly code: number; + readonly stdout: Uint8Array; + /** Still decoded: a failing command explains itself here, and that is text. */ + readonly stderr: string; +} + +/** Run this command and decode its standard output as UTF-8 with replacement. */ +export function* runProcess(invocation: ProcessInvocation): Operation { + const captured = yield* run(invocation, "text"); + return { code: captured.code, stdout: captured.text, stderr: captured.stderr }; +} + +/** Run this command and keep its standard output as the bytes it wrote. */ +export function* runProcessBytes(invocation: ProcessInvocation): Operation { + const captured = yield* run(invocation, "bytes"); + return { code: captured.code, stdout: captured.bytes, stderr: captured.stderr }; +} + +interface Captured { + readonly code: number; + readonly text: string; + readonly bytes: Uint8Array; + readonly stderr: string; +} + +function* run( + { command, args, cwd, env, input, observe }: ProcessInvocation, + capture: "text" | "bytes", +): Operation { // `node:child_process` rather than the runtime's own global: this adapter is // selected by the host, not written against one, and `spawn` replaces the // child's environment outright when `env` is given — which is the whole point @@ -85,12 +119,31 @@ export function* runProcess({ // process ended; it does not say the pipes this operation inherited have. let closed = false; let stdout = ""; + const chunks: Uint8Array[] = []; let stderr = ""; + // Joined only once the command has closed, so a partial multi-byte character + // at a chunk boundary is never decoded as two. + const bytes = (): Uint8Array => { + let length = 0; + for (const chunk of chunks) { + length += chunk.length; + } + const joined = new Uint8Array(length); + let at = 0; + for (const chunk of chunks) { + joined.set(chunk, at); + at += chunk.length; + } + return joined; + }; // Declared after what it reads, so an observer may call it at once rather // than only from a later turn. - const captured = () => ({ stdout, stderr }); + const captured = () => ({ + stdout: capture === "text" ? stdout : new TextDecoder().decode(bytes()), + stderr, + }); - const outcome = withResolvers(); + const outcome = withResolvers(); // A pipe the command stops reading is the command's answer, and its exit // status is what says so — but the write still fails, and an unhandled @@ -100,8 +153,12 @@ export function* runProcess({ const onStdinError = (error: Error): void => { outcome.reject(error); }; - const onStdout = (chunk: string): void => { - stdout += chunk; + const onStdout = (chunk: string | Uint8Array): void => { + if (typeof chunk === "string") { + stdout += chunk; + return; + } + chunks.push(chunk); }; const onStderr = (chunk: string): void => { stderr += chunk; @@ -111,7 +168,7 @@ export function* runProcess({ // whatever had arrived when it stopped. const onExit = (code: number | null): void => { closed = true; - outcome.resolve({ code: code ?? -1, stdout, stderr }); + outcome.resolve({ code: code ?? -1, text: stdout, bytes: bytes(), stderr }); }; const onFailure = (error: Error): void => { outcome.reject(error); @@ -185,7 +242,12 @@ export function* runProcess({ observe?.(child, captured); - child.stdout.setEncoding("utf8"); + // Text mode decodes at the stream, which is what every existing caller reads. + // Byte mode deliberately does not: chunks arrive as buffers and are joined + // once, so nothing is substituted for a character split across two of them. + if (capture === "text") { + child.stdout.setEncoding("utf8"); + } child.stderr.setEncoding("utf8"); child.on("close", onExit); child.on("error", onFailure); diff --git a/packages/git/src/deno/retained-repository/git.ts b/packages/git/src/deno/retained-repository/git.ts index 67a32f3ec..82fa79928 100644 --- a/packages/git/src/deno/retained-repository/git.ts +++ b/packages/git/src/deno/retained-repository/git.ts @@ -122,7 +122,7 @@ import type { SlotKind } from "../run-composition/leases.ts"; import type { SelectionRegistry } from "../selections.ts"; import { holdManagedPlacement } from "./leases.ts"; import { retainedRecordAhead } from "./local.ts"; -import { placementSubject } from "./selections.ts"; +import { nativeGitTurn, placementSubject } from "./selections.ts"; import type { ReplPlacementRegistration } from "./selections.ts"; /** @@ -420,7 +420,7 @@ function* live( ): Operation { const target = yield* routing(); return yield* entry.serialize( - `git ${target.registration.placement.commonDirectory}`, + nativeGitTurn(target.registration.placement.commonDirectory), function* () { yield* own(entry, target); return yield* once(); diff --git a/packages/git/src/deno/retained-repository/observations.ts b/packages/git/src/deno/retained-repository/observations.ts new file mode 100644 index 000000000..2b765b9ea --- /dev/null +++ b/packages/git/src/deno/retained-repository/observations.ts @@ -0,0 +1,569 @@ +/** + * One REPL entry's four local Git reads: performed once, answered from what + * performing them produced. + * + * These are the ordinary performers an `xmd run` uses, wrapped in this entry's + * durable boundary. What is added is memory: a completed read answers with its + * saved page and runs no `git`, however the checkout has changed since — and a + * read whose checkout has been deleted still answers, because answering it + * consults nothing. + * + * ## A saved page is history, not a prediction + * + * The honesty the REPL trades for not snapshotting anybody's repository is + * explicit here. A completed listing names the paths the index held then; a + * completed patch window is the text that comparison produced then. Reopening + * the page performs no check against today's repository. Using the `next` it + * carries in a *new* read does, which is where a changed observation refuses — + * that is a live continuation check rather than something replay decides. + * + * ## Why this reads the cursor before it resolves anything + * + * Routing a read means canonicalizing its working directory and choosing among + * this entry's registered checkouts; performing one means creating a scratch + * directory, proving an object database contained and copying an index. None of + * that is anything a saved page is about. So the retained cursor is read first, + * without consuming anything, and only a read with nothing saved routes, + * acquires and performs. + * + * The read is a hint and never an authority. `createDurableOperation` remains + * the only judge of position, type and name, so the recording callback asks for + * the same routing and the same hold again — idempotently — around work that + * really does run live. + * + * ## One slot, no lease, one turn + * + * A live read takes the selected placement's managed slot where it has one, and + * nothing else: it changes no byte of the repository, so there is nothing for a + * mutation lease to protect. It does take this entry's existing per-common- + * directory turn, so the entry's own native Git attempts do not interleave each + * other's command sequences. + */ + +import { Err, Ok, scoped, until } from "effection"; +import type { Operation, Result } from "effection"; +import { realpath } from "node:fs/promises"; +import { createDurableOperation, StaleInputError } from "@executablemd/durable-streams"; +import type { + ActivateDurabilityFailure, + Json, + LiveDurableOperationCoordinator, + Result as ProtocolResult, +} from "@executablemd/durable-streams"; +import type { Around } from "@effectionx/context-api"; +import { GitObservationError, GitOperationAdmissionError } from "../../composition/errors.ts"; +import type { GitQueryApi } from "../../git.ts"; +import { + admitChangesInvocation, + admitDiffInvocation, + admitFileInvocation, + admitFilesInvocation, +} from "../../composition/observation-api.ts"; +import type { + GitChangesInvocation, + GitChangesResult, + GitDiffInvocation, + GitDiffResult, + GitFileInvocation, + GitFileResult, + GitFilesInvocation, + GitFilesResult, + GitObservationRequest, + GitObservationValue, +} from "../../composition/observation-api.ts"; +import { + CHANGES_ELEMENT, + DIFF_ELEMENT, + FILE_ELEMENT, + FILES_ELEMENT, +} from "../../composition/components/GitReads.ts"; +import { sameRepositoryIdentity } from "../../composition/selection.ts"; +import type { RepositorySelection } from "../../composition/selection.ts"; +import { + elementOf, + parseReplObservationRecord, + replObservationFingerprint, + replObservationRecordJson, + replObservationRecordName, + replObservationRefusalOf, + ReplObservationRefusal, + REPL_REPOSITORY_OBSERVATION, +} from "../../retained-repository/observation-records.ts"; +import type { + ReplObservationCheckout, + ReplObservationRecord, + ReplObservationRequest, +} from "../../retained-repository/observation-records.ts"; +import { performGitObservation } from "../composition/observations.ts"; +import type { ObservationPlace } from "../composition/observations.ts"; +import type { RepositoryHost } from "../composition/host.ts"; +import { selectCheckout } from "../run-composition/operations.ts"; +import { nativeGitTurn } from "./selections.ts"; +import type { SlotKind } from "../run-composition/leases.ts"; +import type { SelectionRegistry } from "../selections.ts"; +import { holdManagedPlacement } from "./leases.ts"; +import { retainedRecordAhead } from "./local.ts"; +import { placementSubject } from "./selections.ts"; +import type { ReplPlacementRegistration } from "./selections.ts"; +import type { ReplGitCheckout } from "./git.ts"; + +/** + * A saved page standing where this read belongs that it cannot read. + * + * Fatal: a value that does not describe this read is not an outcome of it, so + * it does not belong in the result channel. A refusal there is the read's own + * sentence, which `` may print and carry on past. + */ +export class ReplObservationRecordError extends StaleInputError { + override name = "ReplObservationRecordError"; +} + +/** What recorded local Git reads need from the entry that owns them. */ +export interface ReplObservationEntry { + /** The native Git host one contained observation performs through. */ + readonly host: RepositoryHost; + /** Hold a managed slot for the entry, single-flight per root and slot. */ + holdSlot(root: string, kind: SlotKind, path: string, subject: string): Operation; + /** Run live native Git for one shared Git directory in order within this entry. */ + serialize(key: string, work: () => Operation): Operation; + /** This entry's private map from the selections it minted to what it holds. */ + readonly selections: SelectionRegistry; + /** The checkouts this entry has registered, as they stand now. */ + checkouts(): readonly ReplGitCheckout[]; +} + +/** + * The recorded four reads this entry installs. + * + * Installed at the implementation slot, exactly where the ordinary provider + * installs its own, so this is ordinary dispatch rather than a protected + * channel: middleware above may answer without reaching here, refuse, or + * transform and delegate — and when it delegates, the request that arrives is + * the request recorded. A read answered wholly by a replacement never reaches + * this provider and creates no observation record. + */ +export function replObservationMiddleware( + entry: ReplObservationEntry, +): Partial> { + return { + *files([invocation]: [GitFilesInvocation]): Operation { + const request = admitFilesInvocation(FILES_ELEMENT, invocation.request); + const value = yield* recorded(entry, { ...invocation, request }); + if (!("paths" in value)) { + throw mismatch(FILES_ELEMENT); + } + return value; + }, + + *changes([invocation]: [GitChangesInvocation]): Operation { + const request = admitChangesInvocation(CHANGES_ELEMENT, invocation.request); + const value = yield* recorded(entry, { ...invocation, request }); + if (!("changes" in value)) { + throw mismatch(CHANGES_ELEMENT); + } + return value; + }, + + *diff([invocation]: [GitDiffInvocation]): Operation { + const request = admitDiffInvocation(DIFF_ELEMENT, invocation.request); + const value = yield* recorded(entry, { ...invocation, request }); + if (!("text" in value) || "commit" in value) { + throw mismatch(DIFF_ELEMENT); + } + return value; + }, + + *file([invocation]: [GitFileInvocation]): Operation { + const request = admitFileInvocation(FILE_ELEMENT, invocation.request); + const value = yield* recorded(entry, { ...invocation, request }); + if (!("commit" in value)) { + throw mismatch(FILE_ELEMENT); + } + return value; + }, + }; +} + +/** + * Where this read runs, as this entry's own registry describes it. + * + * The received selection is authenticated before anything else and its *parsed* + * facts are what the request carries. The fresh opaque identifier the provider + * minted for this entry never appears: it names nothing once the entry ends, so + * a durable name built from one would differ on every reopen of the same + * question. + */ +function place( + entry: ReplObservationEntry, + element: string, + invocation: { readonly repository: RepositorySelection; readonly workingDirectory: string }, + query: GitObservationRequest, +): ReplObservationRequest { + const placement = entry.selections.authenticate( + invocation.repository, + () => + new GitOperationAdmissionError( + element, + "the Repository in scope is not one this entry selected, so it names no checkout", + ), + ).placement; + return Object.freeze({ + repository: placement.identity, + selectionCheckout: placement.checkoutRoot, + commonDirectory: placement.commonDirectory, + // Exactly as received. Resolving it is live routing, which a completed read + // does not do, so the name cannot depend on it. + workingDirectory: invocation.workingDirectory, + query, + }); +} + +/** + * Perform this read, or answer with what performing it produced before. + * + * The page is the read's own — the same value a live call produces and a replay + * restores — so a caller cannot tell from it which of the two happened, which + * is the point. An ordinary refusal is raised from the saved classification on + * both paths, by this one route, so a live refusal and a replayed one are the + * same error for the same element. + */ +function* recorded( + entry: ReplObservationEntry, + invocation: { + readonly repository: RepositorySelection; + readonly workingDirectory: string; + readonly request: GitObservationRequest; + }, +): Operation { + const element = elementOf(invocation.request.kind); + const request = place(entry, element, invocation, invocation.request); + const fingerprint = yield* replObservationFingerprint(request); + const name = replObservationRecordName({ + operation: invocation.request.kind, + fingerprint, + }); + + // Resolved at most once for this call, because the live path routes before it + // takes the turn and the recording callback asks again inside it. + let routed: ReplGitCheckout | undefined; + function* routing(): Operation { + if (routed === undefined) { + routed = yield* route(entry, element, request); + } + return routed; + } + + function* once(): Operation { + return yield createDurableOperation( + { type: REPL_REPOSITORY_OBSERVATION, name }, + function* () { + const target = yield* routing(); + yield* own(entry, target); + const checkout = describing(target); + const outcome = yield* attempted(element, function* () { + return yield* performGitObservation( + entry.host, + element, + invocation.request, + placeOf(request, target), + ); + }); + const payload = replObservationRecordJson({ + fingerprint, + request, + checkout, + outcome, + }); + // Read back before the coordinator publishes it. A payload this build + // cannot parse, or one describing another request or a checkout this + // entry holds no placement for, is not an outcome to acknowledge: it + // takes the fatal path here, so nothing is appended and a continuation + // meets today's checkout instead of a record it could never read. + readback(entry, element, request, fingerprint, payload); + return payload; + }, + { coordinator: observationCoordinator }, + ); + } + + // Journal data on a replay, so it arrives as the unknown it is and is parsed + // below rather than trusted as this code's own. + const stored: unknown = (yield* retainedRecordAhead(REPL_REPOSITORY_OBSERVATION, name)) + ? yield* once() + : yield* live(entry, routing, once); + + const parsed = readback(entry, element, request, fingerprint, stored); + if (!parsed.outcome.ok) { + throw failure(element, parsed.outcome.error); + } + return parsed.outcome.value; +} + +/** Where the contained native read runs, once this entry has routed it. */ +function placeOf(request: ReplObservationRequest, target: ReplGitCheckout): ObservationPlace { + return { + repository: request.repository, + root: target.checkout.root, + commonDirectory: target.registration.placement.commonDirectory, + }; +} + +/** The checkout a record names, derived from the route rather than the journal. */ +function describing(target: ReplGitCheckout): ReplObservationCheckout { + return Object.freeze({ + root: target.checkout.root, + commonDirectory: target.registration.placement.commonDirectory, + repositoryName: target.checkout.repositoryName, + worktreeName: target.checkout.worktreeName, + }); +} + +/** + * The record this value is for this read, or the fatal failure it is not. + * + * The same three questions on both paths, which is the point: a live outcome is + * held to them before it is published, and a restored one before it is + * returned. The payload parses, it describes this request, and the checkout it + * names is one this entry registered under the repository it authenticated — + * with parsed facts alone, so a result path never authorizes a native call and + * a new read chooses its checkout independently. + */ +function readback( + entry: ReplObservationEntry, + element: string, + request: ReplObservationRequest, + fingerprint: string, + stored: unknown, +): ReplObservationRecord { + const parsed = parseReplObservationRecord(stored); + if (parsed === undefined) { + throw new ReplObservationRecordError( + `the saved page for ${element} is not one this host can read, so what it says about work ` + + "already done is unknown.", + ); + } + if (parsed.operation !== request.query.kind || parsed.fingerprint !== fingerprint) { + throw mismatch(element); + } + authorized(entry, element, request, parsed.checkout); + return parsed; +} + +/** + * Route, take the turn, take the slot, and perform. + * + * Acquisition happens outside the recorded operation, which is what keeps a + * slot another entry holds a fatal diagnostic the person can act on rather than + * a durable outcome — and keeps temporary contention out of the journal + * entirely. + * + * The turn is `nativeGitTurn()`, the same one this entry's Add, Switch, Commit + * and publication take for the same shared Git directory. A read runs several + * Git commands against one private index copy and one pair of resolved commit + * ids, and a mutation of this entry's own entering between them would move the + * repository underneath a sequence already in progress. A key of this read's + * own would be a second queue: the serializer is shared, but a turn is the key. + * + * It is held for the whole of `once()` — the native work, its joined teardown + * and the coordinator's acknowledged append — so the turn ends when the + * observation is recorded rather than when Git stops running. Cross-entry + * independence is unaffected: this queue is per entry, and another entry's + * mutation lease is a different mechanism. + */ +function* live( + entry: ReplObservationEntry, + routing: () => Operation, + once: () => Operation, +): Operation { + const target = yield* routing(); + return yield* entry.serialize( + nativeGitTurn(target.registration.placement.commonDirectory), + function* () { + yield* own(entry, target); + return yield* once(); + }, + ); +} + +/** + * Which registered checkout this repository and working directory select. + * + * The ordinary selector over this entry's own registrations, restricted by the + * whole authenticated Repository identity and resolving the received directory + * first — a checkout root is the path Git resolved, and a working directory + * reached through a link is the same place under another name. The longest + * covering checkout wins, so a subdirectory, a linked worktree and a captured + * path entered through `` each select the checkout their own paths belong + * to. + */ +function* route( + entry: ReplObservationEntry, + element: string, + request: ReplObservationRequest, +): Operation { + const registered = entry.checkouts(); + const selected = selectCheckout( + registered.map((held) => held.checkout), + request.repository, + yield* canonicalPath(request.workingDirectory), + element, + ); + const found = registered.find((held) => held.checkout.root === selected.root); + if (found === undefined) { + throw new GitOperationAdmissionError( + element, + "the checkout it selected is not one this entry holds a placement for", + ); + } + return found; +} + +/** + * Take the hold a read needs, which is the managed slot and nothing else. + * + * The slot says which entry owns a managed checkout, and a restored placement + * took none when it replayed. No mutation lease: a read changes nothing in the + * repository, so there is nothing for one to protect — and taking one would + * make reading a checkout block an editor committing in it. + */ +function* own(entry: ReplObservationEntry, target: ReplGitCheckout): Operation { + const registration = target.registration; + const slot = registration.slot; + // The root this placement recorded, not the one configured now: a restored + // placement's slot belongs to the root it was selected under. + const root = registration.placement.managedRoot; + if (slot !== undefined && root !== undefined) { + yield* holdManagedPlacement(() => + entry.holdSlot(root, slot.kind, slot.path, placementSubject(registration.placement)), + ); + } +} + +/** + * Run the live read, or classify its refusal as one this record may carry. + * + * Joined, so every per-call native child has finished and closed and the + * scratch it worked in has been removed before the outcome exists. A classified + * refusal is an answer the document asked for and is written down. Anything + * else is infrastructure: it publishes no completion, so a continuation reaches + * the ordinary live path rather than replaying a verdict this host has no word + * for. + */ +function* attempted( + element: string, + perform: () => Operation, +): Operation> { + try { + return Ok(yield* scoped(perform)); + } catch (error) { + const refusal = replObservationRefusalOf(error); + if (refusal === undefined) { + throw error; + } + return Err(new ReplObservationRefusal(element, refusal.reason, refusal.sentence)); + } +} + +/** + * The failure a refused read reaches the document as. + * + * Built here on a live refusal and on a replayed one alike, so the two cannot + * diverge: what the record carries is a word and this host's own sentence, and + * this is the one place either of them becomes an error a component shows. + */ +function failure(element: string, error: Error): Error { + if (!(error instanceof ReplObservationRefusal)) { + return new ReplObservationRecordError( + `the saved refusal for ${element} carries no closed classification, so what it refused is ` + + "unknown.", + ); + } + return new GitObservationError(element, error.reason, error.message); +} + +/** + * That the checkout a saved page names is one this entry registered. + * + * Pure arithmetic over parsed facts. The identity has to be the one this call + * authenticated, and the root, the common directory and the worktree name have + * to be a placement this entry holds. What it deliberately does not require is + * that the page name the selection's own checkout: a lexical `` can + * own several worktrees, and a `` into one of them selects that one. + * + * Effective-cwd selection is checked on the live path, where a directory is + * resolved. A replay does not resolve an old alias again to re-prove routing + * that already happened. + */ +function authorized( + entry: ReplObservationEntry, + element: string, + request: ReplObservationRequest, + checkout: ReplObservationCheckout, +): void { + for (const candidate of entry.checkouts()) { + if ( + candidate.checkout.root === checkout.root && + candidate.registration.placement.commonDirectory === checkout.commonDirectory && + candidate.checkout.repositoryName === checkout.repositoryName && + candidate.checkout.worktreeName === checkout.worktreeName && + sameRepositoryIdentity(candidate.checkout.identity, request.repository) + ) { + return; + } + } + throw new ReplObservationRecordError( + `the saved page for ${element} names a checkout this entry holds no placement for, so this ` + + "history does not say where the read ran.", + ); +} + +function mismatch(element: string): ReplObservationRecordError { + return new ReplObservationRecordError( + `the saved page standing where ${element} belongs describes another read, so this history ` + + "does not say what happened here.", + ); +} + +/** The canonical directory this path resolves to, or the path as written. */ +function* canonicalPath(path: string): Operation { + try { + return yield* until(realpath(path)); + } catch { + return path; + } +} + +/** + * The live boundary one recorded read publishes through. + * + * The same contract this feature's file work, its selections and its local Git + * publish under. The default coordinator would serialize whatever was thrown + * and publish it as this read's outcome, which is the one thing that must not + * happen: a performer that broke its contract may or may not have finished, so + * recording a completion for it would let a continuation skip a read that never + * answered. + * + * A throw therefore activates the durability fence and is re-raised with its + * identity intact, publishing nothing — and a continuation takes the ordinary + * live path, where today's repository decides what happens next. A valid + * outcome, page or classified refusal, is published once and returned only + * after the append is acknowledged. A failing append is the engine's own + * persistence fence and is not caught here. + */ +const observationCoordinator: LiveDurableOperationCoordinator = { + *run( + execute: () => Operation, + publish: (result: ProtocolResult) => Operation, + activateFailure: ActivateDurabilityFailure, + ): Operation { + let value: T; + try { + value = yield* execute(); + } catch (error) { + throw activateFailure(error); + } + const result: ProtocolResult = { status: "ok", value }; + yield* publish(result); + return result; + }, +}; diff --git a/packages/git/src/deno/retained-repository/provider.ts b/packages/git/src/deno/retained-repository/provider.ts index 44fbbb20e..a2466dcbd 100644 --- a/packages/git/src/deno/retained-repository/provider.ts +++ b/packages/git/src/deno/retained-repository/provider.ts @@ -21,9 +21,10 @@ * selects to do it in, the three local Git operations — authored and generated * writes, authored deletions, the directory an authored `` ensures, the * `` and `` selections that say where, and ``, - * `` and `` in the checkout those select — and, for an - * entry whose capability was built per entry below, the publication an authored - * `` asks for. + * `` and `` in the checkout those select — the four + * local Git reads ``, ``, `` and + * ``, and, for an entry whose capability was built per entry below, + * the publication an authored `` asks for. * * A pull request and an Issue tracker are not here: each has its own protocol * and its own evidence, and a local result and a publication authorize neither. @@ -76,6 +77,9 @@ import { replFileMiddleware } from "./files.ts"; import type { ReplLocalRecorder, ReplManagedPlacement } from "./files.ts"; import { replGitMiddleware } from "./git.ts"; import type { ReplGitCheckout, ReplGitEntry } from "./git.ts"; +import { replObservationMiddleware } from "./observations.ts"; +import type { ReplObservationEntry } from "./observations.ts"; +import { GitQuery } from "../../git.ts"; import { replPushAdmission, replPushComponent } from "./push.ts"; import type { ReplPushEntry } from "./push.ts"; import { replIssueInstallation } from "./issues.ts"; @@ -512,6 +516,19 @@ function* initializeReplRepository( // in an ordinary run. yield* Git.around(replGitMiddleware(gitEntry), { at: "min" }); + // The four reads, at the same slot and on the same terms. They share this + // entry's registry, its registered checkouts, its slot acquisition and its + // native turn, and take no mutation lease: a read changes nothing, so there + // is nothing for one to protect. + const observationEntry: ReplObservationEntry = { + host, + holdSlot, + serialize, + selections, + checkouts: () => gitCheckouts, + }; + yield* GitQuery.around(replObservationMiddleware(observationEntry), { at: "min" }); + // Last, and only for the route that asked: everything above is in place, so // what a publication reaches is an initialized entry rather than a // half-assembled one. An entry with no managed root returns before this, diff --git a/packages/git/src/deno/retained-repository/push.ts b/packages/git/src/deno/retained-repository/push.ts index 2c1627fdf..f41a4a515 100644 --- a/packages/git/src/deno/retained-repository/push.ts +++ b/packages/git/src/deno/retained-repository/push.ts @@ -135,7 +135,7 @@ import type { SelectionRegistry } from "../selections.ts"; import { holdManagedPlacement } from "./leases.ts"; import type { ReplGitCheckout } from "./git.ts"; import { retainedRecordAhead } from "./local.ts"; -import { placementSubject } from "./selections.ts"; +import { nativeGitTurn, placementSubject } from "./selections.ts"; import type { ReplPlacementRegistration } from "./selections.ts"; /** @@ -416,7 +416,7 @@ function* resolve(entry: ReplPushEntry, received: ReplPushPlace): Operation` of the same repository would describe a moment neither of // them is at. The turn is held for these two short reads and released before // anything is published: a network round trip never holds it. - const read = yield* entry.serialize(`git ${placement.commonDirectory}`, function* () { + const read = yield* entry.serialize(nativeGitTurn(placement.commonDirectory), function* () { const checkout = liveCheckout(git, routed.checkout, received.workingDirectory); const branch = yield* currentBranch(checkout.git, checkout.directory); if (branch === undefined) { diff --git a/packages/git/src/deno/retained-repository/selections.ts b/packages/git/src/deno/retained-repository/selections.ts index 18af8f51f..90b074abc 100644 --- a/packages/git/src/deno/retained-repository/selections.ts +++ b/packages/git/src/deno/retained-repository/selections.ts @@ -807,6 +807,20 @@ const placementCoordinator: LiveDurableOperationCoordinator = { * rather than stranding it, which is the one case the two cleanups have to tell * apart. */ +/** + * The one turn an entry's native Git for one shared Git directory takes. + * + * Named here rather than spelled at each caller, because the key *is* the + * exclusion: every operation of this entry that runs `git` against one common + * directory — Add, Switch, Commit, the publication's observation, and the four + * reads — has to name the same string. A caller that spelled its own would be + * handed its own queue by a serializer the two were sharing, which looks like + * participation and is not. + */ +export function nativeGitTurn(commonDirectory: string): string { + return `git ${commonDirectory}`; +} + export function slotTurns(): (key: string, work: () => Operation) => Operation { interface Waiting { readonly granted: WithResolvers; diff --git a/packages/git/src/deno/run-composition/provider.ts b/packages/git/src/deno/run-composition/provider.ts index b5469c639..781bcce4a 100644 --- a/packages/git/src/deno/run-composition/provider.ts +++ b/packages/git/src/deno/run-composition/provider.ts @@ -2,9 +2,9 @@ * The ordinary run's repository provider: what `xmd run` installs under Deno * and inside the compiled binary. * - * It answers the same four Apis the workflow provider answers, so a document - * writes the same thirteen components either way. What differs is everything - * about lifetime and ownership. + * It answers the same Apis the workflow provider answers, so a document writes + * the same repository vocabulary either way — the four selected-checkout reads + * included. What differs is everything about lifetime and ownership. * * A workflow run's checkouts are rows in its own database, restored from * retained history under a WorkflowRun the document must never be able to name. @@ -47,13 +47,42 @@ import { getExpansion } from "@executablemd/core"; import { Repository } from "../../composition/api.ts"; import type { RepositoryRequest, WorktreeRequest } from "../../composition/api.ts"; import { Git } from "../../composition/git-api.ts"; +import { GitQuery } from "../../git.ts"; +import { + admitChangesInvocation, + admitDiffInvocation, + admitFileInvocation, + admitFilesInvocation, +} from "../../composition/observation-api.ts"; +import type { + GitChangesInvocation, + GitChangesResult, + GitDiffInvocation, + GitDiffResult, + GitFileInvocation, + GitFileResult, + GitFilesInvocation, + GitFilesResult, +} from "../../composition/observation-api.ts"; +import { + CHANGES_ELEMENT, + DIFF_ELEMENT, + FILE_ELEMENT, + FILES_ELEMENT, +} from "../../composition/components/GitReads.ts"; +import { performGitObservation } from "../composition/observations.ts"; +import type { ObservationPlace } from "../composition/observations.ts"; import type { GitAddInvocation, GitCommitInvocation, GitPushInvocation, GitSwitchInvocation, } from "../../composition/git-api.ts"; -import { GitOperationAdmissionError, RepositorySelectionError } from "../../composition/errors.ts"; +import { + GitOperationAdmissionError, + GitOperationProtocolError, + RepositorySelectionError, +} from "../../composition/errors.ts"; import type { GitAddResult, GitCommitResult, @@ -328,6 +357,23 @@ export function* useRunComposition(options: RunCompositionOptions): Operation { + return (yield* routed(invocationRepository, workingDirectory, operation)).checkout; + } + + /** + * Which registered checkout this invocation selects, and the Git directory + * that checkout shares. + * + * The common directory comes from the authenticated selection rather than + * from the route, because every checkout of one repository shares it and the + * selection is what was authenticated. A read needs it: the objects it may + * traverse are the ones beneath that directory and nothing else. + */ + function* routed( + invocationRepository: RepositorySelection, + workingDirectory: string, + operation: string, + ): Operation<{ readonly checkout: RegisteredCheckout; readonly commonDirectory: string }> { const selected = selections.authenticate( invocationRepository, () => @@ -341,12 +387,29 @@ export function* useRunComposition(options: RunCompositionOptions): Operation { + const target = yield* routed(invocationRepository, workingDirectory, element); + return { + repository: target.checkout.identity, + root: target.checkout.root, + commonDirectory: target.commonDirectory, + }; } yield* Git.around( @@ -412,6 +475,87 @@ export function* useRunComposition(options: RunCompositionOptions): Operation { + const request = admitFilesInvocation(FILES_ELEMENT, invocation_.request); + const value = yield* performGitObservation( + host, + FILES_ELEMENT, + request, + yield* observationPlace( + invocation_.repository, + invocation_.workingDirectory, + FILES_ELEMENT, + ), + ); + if (!("paths" in value)) { + throw answered(FILES_ELEMENT); + } + return value; + }, + + *changes([invocation_]: [GitChangesInvocation]): Operation { + const request = admitChangesInvocation(CHANGES_ELEMENT, invocation_.request); + const value = yield* performGitObservation( + host, + CHANGES_ELEMENT, + request, + yield* observationPlace( + invocation_.repository, + invocation_.workingDirectory, + CHANGES_ELEMENT, + ), + ); + if (!("changes" in value)) { + throw answered(CHANGES_ELEMENT); + } + return value; + }, + + *diff([invocation_]: [GitDiffInvocation]): Operation { + const request = admitDiffInvocation(DIFF_ELEMENT, invocation_.request); + const value = yield* performGitObservation( + host, + DIFF_ELEMENT, + request, + yield* observationPlace( + invocation_.repository, + invocation_.workingDirectory, + DIFF_ELEMENT, + ), + ); + if (!("text" in value) || "commit" in value) { + throw answered(DIFF_ELEMENT); + } + return value; + }, + + *file([invocation_]: [GitFileInvocation]): Operation { + const request = admitFileInvocation(FILE_ELEMENT, invocation_.request); + const value = yield* performGitObservation( + host, + FILE_ELEMENT, + request, + yield* observationPlace( + invocation_.repository, + invocation_.workingDirectory, + FILE_ELEMENT, + ), + ); + if (!("commit" in value)) { + throw answered(FILE_ELEMENT); + } + return value; + }, + }, + { at: "min" }, + ); + // The transport middlewares both profiles share, installed beneath the // ordinary lifecycle below. Installing them reads no configuration and // obtains no credential: each one matches a request against its own target @@ -551,6 +695,11 @@ export function* useRunComposition(options: RunCompositionOptions): Operation entry.root === checkout.root); diff --git a/packages/git/src/git.ts b/packages/git/src/git.ts index c01be4113..443f2e03e 100644 --- a/packages/git/src/git.ts +++ b/packages/git/src/git.ts @@ -7,11 +7,18 @@ * Core never reaches Git at all: ordinary `execute()` and `xmd run` stay * Git-independent. * - * All four questions are about *one* repository — the one containing the + * The four legacy questions are about *one* repository — the one containing the * contextual working directory — and none of them mutates it. Together they are * what an immutable workflow definition is made of: which repository, which * commit, in which object format, and what the root document held in it. * + * The four selected-checkout reads beside them answer the same kind of + * question about a checkout a `` selected, which is what + * ``, ``, `` and `` ask. They carry + * the Repository selection and the working directory the component observed, so + * which checkout answers is the installed provider's decision rather than the + * process's own directory. + * * Asking is all this Api does. The authored transitions a document writes — * ``, ``, ``, `` — are the separate * `Git` Api, and a replacement installed here never becomes a route to one. @@ -21,6 +28,17 @@ import { type Api, createApi, type Operations } from "@effectionx/context-api"; import { scoped } from "effection"; import type { Operation } from "effection"; import { cwd, exec, useQuietProcessOutput } from "@executablemd/runtime"; +import { GitObservationProviderError } from "./composition/errors.ts"; +import type { + GitChangesInvocation, + GitChangesResult, + GitDiffInvocation, + GitDiffResult, + GitFileInvocation, + GitFileResult, + GitFilesInvocation, + GitFilesResult, +} from "./composition/observation-api.ts"; /** * What Git said, without Git having said it to the terminal. @@ -81,6 +99,31 @@ export interface GitQueryApi { * what lets a run use a commit as its identity and mean it. */ read(commit: string, path: string): Operation; + + /** + * The tracked paths one selected checkout's index holds. + * + * Four questions about a checkout a Repository selected, rather than about + * whichever one the contextual working directory happens to be in. They are + * here because they are read-only Git questions and nothing else: answering + * one moves no branch, writes no object and changes no file. + * + * The invocation carries what the component observed — the Repository + * selection and the working directory — and the normalized request. Which + * checkout those select is the installed provider's to decide, and the + * default below decides nothing: a host with no native checkout to read + * refuses rather than falling back to the caller's own filesystem. + */ + files(invocation: GitFilesInvocation): Operation; + + /** What one comparison of the selected checkout changed, path by path. */ + changes(invocation: GitChangesInvocation): Operation; + + /** A window of the unified patch one comparison of the selected checkout produces. */ + diff(invocation: GitDiffInvocation): Operation; + + /** A window of the text one commit of the selected checkout holds at one path. */ + file(invocation: GitFileInvocation): Operation; } /** Git could not answer for this revision. Carries what Git reported, not a guess. */ @@ -173,9 +216,38 @@ export const GitQuery: Api = createApi("executablemd.g } return result.stdout; }, + + // deno-lint-ignore require-yield + *files(_invocation: GitFilesInvocation): Operation { + throw new GitObservationProviderError(""); + }, + // deno-lint-ignore require-yield + *changes(_invocation: GitChangesInvocation): Operation { + throw new GitObservationProviderError(""); + }, + // deno-lint-ignore require-yield + *diff(_invocation: GitDiffInvocation): Operation { + throw new GitObservationProviderError(""); + }, + // deno-lint-ignore require-yield + *file(_invocation: GitFileInvocation): Operation { + throw new GitObservationProviderError(""); + }, }); export const resolveGitRevision: Operations["resolve"] = GitQuery.operations.resolve; export const gitRoot: Operations["root"] = GitQuery.operations.root; export const gitObjectFormat: Operations["format"] = GitQuery.operations.format; export const readGitObject: Operations["read"] = GitQuery.operations.read; + +/** + * The four selected-checkout reads, pre-bound for a caller who wants the + * operation rather than the seam. + * + * Beside the four legacy accessors above and on the same terms: calling one + * reaches whatever is installed, and a host that installs nothing refuses. + */ +export const readGitFiles: Operations["files"] = GitQuery.operations.files; +export const readGitChanges: Operations["changes"] = GitQuery.operations.changes; +export const readGitDiff: Operations["diff"] = GitQuery.operations.diff; +export const readGitFile: Operations["file"] = GitQuery.operations.file; diff --git a/packages/git/src/retained-repository/admission.ts b/packages/git/src/retained-repository/admission.ts index c95e27bd0..b01c694f4 100644 --- a/packages/git/src/retained-repository/admission.ts +++ b/packages/git/src/retained-repository/admission.ts @@ -2,7 +2,9 @@ * Whether one entry's retained history may be continued at all. * * Two questions, and only two. **Integrity**: every record of this feature's - * own types parses and is named by the request it carries. **Nothing else**: no + * own types — the file work, the ambient routing, the placements, the local Git + * transitions and the local Git observations — parses and is named by the + * request it carries. **Nothing else**: no * checkout is inspected, no saved path is compared with the live one, and no * result is handed out — replaying a completed operation is the durable engine's * own positional matching, and this runs before any of it. @@ -45,12 +47,21 @@ import { replPlacementRecordName, REPL_REPOSITORY_PLACEMENT, } from "./placement-records.ts"; +import type { ReplPlacement } from "./placement-records.ts"; import { parseReplGitRecord, replGitFingerprint, replGitRecordName, REPL_REPOSITORY_GIT, } from "./git-records.ts"; +import { + parseReplObservationRecord, + replObservationFingerprint, + replObservationRecordName, + routedByPlacement, + selectedByPlacement, + REPL_REPOSITORY_OBSERVATION, +} from "./observation-records.ts"; /** The durable types this reader owns, and the only ones it reads. */ const OWNED: ReadonlySet = new Set([ @@ -58,6 +69,7 @@ const OWNED: ReadonlySet = new Set([ REPL_REPOSITORY_SELECTION, REPL_REPOSITORY_PLACEMENT, REPL_REPOSITORY_GIT, + REPL_REPOSITORY_OBSERVATION, ]); /** Why a history cannot be continued. */ @@ -67,7 +79,9 @@ export type ReplRepositoryRefusal = /** A record stating a version this host does not implement. */ | "unknown-version" /** A record whose name is not the name of the request it carries. */ - | "request-fingerprint"; + | "request-fingerprint" + /** An observation naming a route no preceding placement in it selected. */ + | "unrouted-observation"; /** * A history this host cannot continue from. @@ -95,6 +109,10 @@ export class ReplRepositoryHistoryError extends StaleInputError { * feature's type that does not parse is, and refuses. */ export function* admitReplRepositoryHistory(retained: readonly DurableEvent[]): Operation { + // The placements this history has selected *so far*. An observation names the + // checkout it read, and nothing outside this snapshot may authorize one: the + // route has to be a placement a preceding record in the same history made. + const placements: ReplPlacement[] = []; for (const event of retained) { if (event.type !== "yield") { continue; @@ -109,8 +127,8 @@ export function* admitReplRepositoryHistory(retained: readonly DurableEvent[]): // this host ever wrote — and what it says about work already done is // unknown. The three older types keep their existing policy, where a // non-ok envelope is the engine's own failure rather than theirs. - if (type === REPL_REPOSITORY_GIT) { - throw malformed(REPL_REPOSITORY_GIT); + if (type === REPL_REPOSITORY_GIT || type === REPL_REPOSITORY_OBSERVATION) { + throw malformed(type); } continue; } @@ -128,13 +146,20 @@ export function* admitReplRepositoryHistory(retained: readonly DurableEvent[]): continue; } if (type === REPL_REPOSITORY_PLACEMENT) { - yield* admitPlacement(payload, event.description.name); + const placement = yield* admitPlacement(payload, event.description.name); + if (placement !== undefined) { + placements.push(placement); + } continue; } if (type === REPL_REPOSITORY_GIT) { yield* admitGit(payload, event.description.name); continue; } + if (type === REPL_REPOSITORY_OBSERVATION) { + yield* admitObservation(payload, event.description.name, placements); + continue; + } yield* admitLocal(payload, event.description.name); } } @@ -165,6 +190,58 @@ function* admitGit(payload: unknown, name: string): Operation { } } +/** + * Hold one recorded observation to the request it carries, and to the route + * this history actually selected. + * + * The two questions the other records answer, over a record whose success side + * is one page: a saved page that does not describe the request beside it — + * another query, another comparison, a continuation on another commit, more + * records than the limit asked for — does not parse, so it is refused here + * rather than binding a listing on replay. + * + * And one more, which the other record types do not need. An observation names + * the checkout it was read from, and a parseable record can name any checkout + * at all: every required member can be present, the request, name and + * fingerprint untouched, and the root somewhere this entry never selected. + * Nothing downstream would catch it — a completed terminal is reused before the + * body that checks the live registry is ever entered — so a route no preceding + * placement authorizes is refused here, before completed or unfinished reuse. + * + * It is decided from this history alone. No checkout is opened, no path is + * canonicalized and no `git` runs: a continuation whose checkout has since been + * deleted still answers, which is the whole point of retaining the page. + */ +function* admitObservation( + payload: unknown, + name: string, + placements: readonly ReplPlacement[], +): Operation { + const record = parseReplObservationRecord(payload); + if (record === undefined) { + throw malformed(REPL_REPOSITORY_OBSERVATION); + } + const fingerprint = yield* replObservationFingerprint(record.request); + if (record.fingerprint !== fingerprint) { + throw changed(REPL_REPOSITORY_OBSERVATION); + } + if (name !== replObservationRecordName({ operation: record.operation, fingerprint })) { + throw misnamed(REPL_REPOSITORY_OBSERVATION); + } + if (!selectedByPlacement(placements, record.request)) { + throw unrouted( + "names a selection no placement in this history made, so this history does not say " + + "which checkout the read was routed to", + ); + } + if (!routedByPlacement(placements, record.request.repository, record.checkout)) { + throw unrouted( + "names a checkout no placement in this history selected, so this history does not say " + + "where the read ran", + ); + } +} + function* admitLocal(payload: unknown, name: string): Operation { const record = parseReplLocalRecord(payload); if (record === undefined) { @@ -195,7 +272,7 @@ function* admitLocal(payload: unknown, name: string): Operation { * It opens nothing. Whether the checkout a placement names is still there is * not this reader's question and is not a question a continuation asks. */ -function* admitPlacement(payload: unknown, name: string): Operation { +function* admitPlacement(payload: unknown, name: string): Operation { const record = parseReplPlacementRecord(payload); if (record === undefined) { throw malformed(REPL_REPOSITORY_PLACEMENT); @@ -207,6 +284,8 @@ function* admitPlacement(payload: unknown, name: string): Operation { if (name !== replPlacementRecordName({ kind: record.kind, fingerprint })) { throw misnamed(REPL_REPOSITORY_PLACEMENT); } + // A refused selection placed nothing, so it authorizes no route. + return record.outcome.ok ? record.outcome.value : undefined; } function* admitSelection(payload: unknown, name: string): Operation { @@ -237,6 +316,10 @@ function changed(type: string): ReplRepositoryHistoryError { ); } +function unrouted(sentence: string): ReplRepositoryHistoryError { + return refusal("unrouted-observation", `a retained ${REPL_REPOSITORY_OBSERVATION} ${sentence}.`); +} + function misnamed(type: string): ReplRepositoryHistoryError { return refusal( "request-fingerprint", diff --git a/packages/git/src/retained-repository/observation-records.ts b/packages/git/src/retained-repository/observation-records.ts new file mode 100644 index 000000000..b7190ca5c --- /dev/null +++ b/packages/git/src/retained-repository/observation-records.ts @@ -0,0 +1,599 @@ +/** + * What one REPL entry's local Git observation looks like on the wire. + * + * An entry that lists tracked files, compares changes, reads a patch window or + * reads a committed file records the page it produced. Reopen the entry and the + * saved page answers again — the same paths, the same statuses, the same + * resolved commit, the same complete `next`, and no `git`. That is what lets an + * entry interrupted at a question be continued rather than repeated, and it is + * why a saved page describes the repository as it was then rather than as it is + * now. + * + * ## Identity is the request, and position is the journal's + * + * A record's name is the digest of the record version, the read kind and the + * whole normalized request: the authenticated Repository identity, the checkout + * that selection named, its common Git directory, the working directory as + * received, and the query itself. Two equal reads produce one name, and that is + * allowed — telling two of them apart is the durable engine's own positional + * matching, and nothing here searches history by name. + * + * The working directory is named as it arrived, and a revision as the document + * wrote it. Resolving either is live work against a repository, so a replay that + * had to resolve one to recognize its own record would be asking today's + * repository what yesterday's question was. + * + * ## What a result may carry + * + * The exact page the read answered with, including its complete `next`. Nothing + * else: no Git output, no credential, no handle, no live selection identifier + * and no second copy of the observation the page was cut from. + * + * A refusal carries a word from the closed observation vocabulary and the + * sentence this host wrote for it, so a continuation raises the same refusal the + * live read raised. The sentence is this host's own prose about what the + * document asked for — nothing a repository or a Git command printed reaches a + * record. + */ + +import { Err, Ok } from "effection"; +import type { Result } from "effection"; +import type { Operation } from "effection"; +import type { Json } from "@executablemd/durable-streams"; +import { canonicalJson } from "@executablemd/workflow"; +import { members, optionalText, text } from "../composition/parse.ts"; +import { GitObservationError } from "../composition/errors.ts"; +import type { GitObservationReason } from "../composition/errors.ts"; +import { + parseRepositoryIdentity, + repositoryIdentityJson, + sameRepositoryIdentity, + type RepositoryIdentity, +} from "../composition/selection.ts"; +import type { ReplPlacement } from "./placement-records.ts"; +import { + GIT_COMPARE_MODES, + type GitComparisonRequest, + type GitObservationKind, + type GitObservationRequest, + type GitObservationValue, +} from "../composition/observation-api.ts"; +import { + gitObservationValueJson, + parseGitObservationValue, +} from "../composition/observation-records.ts"; +import { digest, RECORD_PREFIX, REPL_REPOSITORY_VERSION } from "./records.ts"; + +/** The durable type one completed local Git observation is journaled under. */ +export const REPL_REPOSITORY_OBSERVATION = "repl_repository_observation"; + +const OBSERVATION_KINDS: readonly GitObservationKind[] = ["files", "changes", "diff", "file"]; + +/** + * The words an observation may be refused under and retained as. + * + * Narrower than the component vocabulary on purpose: an invalid invocation and + * a paired form are decided before a provider is reached, so no observation + * happened and there is nothing to write down. + */ +const RETAINABLE: readonly GitObservationReason[] = [ + "invalid-revision", + "absent-path", + "nontext-blob", + "unusable-repository", + "stale-continuation", +]; + +/** Where one observation ran, as the entry authenticated it. */ +export interface ReplObservationPlace { + /** All six members of the authenticated Repository identity. */ + readonly repository: RepositoryIdentity; + /** The checkout root the authenticated selection named. */ + readonly selectionCheckout: string; + /** That selection's canonical common Git directory. */ + readonly commonDirectory: string; + /** The working directory exactly as it was received, uncanonicalized. */ + readonly workingDirectory: string; +} + +/** What one observation asked for, as the record names it. */ +export interface ReplObservationRequest extends ReplObservationPlace { + readonly query: GitObservationRequest; +} + +/** The checkout the route reached, as the provider resolved it. */ +export interface ReplObservationCheckout { + readonly root: string; + readonly commonDirectory: string; + readonly repositoryName: string; + /** The Worktree's name, or `null` for a repository's own checkout. */ + readonly worktreeName: string | null; +} + +/** One completed observation, as a continuation reads it back. */ +export interface ReplObservationRecord { + readonly version: typeof REPL_REPOSITORY_VERSION; + readonly operation: GitObservationKind; + /** The digest of the version, the kind and the normalized request. */ + readonly fingerprint: string; + readonly request: ReplObservationRequest; + readonly checkout: ReplObservationCheckout; + /** The page it answered with, or the refusal it answered with. */ + readonly outcome: Result; +} + +/** + * A refused observation, as the record carries it. + * + * The failure side of the internal `Result`, and never what a document sees: + * the adapter rebuilds the ordinary `GitObservationError` by the same route on + * a live refusal and on a replayed one, so the two cannot diverge. + */ +export class ReplObservationRefusal extends Error { + override name = "ReplObservationRefusal"; + + readonly element: string; + readonly reason: GitObservationReason; + + constructor(element: string, reason: GitObservationReason, sentence: string) { + super(sentence); + this.element = element; + this.reason = reason; + } +} + +/** The stable name one observation's record is journaled under. */ +export function replObservationRecordName(record: { + readonly operation: GitObservationKind; + readonly fingerprint: string; +}): string { + return canonicalJson([ + RECORD_PREFIX, + "observation", + REPL_REPOSITORY_VERSION, + record.operation, + record.fingerprint, + ]); +} + +/** The digest of everything one observation asks for. */ +export function replObservationFingerprint(request: ReplObservationRequest): Operation { + return digest({ + version: REPL_REPOSITORY_VERSION, + operation: request.query.kind, + request: requestJson(request), + }); +} + +/** The payload one completed observation is journaled with. */ +export function replObservationRecordJson(input: { + readonly fingerprint: string; + readonly request: ReplObservationRequest; + readonly checkout: ReplObservationCheckout; + readonly outcome: Result; +}): Json { + return { + version: REPL_REPOSITORY_VERSION, + operation: input.request.query.kind, + fingerprint: input.fingerprint, + request: requestJson(input.request), + checkout: checkoutJson(input.checkout), + outcome: outcomeJson(input.request.query.kind, input.outcome), + }; +} + +/** + * The observation this saved value describes, or `undefined`. + * + * Total, and exact about every member. A payload carrying anything this version + * did not write describes something other than an observation this host made, + * and answering a document from it would bind a page nobody read. + */ +export function parseReplObservationRecord(value: unknown): ReplObservationRecord | undefined { + const read = members(value, [ + "version", + "operation", + "fingerprint", + "request", + "checkout", + "outcome", + ]); + if (read === undefined || read["version"] !== REPL_REPOSITORY_VERSION) { + return undefined; + } + const operation = parseReplObservationKind(read["operation"]); + const fingerprint = sha256(read["fingerprint"]); + if (operation === undefined || fingerprint === undefined) { + return undefined; + } + const request = parseRequest(read["request"], operation); + const checkout = parseCheckout(read["checkout"]); + if (request === undefined || checkout === undefined) { + return undefined; + } + const outcome = parseOutcome(read["outcome"], request.query); + if (outcome === undefined) { + return undefined; + } + return Object.freeze({ + version: REPL_REPOSITORY_VERSION, + operation, + fingerprint, + request, + checkout, + outcome, + }); +} + +/** The read kind this value names, or `undefined`. */ +export function parseReplObservationKind(value: unknown): GitObservationKind | undefined { + return OBSERVATION_KINDS.find((kind) => kind === value); +} + +/** + * The refusal data this error carries, or `undefined` when it carries none. + * + * What decides whether a failed observation may be written down at all: a + * refusal in the closed vocabulary is an answer the document asked for, and + * anything else describes a condition this host has no word for. + */ +export function replObservationRefusalOf( + error: unknown, +): { readonly reason: GitObservationReason; readonly sentence: string } | undefined { + // Both sides of the one round trip. A live read raises the component's own + // `GitObservationError`; the record's failure side carries this module's + // refusal; and the same classification has to be read out of either, because + // the record is written from the first and read back as the second. + if (!(error instanceof GitObservationError) && !(error instanceof ReplObservationRefusal)) { + return undefined; + } + const reason = RETAINABLE.find((word) => word === error.reason); + return reason === undefined ? undefined : { reason, sentence: error.message }; +} + +function requestJson(request: ReplObservationRequest): Json { + return { + repository: repositoryIdentityJson(request.repository), + selectionCheckout: request.selectionCheckout, + commonDirectory: request.commonDirectory, + workingDirectory: request.workingDirectory, + query: queryJson(request.query), + }; +} + +function comparisonJson(comparison: GitComparisonRequest): Json { + if (comparison.compare === "commits") { + return { compare: "commits", base: comparison.base, head: comparison.head }; + } + return { compare: comparison.compare }; +} + +/** + * The query as the document asked it, with every optional member stated. + * + * Absence is written as `null` rather than omitted, because this is the value a + * fingerprint is computed over: a record whose request omitted a member and one + * whose request held `null` for it would otherwise be the same name for two + * different questions. + */ +function queryJson(query: GitObservationRequest): Json { + if (query.kind === "files") { + return { kind: "files", prefix: query.prefix, limit: query.limit, cursor: query.cursor }; + } + if (query.kind === "file") { + return { + kind: "file", + commit: query.commit, + path: query.path, + offset: query.offset, + limit: query.limit, + }; + } + const shared = { + comparison: comparisonJson(query.comparison), + paths: query.paths === null ? null : [...query.paths], + limit: query.limit, + cursor: query.cursor, + }; + if (query.kind === "changes") { + return { kind: "changes", ...shared }; + } + return { kind: "diff", ...shared, offset: query.offset }; +} + +function checkoutJson(checkout: ReplObservationCheckout): Json { + return { + root: checkout.root, + commonDirectory: checkout.commonDirectory, + repositoryName: checkout.repositoryName, + worktreeName: checkout.worktreeName, + }; +} + +function outcomeJson(kind: GitObservationKind, outcome: Result): Json { + if (!outcome.ok) { + const data = replObservationRefusalOf(outcome.error); + if (data === undefined) { + throw new TypeError("a retained Git observation refusal carries no closed classification"); + } + return { ok: false, error: { reason: data.reason, sentence: data.sentence } }; + } + return { ok: true, value: gitObservationValueJson(kind, outcome.value) }; +} + +const PLACE_MEMBERS = [ + "repository", + "selectionCheckout", + "commonDirectory", + "workingDirectory", + "query", +] as const; + +function parseRequest( + value: unknown, + operation: GitObservationKind, +): ReplObservationRequest | undefined { + const read = members(value, [...PLACE_MEMBERS]); + if (read === undefined) { + return undefined; + } + const repository = parseRepositoryIdentity(read["repository"]); + const selectionCheckout = text(read["selectionCheckout"]); + const commonDirectory = text(read["commonDirectory"]); + const workingDirectory = text(read["workingDirectory"]); + const query = parseQuery(read["query"], operation); + if ( + repository === undefined || + selectionCheckout === undefined || + commonDirectory === undefined || + workingDirectory === undefined || + query === undefined + ) { + return undefined; + } + return Object.freeze({ + repository, + selectionCheckout, + commonDirectory, + workingDirectory, + query, + }); +} + +function parseCheckout(value: unknown): ReplObservationCheckout | undefined { + const read = members(value, ["root", "commonDirectory", "repositoryName", "worktreeName"]); + if (read === undefined) { + return undefined; + } + const root = text(read["root"]); + const commonDirectory = text(read["commonDirectory"]); + const repositoryName = text(read["repositoryName"]); + const worktreeName = optionalText(read["worktreeName"]); + if ( + root === undefined || + commonDirectory === undefined || + repositoryName === undefined || + worktreeName === undefined + ) { + return undefined; + } + return Object.freeze({ root, commonDirectory, repositoryName, worktreeName }); +} + +function parseComparison(value: unknown): GitComparisonRequest | undefined { + const live = members(value, ["compare"]); + if (live !== undefined) { + const mode = GIT_COMPARE_MODES.find((candidate) => candidate === live["compare"]); + if (mode === undefined || mode === "commits") { + return undefined; + } + return Object.freeze({ compare: mode }); + } + const commits = members(value, ["compare", "base", "head"]); + if (commits === undefined || commits["compare"] !== "commits") { + return undefined; + } + const base = text(commits["base"]); + const head = text(commits["head"]); + if (base === undefined || head === undefined) { + return undefined; + } + return Object.freeze({ compare: "commits" as const, base, head }); +} + +function parseQuery( + value: unknown, + operation: GitObservationKind, +): GitObservationRequest | undefined { + if (operation === "files") { + const read = members(value, ["kind", "prefix", "limit", "cursor"]); + if (read === undefined || read["kind"] !== "files") { + return undefined; + } + const prefix = nullableText(read["prefix"]); + const limit = positive(read["limit"]); + const cursor = nullableText(read["cursor"]); + if (prefix === undefined || limit === undefined || cursor === undefined) { + return undefined; + } + return Object.freeze({ kind: "files" as const, prefix, limit, cursor }); + } + if (operation === "file") { + const read = members(value, ["kind", "commit", "path", "offset", "limit"]); + if (read === undefined || read["kind"] !== "file") { + return undefined; + } + const commit = text(read["commit"]); + const path = text(read["path"]); + const offset = count(read["offset"]); + const limit = positive(read["limit"]); + if (commit === undefined || path === undefined || offset === undefined || limit === undefined) { + return undefined; + } + return Object.freeze({ kind: "file" as const, commit, path, offset, limit }); + } + const own = + operation === "changes" + ? ["kind", "comparison", "paths", "limit", "cursor"] + : ["kind", "comparison", "paths", "limit", "cursor", "offset"]; + const read = members(value, own); + if (read === undefined || read["kind"] !== operation) { + return undefined; + } + const comparison = parseComparison(read["comparison"]); + const paths = nullablePaths(read["paths"]); + const limit = positive(read["limit"]); + const cursor = nullableText(read["cursor"]); + if ( + comparison === undefined || + paths === undefined || + limit === undefined || + cursor === undefined + ) { + return undefined; + } + if (operation === "changes") { + return Object.freeze({ kind: "changes" as const, comparison, paths, limit, cursor }); + } + const offset = count(read["offset"]); + return offset === undefined + ? undefined + : Object.freeze({ kind: "diff" as const, comparison, paths, offset, limit, cursor }); +} + +function parseOutcome( + value: unknown, + query: GitObservationRequest, +): Result | undefined { + const refused = members(value, ["ok", "error"]); + if (refused !== undefined) { + if (refused["ok"] !== false) { + return undefined; + } + const data = members(refused["error"], ["reason", "sentence"]); + if (data === undefined) { + return undefined; + } + const reason = RETAINABLE.find((word) => word === data["reason"]); + const sentence = text(data["sentence"]); + if (reason === undefined || sentence === undefined) { + return undefined; + } + return Err(new ReplObservationRefusal(elementOf(query.kind), reason, sentence)); + } + const succeeded = members(value, ["ok", "value"]); + if (succeeded === undefined || succeeded["ok"] !== true) { + return undefined; + } + const page = parseGitObservationValue(succeeded["value"], query); + return page === undefined ? undefined : Ok(page); +} + +/** The component name one read kind is written and refused as. */ +export function elementOf(kind: GitObservationKind): string { + if (kind === "files") { + return ""; + } + if (kind === "changes") { + return ""; + } + return kind === "diff" ? "" : ""; +} + +function nullableText(value: unknown): string | null | undefined { + return value === null ? null : text(value); +} + +function nullablePaths(value: unknown): readonly string[] | null | undefined { + if (value === null) { + return null; + } + if (!Array.isArray(value)) { + return undefined; + } + const entries: string[] = []; + for (const entry of value) { + const path = text(entry); + if (path === undefined) { + return undefined; + } + entries.push(path); + } + return Object.freeze(entries); +} + +function sha256(value: unknown): string | undefined { + const candidate = text(value); + return candidate !== undefined && /^[0-9a-f]{64}$/.test(candidate) ? candidate : undefined; +} + +function count(value: unknown): number | undefined { + return typeof value === "number" && Number.isSafeInteger(value) && value >= 0 ? value : undefined; +} + +function positive(value: unknown): number | undefined { + const parsed = count(value); + return parsed === undefined || parsed < 1 ? undefined : parsed; +} + +/** + * The route one historical placement authorizes. + * + * Derived from the placement exactly as the live provider derives a registered + * checkout from it, so what admission compares a record against is the same + * relationship the runtime enforces — read out of history rather than out of + * today's registry. + */ +export function observationRouteOf(placement: ReplPlacement): ReplObservationCheckout { + return Object.freeze({ + root: placement.checkoutRoot, + commonDirectory: placement.commonDirectory, + repositoryName: placement.identity.name, + worktreeName: placement.kind === "worktree" ? placement.displayName : null, + }); +} + +/** + * Whether one of these historical placements authorizes this recorded route. + * + * Every member, under the whole Repository identity the request names. A route + * matching on a root while naming another repository, worktree or Git directory + * is a route no placement in this history ever selected. + */ +export function routedByPlacement( + placements: readonly ReplPlacement[], + repository: RepositoryIdentity, + checkout: ReplObservationCheckout, +): boolean { + return placements.some((placement) => { + if (!sameRepositoryIdentity(placement.identity, repository)) { + return false; + } + const route = observationRouteOf(placement); + return ( + route.root === checkout.root && + route.commonDirectory === checkout.commonDirectory && + route.repositoryName === checkout.repositoryName && + route.worktreeName === checkout.worktreeName + ); + }); +} + +/** + * Whether one of these historical placements is the selection this read named. + * + * The request carries the checkout the authenticated selection pointed at and + * that selection's common Git directory. A record naming a selection no + * placement in this history made describes a read this entry never routed. + */ +export function selectedByPlacement( + placements: readonly ReplPlacement[], + request: ReplObservationRequest, +): boolean { + return placements.some( + (placement) => + sameRepositoryIdentity(placement.identity, request.repository) && + placement.checkoutRoot === request.selectionCheckout && + placement.commonDirectory === request.commonDirectory, + ); +} diff --git a/packages/git/tests/api-entrypoint.test.ts b/packages/git/tests/api-entrypoint.test.ts index 638dffdbc..3f2938fab 100644 --- a/packages/git/tests/api-entrypoint.test.ts +++ b/packages/git/tests/api-entrypoint.test.ts @@ -130,7 +130,7 @@ const CONTEXTUAL_APIS: readonly string[] = [ const OPERATIONS: readonly (readonly [string, readonly string[]])[] = [ ["Git", ["add", "commit", "push", "switch"]], ["GitHost", ["route"]], - ["GitQuery", ["format", "read", "resolve", "root"]], + ["GitQuery", ["changes", "diff", "file", "files", "format", "read", "resolve", "root"]], ["IssueApi", ["read", "upsert"]], ["IssueTrackerContext", ["current"]], ["PullRequestAPI", ["read", "upsert"]], @@ -154,6 +154,10 @@ const COMPANIONS: readonly string[] = [ "currentRepository", "gitObjectFormat", "gitRoot", + "readGitChanges", + "readGitDiff", + "readGitFile", + "readGitFiles", "readGitObject", "reconcileGitHostEffect", "resolveGitRevision", diff --git a/packages/git/tests/git-observation-records.test.ts b/packages/git/tests/git-observation-records.test.ts new file mode 100644 index 000000000..bc2e55ff9 --- /dev/null +++ b/packages/git/tests/git-observation-records.test.ts @@ -0,0 +1,476 @@ +/** + * Tier GOR — the shared codecs the four Git reads are carried by. + * + * Result JSON, the total parsers a continuation and a journal read it back + * through, the code-point windows a page is cut with, and the version-1 cursor. + * These are decided by arithmetic rather than by a repository, which is why they + * are asserted here: every one of them has a wrong answer that no fixture would + * make obvious. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Json } from "@executablemd/durable-streams"; +import type { + GitChangesRequest, + GitDiffRequest, + GitFileRequest, + GitFilesRequest, + GitObservationRequest, +} from "@executablemd/git/api"; +import { + DEFAULT_RECORD_LIMIT, + DEFAULT_TEXT_LIMIT, + admitChangesRequest, + admitDiffRequest, + admitFileRequest, + admitFilesRequest, + selectedChange, + selectsPath, + sortedChanges, +} from "../src/composition/observation-api.ts"; +import { + codePointLength, + encodeGitObservationCursor, + gitChangesResultJson, + gitDiffResultJson, + gitFileResultJson, + gitFilesResultJson, + gitObservationQuery, + gitRecordWindow, + gitTextWindow, + observationDigest, + parseGitObservationCursor, + parseGitObservationValue, + sameObservationCheckout, +} from "../src/composition/observation-records.ts"; +import type { GitObservationCheckout } from "../src/composition/observation-records.ts"; +import { GitObservationError } from "../src/composition/errors.ts"; + +const CHECKOUT: GitObservationCheckout = Object.freeze({ + repository: Object.freeze({ + name: "project", + locatorFingerprint: "a".repeat(64), + requestedBase: null, + creationCommit: "b".repeat(40), + primaryBranch: "main", + objectFormat: "sha1" as const, + }), + root: "/checkouts/project", + commonDirectory: "/checkouts/project/.git", +}); + +const FILES_REQUEST: GitFilesRequest = Object.freeze({ + kind: "files", + prefix: "src/", + limit: 100, + cursor: null, +}); + +function filesCursor(overrides: Partial> = {}): string { + const base = { + version: 1 as const, + kind: "files" as const, + query: gitObservationQuery(FILES_REQUEST, null), + checkout: CHECKOUT, + comparison: null, + observation: "c".repeat(64), + position: 3, + }; + return encodeGitObservationCursor({ ...base, ...overrides }); +} + +describe("GOR1 — windows count code points", () => { + it("never cuts a surrogate pair, and reports the whole length", function* () { + const astral = "\u{1D11E}"; + const text = `ab${astral}cd`; + expect(codePointLength(text)).toBe(5); + expect(text.length).toBe(6); + + const first = gitTextWindow(text, 0, 3); + expect(first.text).toBe(`ab${astral}`); + expect(first.length).toBe(5); + expect(first.nextOffset).toBe(3); + + const second = gitTextWindow(text, 3, 3); + expect(second.text).toBe("cd"); + expect(second.nextOffset).toBe(null); + }); + + it("answers an empty window at or past the end, with the whole length", function* () { + const text = "abcd"; + for (const offset of [4, 5, 4096]) { + const window = gitTextWindow(text, offset, 10); + expect(`${offset}: ${window.text}|${window.length}|${window.nextOffset}`).toBe( + `${offset}: |4|null`, + ); + } + const empty = gitTextWindow("", 0, 10); + expect(`${empty.text}|${empty.length}|${empty.nextOffset}`).toBe("|0|null"); + }); + + it("does not overflow a safe-integer limit added to an offset", function* () { + const window = gitTextWindow("abcd", 1, Number.MAX_SAFE_INTEGER); + expect(window.text).toBe("bcd"); + expect(window.nextOffset).toBe(null); + }); + + it("pages records by index, with no omission or duplication", function* () { + const records = ["a", "b", "c", "d", "e"]; + const first = gitRecordWindow(records, 0, 2); + const second = gitRecordWindow(records, first.nextIndex ?? 0, 2); + const third = gitRecordWindow(records, second.nextIndex ?? 0, 2); + expect([...first.page, ...second.page, ...third.page]).toEqual(records); + expect(third.nextIndex).toBe(null); + expect(gitRecordWindow(records, 5, 2).page).toEqual([]); + }); +}); + +describe("GOR2 — filters are literal and ordering is by code point", () => { + it("matches a path, its descendants and a trailing-slash selector", function* () { + expect(selectsPath("src", "src")).toBe(true); + expect(selectsPath("src", "src/a.ts")).toBe(true); + expect(selectsPath("src", "srcfile.ts")).toBe(false); + expect(selectsPath("src/", "src")).toBe(false); + expect(selectsPath("src/", "src/a.ts")).toBe(true); + // The characters a glob would expand are those characters. + expect(selectsPath("a*b", "a*b")).toBe(true); + expect(selectsPath("a*b", "axb")).toBe(false); + }); + + it("matches a rename on either of its paths", function* () { + const renamed = { path: "new.md", status: "R" as const, previousPath: "old.md" }; + expect(selectedChange(["old.md"], renamed)).toBe(true); + expect(selectedChange(["new.md"], renamed)).toBe(true); + expect(selectedChange(["other.md"], renamed)).toBe(false); + // Omission selects everything; an empty selector selects nothing. + expect(selectedChange(null, renamed)).toBe(true); + expect(selectedChange([], renamed)).toBe(false); + }); + + it("orders by current path, then previous path", function* () { + expect( + sortedChanges([ + { path: "b", status: "M", previousPath: null }, + { path: "a", status: "R", previousPath: "z" }, + { path: "a", status: "R", previousPath: "y" }, + ]), + ).toEqual([ + { path: "a", status: "R", previousPath: "y" }, + { path: "a", status: "R", previousPath: "z" }, + { path: "b", status: "M", previousPath: null }, + ]); + }); +}); + +describe("GOR3 — admission happens before any provider", () => { + it("fills the defaults and keeps every spelling", function* () { + expect(admitFilesRequest("", {})).toEqual({ + kind: "files", + prefix: null, + limit: DEFAULT_RECORD_LIMIT, + cursor: null, + }); + expect(admitDiffRequest("", { compare: "staged" })).toEqual({ + kind: "diff", + comparison: { compare: "staged" }, + paths: null, + offset: 0, + limit: DEFAULT_TEXT_LIMIT, + cursor: null, + }); + // A filename holding a space, a tab, a newline and a non-ASCII character + // arrives untrimmed and unnormalized. + const awkward = "uni/é a\tb\nc.md"; + expect(admitFileRequest("", { commit: "HEAD", path: awkward }).path).toBe(awkward); + // An empty prefix is the whole repository, which is what omission means. + expect(admitFilesRequest("", { prefix: "" }).prefix).toBe(null); + // A prefix may stop inside its final segment. + expect(admitFilesRequest("", { prefix: "packages/co" }).prefix).toBe("packages/co"); + }); + + it("refuses a limit, an offset, a path and a comparison it cannot honor", function* () { + const refusals: [string, () => unknown][] = [ + ["zero limit", () => admitFilesRequest("", { limit: 0 })], + ["fractional limit", () => admitFilesRequest("", { limit: 1.5 })], + ["unsafe limit", () => admitFilesRequest("", { limit: 2 ** 53 })], + ["negative offset", () => admitDiffRequest("", { compare: "staged", offset: -1 })], + [ + "absolute path", + () => admitFileRequest("", { commit: "HEAD", path: "/etc/passwd" }), + ], + ["parent segment", () => admitFileRequest("", { commit: "HEAD", path: "a/../b" })], + [ + "empty filter entry", + () => admitChangesRequest("", { compare: "staged", paths: [""] }), + ], + [ + "revisions on a live comparison", + () => admitChangesRequest("", { compare: "staged", base: "HEAD" }), + ], + [ + "a commit comparison missing head", + () => admitChangesRequest("", { compare: "commits", base: "HEAD" }), + ], + ["an unknown comparison", () => admitDiffRequest("", { compare: "merge-base" })], + ]; + for (const [named, attempt] of refusals) { + let refused: unknown; + try { + attempt(); + } catch (error) { + refused = error; + } + expect(`${named}: ${refused instanceof GitObservationError}`).toBe(`${named}: true`); + expect(`${named}: ${refused instanceof GitObservationError ? refused.reason : "none"}`).toBe( + `${named}: invalid-invocation`, + ); + } + }); + + it("honors a limit far above the superseded proposed cap", function* () { + expect(admitDiffRequest("", { compare: "staged", limit: 1_000_000 }).limit).toBe( + 1_000_000, + ); + expect(admitFilesRequest("", { limit: 1002 }).limit).toBe(1002); + }); +}); + +describe("GOR4 — the cursor is data, bound to one observation", () => { + it("round-trips, and holds no path the caller could use as a destination", function* () { + const encoded = filesCursor(); + const parsed = parseGitObservationCursor(encoded); + expect(parsed?.kind).toBe("files"); + expect(parsed?.position).toBe(3); + expect(parsed?.comparison).toBe(null); + expect(sameObservationCheckout(parsed?.checkout ?? CHECKOUT, CHECKOUT)).toBe(true); + // Opaque: nothing about the encoding is a promise, and the decoded payload + // holds the checkout the first page was authenticated against and nothing + // a later call could be redirected by. + expect(Object.keys(parsed ?? {}).sort()).toEqual([ + "checkout", + "comparison", + "kind", + "observation", + "position", + "query", + "version", + ]); + }); + + it("refuses a malformed string, a wrong version and an unknown kind", function* () { + expect(parseGitObservationCursor("not base64url!!")).toBe(undefined); + expect(parseGitObservationCursor(btoa("{}"))).toBe(undefined); + expect(parseGitObservationCursor(btoa('{"version":2}'))).toBe(undefined); + expect(parseGitObservationCursor(filesCursor({ version: 2 }))).toBe(undefined); + expect(parseGitObservationCursor(filesCursor({ kind: "file" }))).toBe(undefined); + expect(parseGitObservationCursor(filesCursor({ observation: "short" }))).toBe(undefined); + expect(parseGitObservationCursor(filesCursor({ position: -1 }))).toBe(undefined); + }); + + it("names a different checkout, query and position when any of them differs", function* () { + const elsewhere: GitObservationCheckout = { ...CHECKOUT, root: "/checkouts/other" }; + expect(sameObservationCheckout(CHECKOUT, elsewhere)).toBe(false); + expect( + sameObservationCheckout(CHECKOUT, { + ...CHECKOUT, + repository: { ...CHECKOUT.repository, name: "renamed" }, + }), + ).toBe(false); + + // A query excludes the limit, so raising it is not a different observation. + const wider: GitFilesRequest = { ...FILES_REQUEST, limit: 500 }; + expect(yield* observationDigest(gitObservationQuery(wider, null))).toBe( + yield* observationDigest(gitObservationQuery(FILES_REQUEST, null)), + ); + // The prefix is part of it, so changing that is. + const narrower: GitFilesRequest = { ...FILES_REQUEST, prefix: "src/a" }; + expect(yield* observationDigest(gitObservationQuery(narrower, null))).not.toBe( + yield* observationDigest(gitObservationQuery(FILES_REQUEST, null)), + ); + }); + + it("uses the resolved commit ids for a commit comparison's query", function* () { + const authored: GitChangesRequest = { + kind: "changes", + comparison: { compare: "commits", base: "main", head: "HEAD" }, + paths: null, + limit: 100, + cursor: null, + }; + const resolved: GitChangesRequest = { + kind: "changes", + comparison: { compare: "commits", base: "d".repeat(40), head: "e".repeat(40) }, + paths: null, + limit: 100, + cursor: null, + }; + const comparison = { + compare: "commits" as const, + base: "d".repeat(40), + head: "e".repeat(40), + }; + // The follow-up calls with the ids the first page's `next` carried, and the + // query it computes is the query the first page recorded — which is what + // stops a branch that moved from retargeting a later page. + expect(yield* observationDigest(gitObservationQuery(authored, comparison))).toBe( + yield* observationDigest(gitObservationQuery(resolved, comparison)), + ); + }); +}); + +describe("GOR5 — a result is written and parsed by one contract", () => { + it("round-trips all four results, including their next arguments", function* () { + const files = { + paths: ["a.ts", "b.ts"], + next: { prefix: "src/", limit: 100, cursor: "x" }, + }; + expect(parseGitObservationValue(gitFilesResultJson(files), FILES_REQUEST)).toEqual(files); + + const changesRequest: GitChangesRequest = { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: ["src/"], + limit: 100, + cursor: null, + }; + const changes = { + comparison: { compare: "unstaged" as const }, + changes: [{ path: "a.ts", status: "R" as const, previousPath: "b.ts" }], + next: { compare: "unstaged" as const, paths: ["src/"], limit: 100, cursor: "x" }, + }; + expect(parseGitObservationValue(gitChangesResultJson(changes), changesRequest)).toEqual( + changes, + ); + + const diffRequest: GitDiffRequest = { + kind: "diff", + comparison: { compare: "commits", base: "d".repeat(40), head: "e".repeat(40) }, + paths: null, + offset: 0, + limit: 4096, + cursor: null, + }; + const diff = { + comparison: { + compare: "commits" as const, + base: "d".repeat(40), + head: "e".repeat(40), + }, + text: "patch", + offset: 0, + length: 12, + next: { + compare: "commits" as const, + base: "d".repeat(40), + head: "e".repeat(40), + offset: 5, + limit: 4096, + cursor: "x", + }, + }; + expect(parseGitObservationValue(gitDiffResultJson(diff), diffRequest)).toEqual(diff); + + const fileRequest: GitFileRequest = { + kind: "file", + commit: "HEAD~1", + path: "a.ts", + offset: 0, + limit: 4096, + }; + const file = { + commit: "f".repeat(40), + path: "a.ts", + text: "source", + offset: 0, + length: 20, + next: { commit: "f".repeat(40), path: "a.ts", offset: 6, limit: 4096 }, + }; + expect(parseGitObservationValue(gitFileResultJson(file), fileRequest)).toEqual(file); + }); + + it("refuses a page that describes another request", function* () { + const wrong: [string, Json, GitObservationRequest][] = [ + [ + "a page longer than the limit", + { paths: ["a", "b", "c"], next: null }, + { ...FILES_REQUEST, limit: 2 }, + ], + [ + "a next naming another prefix", + { paths: [], next: { prefix: "doc/", limit: 100, cursor: "x" } }, + FILES_REQUEST, + ], + [ + "a rename status with no previous path", + { + comparison: { compare: "unstaged" }, + changes: [{ path: "a", status: "R", previousPath: null }], + next: null, + }, + { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: null, + limit: 100, + cursor: null, + }, + ], + [ + "a previous path on a status that is not a rename", + { + comparison: { compare: "unstaged" }, + changes: [{ path: "a", status: "M", previousPath: "b" }], + next: null, + }, + { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: null, + limit: 100, + cursor: null, + }, + ], + [ + "a comparison the request did not ask for", + { comparison: { compare: "staged", base: null }, changes: [], next: null }, + { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: null, + limit: 100, + cursor: null, + }, + ], + [ + "a file continuation on another commit", + { + commit: "f".repeat(40), + path: "a.ts", + text: "x", + offset: 0, + length: 2, + next: { commit: "0".repeat(40), path: "a.ts", offset: 1, limit: 10 }, + }, + { kind: "file", commit: "HEAD", path: "a.ts", offset: 0, limit: 10 }, + ], + [ + "an offset the request did not ask for", + { + commit: "f".repeat(40), + path: "a.ts", + text: "x", + offset: 7, + length: 2, + next: null, + }, + { kind: "file", commit: "HEAD", path: "a.ts", offset: 0, limit: 10 }, + ], + ]; + for (const [named, payload, request] of wrong) { + expect(`${named}: ${parseGitObservationValue(payload, request) === undefined}`).toBe( + `${named}: true`, + ); + } + }); +}); diff --git a/packages/git/tests/git-observations-deno.test.ts b/packages/git/tests/git-observations-deno.test.ts new file mode 100644 index 000000000..9ad900813 --- /dev/null +++ b/packages/git/tests/git-observations-deno.test.ts @@ -0,0 +1,95 @@ +/** + * Tier GO9, the half that needs a runtime which operates repositories. + * + * One case, kept apart from `git-observations.test.ts` because its subject is a + * *managed* selection: it clones a remote into a managed root and adds a + * worktree, which this repository holds under a kernel-released exclusive + * advisory lock taken through the Deno runtime. Node and Bun expose no + * equivalent, so there the document cannot reach the checkout at all and the + * case fails on the lock rather than on anything it asserts. + * + * Everything else about which checkout answers stays portable and stays in + * `git-observations.test.ts`: the sibling checkouts distinguished by content, + * and the foreign and edited selections refused before a host is reached. What + * only this file can observe is that an ambient checkout, a lexical Repository, + * a Worktree and a `` inside one each answer from their own checkout. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Json } from "@executablemd/durable-streams"; +import { capturingProbe, writeFixtureFile } from "./support/observations.ts"; +import { runOrdinaryDocument, useHostCheckout, useManagedRoot } from "./support/run-composition.ts"; +import { useBareRemote } from "./support/git-remotes.ts"; +import { REMOTE } from "./support/run-composition-tier.ts"; + +/** The one bound value a case's document projected, as an object. */ +function object(value: Json | undefined): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error(`expected one bound object, received ${JSON.stringify(value)}`); + } + return value; +} + +function paths(value: Json | undefined): string[] { + const listed = object(value).paths; + if (!Array.isArray(listed)) { + throw new Error(`expected a paths array, received ${JSON.stringify(listed)}`); + } + return listed.map((entry) => String(entry)); +} + +describe("GO9 — which checkout answers, through a managed selection", () => { + it("reads the ambient checkout, a lexical Repository, a Worktree and a Dir inside one", function* () { + const remote = yield* useBareRemote(REMOTE); + const root = yield* useManagedRoot(); + const ambient = yield* useHostCheckout(remote.locator); + yield* writeFixtureFile(ambient.root, "only-here.txt", "ambient\n"); + ambient.run("add", "--", "only-here.txt"); + ambient.run("commit", "-m", "ambient only"); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + ``, + "", + '', + "", + "", + "", + '', + "", + '', + "", + "", + "", + '', + "", + '', + "", + "", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"), + { root, cwd: ambient.root, components: [capturingProbe(seen)] }, + ); + + // The ambient checkout holds a path the managed clone never received, which + // is how a read that answered from the wrong one would say so. + expect(paths(seen[0])).toContain("only-here.txt"); + expect(paths(seen[1])).toEqual(["which.txt"]); + expect(object(seen[2]).changes).toEqual([]); + // A `` beneath a Worktree stays inside that checkout, and the answer + // is still relative to its repository root. + expect(paths(seen[3])).toEqual(["which.txt"]); + }); +}); diff --git a/packages/git/tests/git-observations.test.ts b/packages/git/tests/git-observations.test.ts new file mode 100644 index 000000000..14439de91 --- /dev/null +++ b/packages/git/tests/git-observations.test.ts @@ -0,0 +1,1661 @@ +/** + * Tier GO — the four local Git observations, against real repositories. + * + * ``, ``, `` and `` read a checkout + * a Repository selected and change nothing in it. Every assertion here runs + * against a real checkout with real commits and real `git`: what a read has to + * get right is which repository it addressed, which data it preserved, and what + * it left exactly as it found it, and none of those is a question a substituted + * repository can answer. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Json } from "@executablemd/durable-streams"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { scoped, until } from "effection"; +import { join } from "node:path"; +import { utimes } from "node:fs/promises"; +import { lstat } from "@effectionx/fs"; +import { collect, execute, inlineSource, registerComponents } from "@executablemd/core"; +import { canonicalJson } from "@executablemd/workflow"; +import { COMPOSITION_ORIGIN } from "../src/composition/definitions.ts"; +import { + COMPOSITION_REGISTRATIONS, + useCompositionComponents, +} from "../src/composition/installation.ts"; +import { + DEFAULT_RECORD_LIMIT, + DEFAULT_TEXT_LIMIT, + GIT_COMPARE_MODES, + changesProps, + changesReturns, + diffProps, + diffReturns, + fileProps, + fileReturns, + filesProps, + filesReturns, +} from "../src/composition/observation-api.ts"; +import { + ASTRAL, + AWKWARD_PATH, + LONG_BLOB, + MAGIC_PATH, + actionProbe, + capturingProbe, + useConflictedCheckout, + useFileFixture, + useGeneratedFixture, + fingerprintCheckout, + indexFingerprints, + useLinkedWorktree, + useObservationFixture, + useSentinelConfiguration, + useUnbornCheckout, + writeFixtureFile, +} from "./support/observations.ts"; +import { + commonDirectoryOf, + countingOrdinaryHost, + raised, + runOrdinaryDocument, + subcommands, + useHostCheckout, + useManagedRoot, +} from "./support/run-composition.ts"; +import { useBareRemote } from "./support/git-remotes.ts"; +import { REMOTE } from "./support/run-composition-tier.ts"; +import { exists } from "@effectionx/fs"; + +import type { HostCheckout } from "./support/run-composition.ts"; +import type { ComponentRegistration } from "@executablemd/core"; +import type { Operation } from "effection"; +import { cwd } from "@executablemd/runtime"; +import { selectedRepository } from "../src/composition/context.ts"; +import { + GitObservationError, + GitObservationProviderError, + GitOperationAdmissionError, +} from "../src/composition/errors.ts"; +import { GitQuery, Repository } from "@executablemd/git/api"; +import type { + GitChangesInvocation, + GitChangesResult, + GitDiffInvocation, + GitDiffResult, + GitFileInvocation, + GitFileResult, + GitFilesInvocation, + GitFilesResult, + GitInvocationPlace, + GitObservationRequest, + RepositorySelection, +} from "@executablemd/git/api"; +import { denoRepositoryHost } from "../src/deno/composition/host.ts"; +import type { GitInvocation, RepositoryHost } from "../src/deno/composition/host.ts"; +import { performGitObservation } from "../src/deno/composition/observations.ts"; +import type { ObservationPlace } from "../src/deno/composition/observations.ts"; + +/** + * The place a native read runs in, for a case that drives the performer itself. + * + * The identity's members beyond the object format are what a cursor carries; + * the format is what decides which empty tree an unborn comparison names. + */ +function observationPlaceOf(checkout: HostCheckout): ObservationPlace { + return { + repository: { + name: "fixture", + locatorFingerprint: "0".repeat(64), + requestedBase: null, + creationCommit: "0".repeat(40), + primaryBranch: "main", + objectFormat: "sha1", + }, + root: checkout.root, + commonDirectory: commonDirectoryOf(checkout), + }; +} + +/** + * A component that performs a read loop at the place a document wrote it. + * + * Several of these cases follow a continuation to its end, which is more pages + * than a document should have to spell. The loop reaches the ordinary + * contextual operation with the two observations a component makes, so what it + * drives is the installed provider rather than a shortcut around it. + */ +function followingProbe( + name: string, + run: (place: GitInvocationPlace) => Operation, +): ComponentRegistration { + return { + name, + origin: `test://${name.toLowerCase()}`, + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + const repository = yield* selectedRepository(); + if (repository === undefined) { + throw new Error("the probe is written outside every repository"); + } + yield* run({ repository, workingDirectory: yield* cwd() }); + return ""; + }, + }; +} + +/** + * A host that records every scratch directory a read acquired. + * + * The directory is what the observation owns, so whether it is gone afterwards + * is the whole of what teardown has to show. A wrapper is the only place to + * learn its + * name: the production host hands it to the read and to nobody else. + */ +function recordingDirectories(inner: RepositoryHost, created: string[]): RepositoryHost { + const capture = inner.gitBytes; + const authenticate = inner.useAuthentication; + return { + git: (invocation: GitInvocation) => inner.git(invocation), + *useDirectory(): Operation { + const directory = yield* inner.useDirectory(); + created.push(directory); + return directory; + }, + ...(capture === undefined + ? {} + : { gitBytes: (invocation: GitInvocation) => capture(invocation) }), + ...(authenticate === undefined + ? {} + : { useAuthentication: (locator: string) => authenticate(locator) }), + }; +} + +/** + * How this refusal is classified, in the vocabulary a case can state. + * + * The reads own `GitObservationError`; a missing binding, an unknown prop and a + * limit the schema rejects are the engine's own refusals, and what matters + * about those is that they happened before a provider was asked. + */ +function refusalKind(value: unknown): string { + if (value instanceof GitObservationError) { + return value.reason; + } + return value === undefined ? "bound" : "refused"; +} + +/** The property names one schema declares. */ +function propertiesOf(schema: Json): Record { + if (typeof schema !== "object" || schema === null || Array.isArray(schema)) { + throw new Error("expected a schema object"); + } + const properties = schema.properties; + if (typeof properties !== "object" || properties === null || Array.isArray(properties)) { + throw new Error("expected a schema with properties"); + } + return properties; +} + +/** The four reads, with the smallest invocation each one accepts. */ +const FOUR_ELEMENTS: readonly { readonly name: string; readonly source: string }[] = [ + { name: "Git.Files", source: '' }, + { name: "Git.Changes", source: '' }, + { name: "Git.Diff", source: '' }, + { name: "Git.File", source: '' }, +]; + +/** A selection a host hands over without any native checkout behind it. */ +const SUPPLIED_SELECTION: RepositorySelection = Object.freeze({ + selection: "supplied", + name: "supplied", + identity: Object.freeze({ + name: "supplied", + locatorFingerprint: "1".repeat(64), + requestedBase: null, + creationCommit: "0".repeat(40), + primaryBranch: "main", + objectFormat: "sha1" as const, + }), + checkoutPath: "/nowhere", +}); + +interface NeutralOptions { + /** Whether a Repository provider answers, without any read provider. */ + readonly repository?: boolean; + /** When present, the four reads are supplied and their requests recorded. */ + readonly reads?: string[]; + readonly components?: readonly ComponentRegistration[]; +} + +/** + * One document on a host that installs no native repository provider. + * + * The vocabulary is declared, because declaring it reaches nothing. What a case + * adds is exactly what it is about: a Repository that answers and no reads, or + * reads supplied through the same contextual Api a native host answers. + */ +function runNeutralDocument(source: string, options: NeutralOptions = {}): Operation { + return scoped(function* () { + yield* useCompositionComponents(); + if (options.components !== undefined) { + yield* registerComponents([...options.components]); + } + if (options.repository === true) { + yield* Repository.around( + // deno-lint-ignore require-yield + { + *ambient(): Operation { + return SUPPLIED_SELECTION; + }, + }, + { at: "min" }, + ); + } + const asked = options.reads; + if (asked !== undefined) { + yield* GitQuery.around(suppliedReads(asked), { at: "min" }); + } + return yield* collect( + yield* execute({ ...inlineSource(source), stream: new InMemoryStream() }), + ); + }); +} + +/** + * The four reads, answered by a host rather than by a native checkout. + * + * Every method records the normalized request it received, which is what makes + * "the component composed exactly this" an observation. The answers are the + * smallest valid results of each shape. + */ +function suppliedReads(asked: string[]): Parameters[0] { + const record = (kind: string, request: Json): void => { + asked.push(`${kind} ${canonicalJson(request)}`); + }; + return { + // deno-lint-ignore require-yield + *files([invocation]: [GitFilesInvocation]): Operation { + record("files", requestJson(invocation.request)); + return { paths: ["a/b.ts"], next: null }; + }, + // deno-lint-ignore require-yield + *changes([invocation]: [GitChangesInvocation]): Operation { + record("changes", requestJson(invocation.request)); + return { comparison: { compare: "staged", base: null }, changes: [], next: null }; + }, + // deno-lint-ignore require-yield + *diff([invocation]: [GitDiffInvocation]): Operation { + record("diff", requestJson(invocation.request)); + return { + comparison: { compare: "staged", base: null }, + text: "", + offset: 0, + length: 0, + next: null, + }; + }, + // deno-lint-ignore require-yield + *file([invocation]: [GitFileInvocation]): Operation { + record("file", requestJson(invocation.request)); + return { + commit: "0".repeat(40), + path: invocation.request.path, + text: "supplied", + offset: 0, + length: 8, + next: null, + }; + }, + }; +} + +/** One normalized request, as JSON a case can compare exactly. */ +function requestJson(request: GitObservationRequest): Json { + if (request.kind === "files") { + return { kind: "files", prefix: request.prefix, limit: request.limit, cursor: request.cursor }; + } + if (request.kind === "file") { + return { + kind: "file", + commit: request.commit, + path: request.path, + offset: request.offset, + limit: request.limit, + }; + } + const comparison: Json = + request.comparison.compare === "commits" + ? { + compare: "commits", + base: request.comparison.base, + head: request.comparison.head, + } + : { compare: request.comparison.compare }; + if (request.kind === "changes") { + return { + kind: "changes", + comparison, + paths: request.paths === null ? null : [...request.paths], + limit: request.limit, + cursor: request.cursor, + }; + } + return { + kind: "diff", + comparison, + paths: request.paths === null ? null : [...request.paths], + offset: request.offset, + limit: request.limit, + cursor: request.cursor, + }; +} + +/** The closed reason this raised value refuses under, or what it was instead. */ +function reasonOf(value: unknown): string { + return value instanceof GitObservationError ? value.reason : String(value); +} + +function messageOf(value: unknown): string { + return value instanceof Error ? value.message : String(value); +} + +/** The one bound value a case's document projected, as an object. */ +function object(value: Json | undefined): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error(`expected one bound object, received ${JSON.stringify(value)}`); + } + return value; +} + +function paths(value: Json | undefined): string[] { + const listed = object(value).paths; + if (!Array.isArray(listed)) { + throw new Error(`expected a paths array, received ${JSON.stringify(listed)}`); + } + return listed.map((entry) => String(entry)); +} + +describe("GO1 — discovery addresses the chosen repository", () => { + it("lists the tracked index paths and nothing else", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + expect(seen).toHaveLength(1); + const listed = paths(seen[0]); + expect(listed).toEqual( + [ + AWKWARD_PATH, + "doc/renamed.md", + "link-me.txt", + "mode.sh", + "nested", + "src/a.ts", + "src/added.ts", + "src/bin.dat", + MAGIC_PATH, + ].sort(), + ); + expect(object(seen[0]).next).toBe(null); + }); +}); + +describe("GO2 — discovery excludes what the index does not hold", () => { + it("excludes untracked files, a nested checkout's contents and a sibling checkout", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + yield* writeFixtureFile(fixture.root, "untracked.md", "not tracked\n"); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const listed = paths(seen[0]); + expect(listed).not.toContain("untracked.md"); + // The tracked gitlink is one path. Nothing beneath it, and nothing in the + // untracked checkout beside it, is a path this repository's index holds. + expect(listed).toContain("nested"); + expect(listed.filter((path) => path.startsWith("nested/"))).toEqual([]); + expect(listed.filter((path) => path.startsWith("scratch-repo"))).toEqual([]); + }); + + it("matches prefix literally and answers root-relative paths from a subdirectory", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + '', + "", + "", + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + // An incomplete final segment is a prefix, and the answer is relative to + // the repository root rather than to the directory the element was in. + expect(paths(seen[0])).toEqual(["src/a.ts", "src/added.ts"]); + }); + + it("reads a prefix holding glob characters as those characters", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + expect(paths(seen[0])).toEqual([MAGIC_PATH]); + }); +}); + +describe("GO3 — the three comparisons", () => { + it("compares two commits with every status the fixture produces", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const result = object(seen[0]); + expect(result.comparison).toEqual({ + compare: "commits", + base: fixture.c1, + head: fixture.c2, + }); + expect(result.changes).toEqual([ + { path: "doc/renamed.md", status: "R", previousPath: "doc/orig.md" }, + { path: "gone.ts", status: "D", previousPath: null }, + { path: "link-me.txt", status: "T", previousPath: null }, + { path: "mode.sh", status: "M", previousPath: null }, + { path: "nested", status: "M", previousPath: null }, + { path: "src/added.ts", status: "A", previousPath: null }, + { path: "src/bin.dat", status: "M", previousPath: null }, + ]); + expect(result.next).toBe(null); + }); + + it("compares the index with the working tree, and HEAD with the index", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 2;\n"); + yield* writeFixtureFile(fixture.root, "staged.ts", "export const staged = true;\n"); + fixture.run("add", "--", "staged.ts"); + yield* writeFixtureFile(fixture.root, "untracked.md", "not tracked\n"); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const unstaged = object(seen[0]); + expect(unstaged.comparison).toEqual({ compare: "unstaged" }); + // The untracked file is not a tracked change, so it is not here. + expect(unstaged.changes).toEqual([{ path: "src/a.ts", status: "M", previousPath: null }]); + + const staged = object(seen[1]); + expect(staged.comparison).toEqual({ compare: "staged", base: fixture.c2 }); + expect(staged.changes).toEqual([{ path: "staged.ts", status: "A", previousPath: null }]); + }); + + // Driven at the native performer rather than through a document, because no + // installed provider can select an unborn repository: a selection's identity + // pins a creation commit, and an unborn branch has none. What is under test + // is the comparison a provider would reach if one did, which is the real + // native read against a real unborn checkout. + it("reports a staged addition on an unborn branch against the empty tree", function* () { + const unborn = yield* useUnbornCheckout(); + + const value = yield* performGitObservation( + denoRepositoryHost(), + "", + { + kind: "changes", + comparison: { compare: "staged" }, + paths: null, + limit: 100, + cursor: null, + }, + observationPlaceOf(unborn), + ); + + if (!("changes" in value)) { + throw new Error("expected a changes result"); + } + expect(value.comparison).toEqual({ compare: "staged", base: null }); + expect(value.changes).toEqual([{ path: "first.ts", status: "A", previousPath: null }]); + }); +}); + +describe("GO4 — the patch a comparison produces", () => { + it("answers the comparison's own unified text, with Git's binary and mode metadata", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const result = object(seen[0]); + const patch = String(result.text); + expect(result.comparison).toEqual({ compare: "commits", base: fixture.c1, head: fixture.c2 }); + // Every shape the fixture produces, as patch text rather than as a summary. + expect(patch).toContain("rename from doc/orig.md"); + expect(patch).toContain("rename to doc/renamed.md"); + expect(patch).toContain("Binary files a/src/bin.dat and b/src/bin.dat differ"); + expect(patch).toContain("old mode 100644"); + expect(patch).toContain("new mode 100755"); + expect(patch).toContain("new file mode 100644"); + expect(patch).toContain("deleted file mode 100644"); + expect(patch).toContain("Subproject commit"); + expect(result.offset).toBe(0); + expect(result.length).toBe([...patch].length); + expect(result.next).toBe(null); + }); + + it("narrows to literal files and directories, matching a rename on either side", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ``, + "", + "", + "", + ``, + "", + "", + "", + ``, + "", + "", + "", + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + expect(object(seen[0]).changes).toEqual([ + { path: "src/added.ts", status: "A", previousPath: null }, + { path: "src/bin.dat", status: "M", previousPath: null }, + ]); + // The old path selects the whole rename record, not half of it. + expect(object(seen[1]).changes).toEqual([ + { path: "doc/renamed.md", status: "R", previousPath: "doc/orig.md" }, + ]); + expect(String(object(seen[2]).text)).toContain("rename from doc/orig.md"); + // An empty selector is a thing a document says, and it selects nothing. + expect(object(seen[3]).changes).toEqual([]); + expect(object(seen[4]).text).toBe(""); + expect(object(seen[4]).length).toBe(0); + expect(object(seen[4]).next).toBe(null); + }); + + it("keeps a tracked gitlink's moved pointer without reading inside it", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const patch = String(object(seen[0]).text); + expect(patch).toContain(`-Subproject commit ${fixture.gitlink.before}`); + expect(patch).toContain(`+Subproject commit ${fixture.gitlink.after}`); + // One path and one pointer. Nothing beneath the gitlink is read, so the + // nested repository's own file never appears. + expect(patch).not.toContain("inner.txt"); + }); + + it("collapses an unmerged path to one record", function* () { + const root = yield* useManagedRoot(); + const conflicted = yield* useConflictedCheckout(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + ].join("\n"), + { root, cwd: conflicted.root, components: [capturingProbe(seen)] }, + ); + + expect(object(seen[0]).changes).toEqual([ + { path: "shared.txt", status: "U", previousPath: null }, + ]); + }); +}); + +describe("GO5 — pages are bounded, complete, and about one observation", () => { + it("defaults to 100 records, follows every page once, and honors a larger limit", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(1002); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + expect(paths(seen[0])).toHaveLength(100); + expect(object(seen[0]).next).not.toBe(null); + + // Every page, followed through the ordinary Api with the cursor the + // previous page supplied, until it says there is no more. + const followed: string[] = []; + yield* runOrdinaryDocument("\n", { + root, + cwd: fixture.root, + components: [ + followingProbe("Follows", function* (place) { + let cursor: string | null = null; + for (;;) { + const page: GitFilesResult = yield* GitQuery.operations.files({ + ...place, + request: { kind: "files", prefix: null, limit: 100, cursor }, + }); + followed.push(...page.paths); + if (page.next === null) { + break; + } + cursor = page.next.cursor; + } + }), + ], + }); + expect(followed).toEqual([...fixture.paths]); + expect(new Set(followed).size).toBe(followed.length); + + // One read for all of them, which is the claim that there is no hidden cap. + const whole: Json[] = []; + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(whole)] }, + ); + expect(paths(whole[0])).toEqual([...fixture.paths]); + expect(object(whole[0]).next).toBe(null); + }); + + it("pages changes at a limit of one and at the default", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(1002); + for (const path of fixture.paths) { + yield* writeFixtureFile(fixture.root, path, "changed\n"); + } + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const one = object(seen[0]); + expect(one.changes).toEqual([{ path: fixture.paths[0], status: "M", previousPath: null }]); + expect(one.next).toEqual({ compare: "unstaged", limit: 1, cursor: expect.any(String) }); + + const hundred = object(seen[1]); + expect(Array.isArray(hundred.changes) ? hundred.changes.length : 0).toBe(100); + expect(hundred.next).toEqual({ + compare: "unstaged", + limit: 100, + cursor: expect.any(String), + }); + }); + + it("windows a patch at the default, above 65536, and reassembles the whole text", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(1002); + for (const path of fixture.paths) { + yield* writeFixtureFile(fixture.root, path, "changed\n"); + } + const seen: Json[] = []; + const windows: string[] = []; + let whole = ""; + + yield* runOrdinaryDocument( + ['', "", "", ""].join( + "\n", + ), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + const first = object(seen[0]); + expect([...String(first.text)]).toHaveLength(4096); + expect(first.offset).toBe(0); + expect(first.next).not.toBe(null); + + yield* runOrdinaryDocument("\n", { + root, + cwd: fixture.root, + components: [ + followingProbe("Follows", function* (place) { + let offset = 0; + let cursor: string | null = null; + for (;;) { + const page: GitDiffResult = yield* GitQuery.operations.diff({ + ...place, + request: { + kind: "diff", + comparison: { compare: "unstaged" }, + paths: null, + offset, + limit: 70000, + cursor, + }, + }); + windows.push(page.text); + if (page.next === null) { + whole = windows.join(""); + expect(page.length).toBe([...whole].length); + break; + } + offset = page.next.offset; + cursor = page.next.cursor; + } + }), + ], + }); + + // A limit above the old proposed cap is honored, and the windows are the + // whole patch rather than a prefix of it. + expect([...(windows[0] ?? "")]).toHaveLength(70000); + expect(whole).toBe(fixture.output("diff", "--no-ext-diff", "--no-textconv", "--no-color")); + }); +}); + +describe("GO6 — a changed observation cannot be paged over", () => { + it("refuses a file listing whose paths changed between pages", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(150); + + const raised_ = yield* raised( + runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + ].join("\n"), + { + root, + cwd: fixture.root, + components: [ + actionProbe("Mutates", function* () { + yield* writeFixtureFile(fixture.root, "gen/extra.txt", "extra\n"); + fixture.run("add", "--", "gen/extra.txt"); + }), + ], + }, + ), + ); + + expect(reasonOf(raised_)).toBe("stale-continuation"); + expect(messageOf(raised_)).toBe( + "The Git observation changed. Restart without cursor.", + ); + }); + + // The mutation has to change the *records*, not just the bytes behind one: + // editing a path already reported as modified leaves the normalized listing + // identical, and a read that restarted for that would refuse a page it could + // honestly answer. So this adds a record the first page never described. + it("refuses a changes listing whose records changed between pages", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(150); + for (const path of fixture.paths.slice(0, 120)) { + yield* writeFixtureFile(fixture.root, path, "changed\n"); + } + + const raised_ = yield* raised( + runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + ].join("\n"), + { + root, + cwd: fixture.root, + components: [ + actionProbe("Mutates", function* () { + yield* writeFixtureFile(fixture.root, fixture.paths[149] ?? "", "changed too\n"); + }), + ], + }, + ), + ); + + expect(reasonOf(raised_)).toBe("stale-continuation"); + expect(messageOf(raised_)).toBe( + "The Git observation changed. Restart without cursor.", + ); + }); + + it("refuses a patch whose text changed between windows, and names offset zero", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(150); + for (const path of fixture.paths) { + yield* writeFixtureFile(fixture.root, path, "changed\n"); + } + + const raised_ = yield* raised( + runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + ].join("\n"), + { + root, + cwd: fixture.root, + components: [ + actionProbe("Mutates", function* () { + yield* writeFixtureFile(fixture.root, fixture.paths[0] ?? "", "changed again\n"); + }), + ], + }, + ), + ); + + expect(reasonOf(raised_)).toBe("stale-continuation"); + expect(messageOf(raised_)).toBe( + "The Git observation changed. Restart without cursor and offset={0}.", + ); + }); + + it("follows a continuation unchanged when the repository changed elsewhere", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(150); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { + root, + cwd: fixture.root, + components: [ + capturingProbe(seen), + actionProbe("Mutates", function* () { + // Outside the selected observation: an untracked file changes no + // tracked path, so this page is still a page of the same answer. + yield* writeFixtureFile(fixture.root, "untracked-elsewhere.txt", "elsewhere\n"); + }), + ], + }, + ); + + expect(paths(seen[0])).toEqual([ + "gen/file-00005.txt", + "gen/file-00006.txt", + "gen/file-00007.txt", + "gen/file-00008.txt", + "gen/file-00009.txt", + ]); + }); + + it("refuses a cursor that is malformed, for another read, or for another checkout", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useGeneratedFixture(150); + const other = yield* useGeneratedFixture(150); + const borrowed: string[] = []; + + yield* runOrdinaryDocument("\n", { + root, + cwd: other.root, + components: [ + followingProbe("Follows", function* (place) { + const page: GitFilesResult = yield* GitQuery.operations.files({ + ...place, + request: { kind: "files", prefix: null, limit: 10, cursor: null }, + }); + if (page.next === null) { + throw new Error("the generated fixture should page"); + } + borrowed.push(page.next.cursor); + }), + ], + }); + + for (const cursor of ["not-a-cursor", borrowed[0] ?? ""]) { + const raised_ = yield* raised( + runOrdinaryDocument( + [``, ""].join("\n"), + { root, cwd: fixture.root }, + ), + ); + expect(reasonOf(raised_)).toBe("stale-continuation"); + } + + // A cursor for another kind of read is the same refusal: it describes an + // observation this element never made. + const wrongKind = yield* raised( + runOrdinaryDocument( + [ + '', + "", + '', + "", + ].join("\n"), + { root, cwd: fixture.root }, + ), + ); + expect(reasonOf(wrongKind)).toBe("stale-continuation"); + }); +}); + +describe("GO7 — a file is read as one commit holds it", () => { + it("windows the committed text, stays on its commit, and reassembles it", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useFileFixture(); + const seen: Json[] = []; + const windows: string[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + const first = object(seen[0]); + expect(first.commit).toBe(fixture.f1); + expect(first.path).toBe("big.txt"); + expect(first.offset).toBe(0); + expect(first.length).toBe([...LONG_BLOB].length); + // The default window, counted in code points: the astral character is the + // last one in it and arrives whole. + expect([...String(first.text)]).toHaveLength(4096); + expect(String(first.text).endsWith(ASTRAL)).toBe(true); + expect(first.next).toEqual({ + commit: fixture.f1, + path: "big.txt", + offset: 4096, + limit: 4096, + }); + windows.push(String(first.text)); + + // The branch and the working copy both move before the continuation is + // followed, so a read that resolved `HEAD~1` again would read another file. + fixture.run("commit", "--allow-empty", "-m", "third"); + yield* writeFixtureFile(fixture.root, "big.txt", "working copy\n"); + + yield* runOrdinaryDocument("\n", { + root, + cwd: fixture.root, + components: [ + followingProbe("Follows", function* (place) { + let offset = 4096; + for (;;) { + const page: GitFileResult = yield* GitQuery.operations.file({ + ...place, + request: { + kind: "file", + commit: fixture.f1, + path: "big.txt", + offset, + limit: 70000, + }, + }); + expect(page.commit).toBe(fixture.f1); + windows.push(page.text); + if (page.next === null) { + break; + } + offset = page.next.offset; + } + }), + ], + }); + + expect(windows.join("")).toBe(LONG_BLOB); + }); + + it("answers a middle window, an empty window past the end, and an empty file", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useFileFixture(); + const seen: Json[] = []; + const total = [...LONG_BLOB].length; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ``, + "", + "", + "", + ``, + "", + "", + "", + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + expect(object(seen[0]).text).toBe("aaaaa"); + expect(object(seen[0]).next).toEqual({ + commit: fixture.f1, + path: "big.txt", + offset: 15, + limit: 5, + }); + + for (const past of [object(seen[1]), object(seen[2])]) { + expect(past.text).toBe(""); + expect(past.length).toBe(total); + expect(past.next).toBe(null); + } + expect(object(seen[1]).offset).toBe(total); + expect(object(seen[2]).offset).toBe(total + 500); + + const empty = object(seen[3]); + expect(empty.text).toBe(""); + expect(empty.length).toBe(0); + expect(empty.next).toBe(null); + }); + + it("refuses an unusable revision, an absent path, a tree, a gitlink and a nontext blob", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useFileFixture(); + const main = yield* useObservationFixture(); + + const refusals: { source: string; cwd: string; reason: string }[] = [ + { + source: ``, + cwd: fixture.root, + reason: "invalid-revision", + }, + { + source: ``, + cwd: fixture.root, + reason: "absent-path", + }, + { + source: ``, + cwd: fixture.root, + reason: "nontext-blob", + }, + // Both spellings of the same directory. The argument Git receives is a + // pathspec rather than a lookup, so `dir/` matches the *contents* of the + // directory — and a read that took the first record would answer this + // with the blob at `dir/inside.txt` under a path claiming to be `dir/`. + { + source: ``, + cwd: fixture.root, + reason: "nontext-blob", + }, + // A file is not a directory either way round: a trailing slash on one + // matches nothing at all, which is an absent path rather than that file. + { + source: ``, + cwd: fixture.root, + reason: "absent-path", + }, + { + source: ``, + cwd: main.root, + reason: "nontext-blob", + }, + { + source: ``, + cwd: fixture.root, + reason: "nontext-blob", + }, + { + source: ``, + cwd: fixture.root, + reason: "nontext-blob", + }, + ]; + + for (const entry of refusals) { + const raised_ = yield* raised( + runOrdinaryDocument(`${entry.source}\n`, { root, cwd: entry.cwd }), + ); + expect(`${entry.source}: ${reasonOf(raised_)}`).toBe(`${entry.source}: ${entry.reason}`); + } + }); + + it("still reads a directory's own child by its exact path", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useFileFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + // The control for the two directory refusals above: what they refuse is the + // spelling of a directory, not reading anything beneath one. + expect(object(seen[0]).path).toBe("dir/inside.txt"); + expect(object(seen[0]).text).toBe("inside\n"); + }); + + it("answers a symbolic link's own stored text rather than what it points at", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useFileFixture(); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + ``, + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + // The link's target, as the blob holds it. Nothing followed it. + expect(object(seen[0]).text).toBe("big.txt"); + expect(object(seen[0]).length).toBe(7); + }); +}); + +describe("GO8 — a read runs no repository-defined program and changes nothing", () => { + it("leaves every sentinel unfired and the repository byte-identical", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const sentinels = yield* useSentinelConfiguration(fixture.root, fixture.home); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 3;\n"); + const before = yield* fingerprintCheckout(fixture.root); + const counted = countingOrdinaryHost(); + const scratch: string[] = []; + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + '', + "", + '', + "", + ``, + "", + "", + "", + "", + "", + ].join("\n"), + { + root, + cwd: fixture.root, + components: [capturingProbe(seen)], + host: recordingDirectories(counted.host, scratch), + }, + ); + + // Not one of them ran, and no credential session was opened for a read. + expect(yield* sentinels.fired()).toEqual([]); + expect(counted.counters.sessions).toEqual([]); + // The diff is still the diff: the sentinel would have replaced the text. + expect(String(object(seen[0]).text)).toContain("+export const a = 3;"); + expect(object(seen[1]).text).toBe("export const a = 1;\n"); + + // Nothing in the repository moved, including the index and its mode. + expect(yield* fingerprintCheckout(fixture.root)).toEqual(before); + + // And the scratch each read owned is gone, after the reads were joined. + expect(scratch.length > 0).toBe(true); + for (const directory of scratch) { + expect(yield* exists(directory)).toBe(false); + } + }); + + it("reads a linked checkout's own split index and leaves all three untouched", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const linked = yield* useLinkedWorktree(fixture); + fixture.run("update-index", "--split-index"); + linked.run("update-index", "--split-index"); + yield* writeFixtureFile(linked.root, "src/a.ts", "export const a = 9;\n"); + + const indexes = yield* indexFingerprints(fixture, linked); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { root, cwd: linked.root, components: [capturingProbe(seen)] }, + ); + + // The linked checkout's own index answered: its working tree holds the + // edit, and the primary checkout's does not. + expect(paths(seen[0])).toContain("src/a.ts"); + expect(object(seen[1]).changes).toEqual([ + { path: "src/a.ts", status: "M", previousPath: null }, + ]); + expect(String(object(seen[2]).text)).toContain("+export const a = 9;"); + + expect(yield* indexFingerprints(fixture, linked)).toEqual(indexes); + }); +}); + +describe("GO9 — which checkout answers, and what cannot choose one", () => { + it("distinguishes two sibling checkouts of one repository by their content", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const linked = yield* useLinkedWorktree(fixture); + yield* writeFixtureFile(linked.root, "only-linked.txt", "linked\n"); + linked.run("add", "--", "only-linked.txt"); + linked.run("commit", "-m", "linked only"); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: linked.root, components: [capturingProbe(seen)] }, + ); + expect(paths(seen[0])).toContain("only-linked.txt"); + + const primary: Json[] = []; + yield* runOrdinaryDocument( + ['', "", "", ""].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(primary)] }, + ); + expect(paths(primary[0])).not.toContain("only-linked.txt"); + }); + + it("refuses a foreign selection and an edited one before reaching a host", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const counted = countingOrdinaryHost(); + + const foreign: RepositorySelection = { + selection: "not-one-this-provider-minted", + name: "checkout", + identity: { + name: "checkout", + locatorFingerprint: "0".repeat(64), + requestedBase: null, + creationCommit: fixture.c2, + primaryBranch: "main", + objectFormat: "sha1", + }, + checkoutPath: fixture.root, + }; + + const refusedForeign = yield* raised( + runOrdinaryDocument('\n', { + root, + cwd: fixture.root, + host: counted.host, + contextualRepository: foreign, + }), + ); + expect(refusedForeign instanceof GitOperationAdmissionError).toBe(true); + expect(subcommands(counted.counters)).not.toContain("ls-files"); + + // A selection this provider did mint, with one member changed afterwards. + const edited: RepositorySelection[] = []; + yield* runOrdinaryDocument("\n", { + root, + cwd: fixture.root, + components: [ + followingProbe("Follows", function* (place) { + edited.push({ ...place.repository, name: "renamed" }); + }), + ], + }); + const second = countingOrdinaryHost(); + const refusedEdited = yield* raised( + runOrdinaryDocument('\n', { + root, + cwd: fixture.root, + host: second.host, + contextualRepository: edited[0], + }), + ); + expect(refusedEdited instanceof GitOperationAdmissionError).toBe(true); + expect(subcommands(second.counters)).not.toContain("ls-files"); + }); +}); + +describe("GO10 — the comparison sees an edit the index's own times would hide", () => { + it("reports a modification whose size and recorded time match the index entry", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const indexPath = join(fixture.root, ".git", "index"); + const indexTimes = yield* lstat(indexPath); + // The same length, and the index's own modification time. This is exactly + // the state Git calls racily clean: the cached stat data matches, so the + // only way to see the change is to compare content — and the only reason + // Git does is that the entry is no older than the index holding it. A + // private copy stamped with the current time destroys that relationship. + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 7;\n"); + yield* until(utimes(join(fixture.root, "src", "a.ts"), indexTimes.mtime, indexTimes.mtime)); + const seen: Json[] = []; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { root, cwd: fixture.root, components: [capturingProbe(seen)] }, + ); + + expect(object(seen[0]).changes).toEqual([ + { path: "src/a.ts", status: "M", previousPath: null }, + ]); + expect(String(object(seen[1]).text)).toContain("+export const a = 7;"); + }); +}); + +describe("GO11 — the authored contract, before a provider is reached", () => { + it("refuses a missing binding, a paired form, an extra prop and a mismatched comparison", function* () { + const root = yield* useManagedRoot(); + const fixture = yield* useObservationFixture(); + const counted = countingOrdinaryHost(); + + const refused: { source: string; reason: string }[] = [ + { source: "", reason: "refused" }, + { source: 'content', reason: "unexpected-content" }, + { source: '', reason: "refused" }, + { source: '', reason: "refused" }, + { + source: '', + reason: "invalid-invocation", + }, + { + source: '', + reason: "invalid-invocation", + }, + { source: '', reason: "refused" }, + { source: '', reason: "refused" }, + { + source: '', + reason: "invalid-invocation", + }, + { + source: '', + reason: "invalid-invocation", + }, + ]; + + for (const entry of refused) { + const raised_ = yield* raised( + runOrdinaryDocument(`${entry.source}\n`, { root, cwd: fixture.root, host: counted.host }), + ); + expect(`${entry.source}: ${refusalKind(raised_)}`).toBe(`${entry.source}: ${entry.reason}`); + } + + // Not one of them reached a repository: no read ran, and no `git` for one. + expect(subcommands(counted.counters)).toEqual([]); + }); + + it("declares exactly the four names, forms, props, results and defaults", function* () { + const registered = COMPOSITION_REGISTRATIONS.filter((entry) => + ["Git.Files", "Git.Changes", "Git.Diff", "Git.File"].includes(entry.name), + ); + expect(registered.map((entry) => entry.name)).toEqual([ + "Git.Files", + "Git.Changes", + "Git.Diff", + "Git.File", + ]); + for (const entry of registered) { + // A declared `returns` is what makes `as` required, which is checked + // before the body runs and therefore before any provider is asked. + expect(`${entry.name}: ${entry.returns === undefined ? "open" : "declared"}`).toBe( + `${entry.name}: declared`, + ); + expect(entry.origin).toBe(COMPOSITION_ORIGIN); + expect(entry.forms).toBe(undefined); + } + + expect(Object.keys(propertiesOf(filesProps)).sort()).toEqual(["cursor", "limit", "prefix"]); + expect(Object.keys(propertiesOf(changesProps)).sort()).toEqual([ + "base", + "compare", + "cursor", + "head", + "limit", + "paths", + ]); + expect(Object.keys(propertiesOf(diffProps)).sort()).toEqual([ + "base", + "compare", + "cursor", + "head", + "limit", + "offset", + "paths", + ]); + expect(Object.keys(propertiesOf(fileProps)).sort()).toEqual([ + "commit", + "limit", + "offset", + "path", + ]); + + expect(Object.keys(propertiesOf(filesReturns)).sort()).toEqual(["next", "paths"]); + expect(Object.keys(propertiesOf(changesReturns)).sort()).toEqual([ + "changes", + "comparison", + "next", + ]); + expect(Object.keys(propertiesOf(diffReturns)).sort()).toEqual([ + "comparison", + "length", + "next", + "offset", + "text", + ]); + expect(Object.keys(propertiesOf(fileReturns)).sort()).toEqual([ + "commit", + "length", + "next", + "offset", + "path", + "text", + ]); + + expect(DEFAULT_RECORD_LIMIT).toBe(100); + expect(DEFAULT_TEXT_LIMIT).toBe(4096); + expect([...GIT_COMPARE_MODES]).toEqual(["unstaged", "staged", "commits"]); + }); +}); + +describe("GO12 — availability is the installed provider's, with no fallback", () => { + it("refuses all four on a host that installs nothing for them", function* () { + for (const element of FOUR_ELEMENTS) { + const raised_ = yield* raised(runNeutralDocument(`${element.source}\n`)); + // A neutral host has no Repository provider either, so the refusal is + // the first absent one — and nothing falls back to the caller's own + // filesystem on the way. + expect(`${element.name}: ${raised_ === undefined ? "answered" : "refused"}`).toBe( + `${element.name}: refused`, + ); + } + }); + + it("reaches the reads' own absent-provider refusal once a repository is selected", function* () { + for (const element of FOUR_ELEMENTS) { + const raised_ = yield* raised( + runNeutralDocument(`${element.source}\n`, { repository: true }), + ); + expect(`${element.name}: ${raised_ instanceof GitObservationProviderError}`).toBe( + `${element.name}: true`, + ); + } + }); + + it("answers through the same Api when a host supplies the four methods", function* () { + const asked: string[] = []; + const seen: Json[] = []; + + yield* runNeutralDocument( + [ + '', + "", + "", + "", + '', + "", + "", + "", + '', + "", + "", + "", + '', + "", + "", + "", + ].join("\n"), + { repository: true, reads: asked, components: [capturingProbe(seen)] }, + ); + + // The request each component composed, in the order they ran. + expect(asked).toEqual([ + 'files {"cursor":null,"kind":"files","limit":100,"prefix":"a/"}', + 'changes {"comparison":{"compare":"staged"},"cursor":null,"kind":"changes","limit":100,"paths":null}', + 'diff {"comparison":{"compare":"staged"},"cursor":null,"kind":"diff","limit":4096,"offset":0,"paths":null}', + 'file {"commit":"HEAD~1","kind":"file","limit":4096,"offset":0,"path":"a/b.ts"}', + ]); + // And the typed JSON each one bound. + expect(seen[0]).toEqual({ paths: ["a/b.ts"], next: null }); + expect(seen[1]).toEqual({ + comparison: { compare: "staged", base: null }, + changes: [], + next: null, + }); + expect(seen[2]).toEqual({ + comparison: { compare: "staged", base: null }, + text: "", + offset: 0, + length: 0, + next: null, + }); + expect(seen[3]).toEqual({ + commit: "0".repeat(40), + path: "a/b.ts", + text: "supplied", + offset: 0, + length: 8, + next: null, + }); + }); +}); diff --git a/packages/git/tests/module-partition.test.ts b/packages/git/tests/module-partition.test.ts index 9f06f9283..5e7161c93 100644 --- a/packages/git/tests/module-partition.test.ts +++ b/packages/git/tests/module-partition.test.ts @@ -53,6 +53,7 @@ const PROVIDER_NEUTRAL: readonly string[] = [ "src/composition/components/Issue.ts", "src/composition/components/IssueTracker.ts", "src/composition/components/PullRequest.ts", + "src/composition/components/GitReads.ts", "src/composition/components/PullRequestReads.ts", "src/composition/components/Repository.ts", "src/composition/components/Worktree.ts", @@ -64,6 +65,8 @@ const PROVIDER_NEUTRAL: readonly string[] = [ "src/composition/git-records.ts", "src/composition/installation.ts", "src/composition/locator.ts", + "src/composition/observation-api.ts", + "src/composition/observation-records.ts", "src/composition/parse.ts", "src/composition/pull-request-api.ts", "src/composition/pull-request-operations.ts", @@ -95,6 +98,7 @@ const PROVIDER_NEUTRAL: readonly string[] = [ "src/plugin.ts", "src/retained-repository/admission.ts", "src/retained-repository/git-records.ts", + "src/retained-repository/observation-records.ts", "src/retained-repository/placement-records.ts", "src/retained-repository/push-admission.ts", "src/retained-repository/push-records.ts", @@ -153,6 +157,7 @@ const DENO_ADAPTER: readonly string[] = [ "src/deno/composition/locator.ts", "src/deno/composition/materialize.ts", "src/deno/composition/object-source.ts", + "src/deno/composition/observations.ts", "src/deno/composition/operations.ts", "src/deno/composition/checkout-paths.ts", "src/deno/composition/provider.ts", @@ -169,6 +174,7 @@ const DENO_ADAPTER: readonly string[] = [ "src/deno/retained-repository/issues.ts", "src/deno/retained-repository/leases.ts", "src/deno/retained-repository/local.ts", + "src/deno/retained-repository/observations.ts", "src/deno/retained-repository/provider.ts", "src/deno/retained-repository/push.ts", "src/deno/retained-repository/selections.ts", diff --git a/packages/git/tests/retained-repository-observations-deno.test.ts b/packages/git/tests/retained-repository-observations-deno.test.ts new file mode 100644 index 000000000..df1fa935e --- /dev/null +++ b/packages/git/tests/retained-repository-observations-deno.test.ts @@ -0,0 +1,265 @@ +/** + * Tier ROB5, the half that needs a runtime which operates repositories. + * + * Three cases, kept apart from `retained-repository-observations.test.ts` + * because each one needs the native side of a REPL entry's repository host. + * Cancelling a finished read's append unwinds a real Deno-backed Files + * provider; holding a read while the same entry mutates needs a real `git` + * child to reach; and another holder's mutation lease is a kernel-released + * exclusive advisory lock taken through the Deno runtime. Node and Bun expose + * no equivalent of the lock, install no native provider for the mutation to + * reach, and unwind the Files provider differently, so there these three fail + * on the host rather than on what they assert. + * + * The record format, its admission, every refusal and the rest of + * cancellation and persistence stay portable and stay in + * `retained-repository-observations.test.ts` — ROB1 through ROB4, and ROB5's + * interrupted-child and fatal-append cases, all run on every runtime. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { race, scoped, sleep, spawn, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { exists } from "@effectionx/fs"; +import { join } from "node:path"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, DurableStream, Json } from "@executablemd/durable-streams"; +import { registerComponents } from "@executablemd/core"; +import { + parseReplObservationRecord, + REPL_REPOSITORY_OBSERVATION, +} from "../src/retained-repository/observation-records.ts"; +import { useCommonDirectoryLease } from "../src/deno/retained-repository/leases.ts"; +import { useObservationFixture, writeFixtureFile } from "./support/observations.ts"; +import { + entryCounters, + entrySubcommands, + repoint, + runEntry, + useEntryPlace, +} from "./support/repl-observations.ts"; +import type { EntryCounters } from "./support/repl-observations.ts"; +import { commonDirectoryOf } from "./support/run-composition.ts"; + +function collecting( + events: DurableEvent[], + before?: (event: DurableEvent) => Operation, +): DurableStream { + return { + *readAll(): Operation { + return [...events]; + }, + *append(event: DurableEvent): Operation { + if (before !== undefined) { + yield* before(event); + } + events.push(event); + }, + }; +} + +/** The native reads one entry performed, which is what a read's `git` is. */ +const READ_SUBCOMMANDS: readonly string[] = ["ls-files", "diff", "cat-file", "ls-tree"]; + +function reads(counted: EntryCounters): string[] { + // By exact argument rather than by "the first word that is not an option": + // every read states `-c ` pre-command options, and a setting's value + // is not an option either. + return counted.commands + .map((args) => args.find((arg) => READ_SUBCOMMANDS.includes(arg))) + .filter((name): name is string => name !== undefined); +} + +function observations(events: readonly DurableEvent[]) { + return events + .filter( + (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_OBSERVATION, + ) + .map((event) => { + if (event.type !== "yield" || event.result.status !== "ok") { + throw new Error("an observation record is not a successful envelope"); + } + const parsed = parseReplObservationRecord(event.result.value); + if (parsed === undefined) { + throw new Error("an observation record did not parse"); + } + return { name: event.description.name, record: parsed }; + }); +} + +describe("ROB5 — cancellation, persistence and ownership, on a native host", () => { + it("leaves no completion when a finished read's append is cancelled", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 13;\n"); + const events: DurableEvent[] = []; + const reached = withResolvers(); + const held = withResolvers(); + + // A barrier between the native work and its acknowledgement: the read has + // happened, and the append has not. + const barrier = collecting(events, function* (event) { + if (event.type === "yield" && event.description.type === REPL_REPOSITORY_OBSERVATION) { + reached.resolve(); + yield* held.operation; + } + }); + + const source = [ + "", + '', + "Diff: {diff.length}", + "", + 'sentinel', + ].join("\n"); + + yield* scoped(function* () { + const task = yield* spawn(() => runEntry(place, source, barrier, counted, {})); + yield* reached.operation; + yield* task.halt(); + }); + + // The read ran, and nothing says it did. + expect(reads(counted)).toContain("diff"); + expect(observations(events)).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + + // So a continuation takes the live path, and gets today's answer. + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 17;\n"); + const again = entryCounters(); + const continued = yield* runEntry(place, source, new InMemoryStream([...events]), again); + expect(continued.refusal).toBe(undefined); + expect(reads(again)).toContain("diff"); + const recorded = observations(continued.events); + expect(recorded).toHaveLength(1); + const outcome = recorded[0]?.record.outcome; + expect(outcome?.ok === true && "text" in outcome.value ? outcome.value.text : "").toContain( + "+export const a = 17;", + ); + }); + + it("keeps the same entry's mutation out of a held read, then lets it through", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 123;\n"); + + const readHeld = withResolvers(); + const releaseRead = withResolvers(); + const mutationEntered = withResolvers(); + let holding = false; + let enteredWhileHeld = false; + + const settled = yield* scoped(function* () { + const task = yield* spawn(() => + runEntry( + place, + [ + "", + '', + '', + "", + "", + ].join("\n"), + new InMemoryStream(), + counted, + { + *between(): Operation { + // The second branch starts its mutation only once the read is + // genuinely held, so the race the regression needs is arranged + // rather than hoped for. + yield* registerComponents([ + { + name: "WaitsForTheRead", + origin: "test://waits-for-the-read", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* readHeld.operation; + return ""; + }, + }, + ]); + }, + *barrier(invocation): Operation { + if (invocation.args.includes("add")) { + if (holding) { + enteredWhileHeld = true; + } + mutationEntered.resolve(); + return; + } + if (invocation.args.includes("diff") && !holding) { + holding = true; + readHeld.resolve(); + // Long enough that a mutation on a queue of its own would have + // reached the host by now, and short enough to end the case. + yield* race([ + mutationEntered.operation, + (function* (): Operation { + yield* sleep(2000); + })(), + ]); + holding = false; + releaseRead.resolve(); + } + }, + }, + ), + ); + yield* releaseRead.operation; + // The read has finished its own turn; the mutation may now proceed. + yield* race([ + mutationEntered.operation, + (function* (): Operation { + yield* sleep(30_000); + throw new Error("the mutation never reached native Git"); + })(), + ]); + return yield* task; + }); + + // Nothing of this entry's own Git entered while its read held the turn, + // and the mutation did run once the read released it. + expect(enteredWhileHeld).toBe(false); + expect(settled.refusal).toBe(undefined); + expect(reads(counted)).toContain("diff"); + expect(entrySubcommands(counted)).toContain("add"); + }); + + it("answers while another holder has the checkout's mutation lease", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const common = commonDirectoryOf({ root: fixture.root, home: fixture.home, run: fixture.run }); + const counted = entryCounters(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 23;\n"); + + const source = [ + '', + "", + "", + "", + ].join("\n"); + + // Held for the whole of both runs by somebody else. A read that took one + // would be refused by the operating system on the second descriptor. + const live = yield* scoped(function* () { + yield* useCommonDirectoryLease(common); + return yield* runEntry(place, source, new InMemoryStream(), counted); + }); + expect(live.refusal).toBe(undefined); + expect(reads(counted)).toContain("diff"); + + // And a completed replay takes neither the lease nor anything else: it + // answers with the lease still held and the checkout gone. + const replayed = entryCounters(); + const again = yield* scoped(function* () { + yield* useCommonDirectoryLease(common); + yield* repoint(place, place.root); + return yield* runEntry(place, source, new InMemoryStream([...live.events]), replayed); + }); + expect(again.output).toBe(live.output); + expect(entrySubcommands(replayed)).toEqual([]); + }); +}); diff --git a/packages/git/tests/retained-repository-observations.test.ts b/packages/git/tests/retained-repository-observations.test.ts new file mode 100644 index 000000000..d65aa8452 --- /dev/null +++ b/packages/git/tests/retained-repository-observations.test.ts @@ -0,0 +1,913 @@ +/** + * Tier ROB — what one REPL entry's local Git reads retain, replay and refuse + * to guess. + * + * Everything here runs a document through canonical execution with the + * production entry host installed, so what is proven is the whole path: the + * authored component asks the contextual Api, the contained performer runs real + * `git` against a real checkout, and the page is recorded once. A fixture that + * asserted on the record without the performer would prove the bookkeeping and + * nothing about the answer. + * + * The contract these rows hold the host to is the REPL's execution model. + * Replay restores recorded pages; it does not restore the checkout or require + * it to match. So the rows that matter most are the ones where the two + * disagree: a checkout whose files have all changed behind a completed listing, + * a selected checkout deleted outright, and a read that physically happened and + * was never acknowledged. + * + * ## How "nothing was resolved" is proven + * + * Counting `git` is not enough: routing a live read canonicalizes its working + * directory and chooses among this entry's registered checkouts, and neither + * touches `git`. So the directory an entry is opened in is a symbolic link, and + * a continuation runs with that link pointing somewhere else entirely. Live + * routing would resolve it, find it inside none of the checkouts this entry + * registered, and fail admission. A continuation that answers normally has + * therefore resolved nothing and selected nothing. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { Err, Ok, race, scoped, sleep, spawn, suspend, until, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { exists, rm, writeTextFile } from "@effectionx/fs"; +import { chmod } from "node:fs/promises"; +import { join } from "node:path"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, DurableStream, Json } from "@executablemd/durable-streams"; +import { GitQuery } from "@executablemd/git/api"; +import type { GitDiffInvocation, GitDiffResult, GitObservationValue } from "@executablemd/git/api"; +import { parseGitObservationValue } from "../src/composition/observation-records.ts"; +import { GitObservationError } from "../src/composition/errors.ts"; +import { + elementOf, + parseReplObservationRecord, + replObservationFingerprint, + replObservationRecordJson, + replObservationRecordName, + REPL_REPOSITORY_OBSERVATION, +} from "../src/retained-repository/observation-records.ts"; +import type { + ReplObservationCheckout, + ReplObservationRequest, +} from "../src/retained-repository/observation-records.ts"; +import { admitReplRepositoryHistory } from "../src/retained-repository/admission.ts"; +import { Elicitation, registerComponents } from "@executablemd/core"; +import { useObservationFixture, writeFixtureFile } from "./support/observations.ts"; +import type { EntryCounters } from "./support/repl-observations.ts"; +import { + appears, + entryCounters, + entrySubcommands, + leftEntry, + repoint, + running, + runEntry, + useEntryPlace, +} from "./support/repl-observations.ts"; +import type { GitInvocation } from "../src/deno/composition/host.ts"; +import { useCommonDirectoryLease } from "../src/deno/retained-repository/leases.ts"; +import { commonDirectoryOf } from "./support/run-composition.ts"; + +/** One journal event standing for a saved observation record. */ +function recordEvent(name: string, payload: Json): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: REPL_REPOSITORY_OBSERVATION, name }, + result: { status: "ok", value: payload }, + }; +} + +/** This payload as an object a case can change one member of. */ +function objectOf(payload: Json): Record { + if (typeof payload !== "object" || payload === null || Array.isArray(payload)) { + throw new Error("expected a record payload"); + } + return payload; +} + +/** One page, as the writer needs it typed. */ +function pageOf(page: Json | undefined): GitObservationValue { + const parsed = page === undefined ? undefined : parseGitObservationValue(page, FILES_QUERY); + if (parsed !== undefined) { + return parsed; + } + if (typeof page !== "object" || page === null || Array.isArray(page)) { + throw new Error("expected a page object"); + } + // Every page in this case is already the shape its own writer accepts; the + // files parser above is only the cheapest way to narrow the common one. + return unchecked(page); +} + +const FILES_QUERY = Object.freeze({ + kind: "files" as const, + prefix: null, + limit: 100, + cursor: null, +}); + +/** The record's own writers refuse a value of the wrong shape, so this narrows. */ +function unchecked(page: Record): GitObservationValue { + const changes = parseGitObservationValue(page, { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: null, + limit: 100, + cursor: null, + }); + if (changes !== undefined) { + return changes; + } + const diff = parseGitObservationValue(page, { + kind: "diff", + comparison: { compare: "unstaged" }, + paths: null, + offset: 0, + limit: 4096, + cursor: null, + }); + if (diff !== undefined) { + return diff; + } + const file = parseGitObservationValue(page, { + kind: "file", + commit: String(page.commit ?? ""), + path: String(page.path ?? ""), + offset: 0, + limit: 4096, + }); + if (file === undefined) { + throw new Error("expected a page of one of the four shapes"); + } + return file; +} + +/** + * A journal that collects what it is given, with one hook before each append. + * + * The hook is where a row cuts or refuses: holding it keeps the native work + * done and the acknowledgement undone, and throwing from it is the engine's own + * persistence failure. + */ +function collecting( + events: DurableEvent[], + before?: (event: DurableEvent) => Operation, +): DurableStream { + return { + *readAll(): Operation { + return [...events]; + }, + *append(event: DurableEvent): Operation { + if (before !== undefined) { + yield* before(event); + } + events.push(event); + }, + }; +} + +/** What a watched `git` child reported about its own lifetime. */ +interface WatchedChildren { + readonly spawned: Operation; + readonly observe: NonNullable; + observed(): number; + closed(): boolean; + alive(): boolean; +} + +/** + * Watch the real children one invocation spawns. + * + * The production host forwards this to the one place a child exists, which is + * the only point at which a row can see the process a command really started. + * A wrapper around the host sees the call and its answer and never the child + * between them, so observing residue afterwards cannot tell an interrupted + * command from one that never began. + */ +function watchChildren(): WatchedChildren { + const pids: number[] = []; + let count = 0; + let closes = 0; + const first = withResolvers(); + return { + spawned: first.operation, + observe(child): void { + count += 1; + if (child.pid !== undefined) { + pids.push(child.pid); + } + child.on("close", () => { + closes += 1; + }); + first.resolve(); + }, + observed: () => count, + closed: () => count > 0 && closes === count, + alive: () => pids.some((pid) => running(pid)), + }; +} + +/** + * A script that reports it ran and then waits. + * + * What makes a child cut deterministic: the marker says the command really + * reached this point, and the wait keeps it there until the halt arrives. The + * process group teardown is what ends it. + */ +function* waitingScript(path: string, marker: string): Operation { + yield* writeTextFile(path, `#!/bin/sh\ntouch ${JSON.stringify(marker)}\nsleep 120\n`); + yield* until(chmod(path, 0o755)); +} + +/** The one observation payload this history holds, as an object. */ +function soundPayload(events: readonly DurableEvent[]): Json { + for (const event of events) { + if ( + event.type === "yield" && + event.description.type === REPL_REPOSITORY_OBSERVATION && + event.result.status === "ok" + ) { + return event.result.value ?? null; + } + } + throw new Error("this history holds no observation record"); +} + +/** The route member of one observation payload. */ +function soundCheckout(payload: Json): Json { + const checkout = objectOf(payload).checkout; + if (checkout === undefined) { + throw new Error("this observation payload holds no checkout"); + } + return checkout; +} + +/** + * This history with its one observation replaced, and everything else intact. + * + * The placements, the ambient routing and the terminal stay exactly as the + * entry wrote them, which is what makes a refusal below about the record that + * changed rather than about a history missing its context. + */ +function replacingObservation( + events: readonly DurableEvent[], + name: string, + payload: Json, +): readonly DurableEvent[] { + return events.map((event) => { + if (event.type !== "yield" || event.description.type !== REPL_REPOSITORY_OBSERVATION) { + return event; + } + return { + ...event, + description: { ...event.description, name }, + result: { status: "ok" as const, value: payload }, + }; + }); +} + +/** The native reads one entry performed, which is what a read's `git` is. */ /** The native reads one entry performed, which is what a read's `git` is. */ +const READ_SUBCOMMANDS: readonly string[] = ["ls-files", "diff", "cat-file", "ls-tree"]; + +function reads(counted: EntryCounters): string[] { + // By exact argument rather than by "the first word that is not an option": + // every read states `-c ` pre-command options, and a setting's value + // is not an option either. + return counted.commands + .map((args) => args.find((arg) => READ_SUBCOMMANDS.includes(arg))) + .filter((name): name is string => name !== undefined); +} + +/** The records of this feature's own observation type, parsed. */ +function observations(events: readonly DurableEvent[]) { + return events + .filter( + (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_OBSERVATION, + ) + .map((event) => { + if (event.type !== "yield" || event.result.status !== "ok") { + throw new Error("an observation record is not a successful envelope"); + } + const parsed = parseReplObservationRecord(event.result.value); + if (parsed === undefined) { + throw new Error("an observation record did not parse"); + } + return { name: event.description.name, record: parsed }; + }); +} + +/** The place and request a hand-built record is named by. */ +function requestOf(fixture: { root: string; c1: string }, common: string): ReplObservationRequest { + return Object.freeze({ + repository: Object.freeze({ + name: "checkout", + locatorFingerprint: "a".repeat(64), + requestedBase: null, + creationCommit: fixture.c1, + primaryBranch: "main", + objectFormat: "sha1" as const, + }), + selectionCheckout: fixture.root, + commonDirectory: common, + workingDirectory: fixture.root, + query: Object.freeze({ + kind: "files" as const, + prefix: null, + limit: 100, + cursor: null, + }), + }); +} + +function checkoutOf(fixture: { root: string }, common: string): ReplObservationCheckout { + return Object.freeze({ + root: fixture.root, + commonDirectory: common, + repositoryName: "checkout", + worktreeName: null, + }); +} + +describe("ROB1 — a completed read answers from its record", () => { + it("restores all four pages with no git, through a link pointing elsewhere", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const first = entryCounters(); + const source = [ + '', + "", + '', + "", + '', + "", + ``, + "", + "", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"); + const stream = new InMemoryStream(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 5;\n"); + const live = yield* runEntry(place, source, stream, first); + expect(live.refusal).toBe(undefined); + const recorded = observations(live.events); + expect(recorded.map((entry) => entry.record.operation)).toEqual([ + "files", + "changes", + "diff", + "file", + ]); + + // Everything the first entry read is now wrong: the files are gone, the + // checkout is gone, and the directory the entry was opened through points + // somewhere else. + yield* rm(fixture.root, { recursive: true, force: true }); + yield* repoint(place, place.root); + + const second = entryCounters(); + const again = yield* runEntry(place, source, new InMemoryStream([...live.events]), second); + expect(again.refusal).toBe(undefined); + expect(again.output).toBe(live.output); + // Not one `git`, no scratch directory, and no session. + expect(entrySubcommands(second)).toEqual([]); + expect(second.directories).toBe(0); + expect(second.sessions).toEqual([]); + }); +}); + +describe("ROB2 — the record is closed, and named by the request it carries", () => { + it("round-trips all four pages and one refusal", function* () { + const fixture = yield* useObservationFixture(); + const common = commonDirectoryOf({ root: fixture.root, home: fixture.home, run: fixture.run }); + const request = requestOf(fixture, common); + const checkout = checkoutOf(fixture, common); + const fingerprint = yield* replObservationFingerprint(request); + + const pages: Json[] = [ + { paths: ["a.ts"], next: null }, + { comparison: { compare: "unstaged" }, changes: [], next: null }, + { comparison: { compare: "unstaged" }, text: "patch", offset: 0, length: 5, next: null }, + { + commit: fixture.c1, + path: "src/a.ts", + text: "text", + offset: 0, + length: 4, + next: null, + }, + ]; + const queries: ReplObservationRequest["query"][] = [ + { kind: "files", prefix: null, limit: 100, cursor: null }, + { + kind: "changes", + comparison: { compare: "unstaged" }, + paths: null, + limit: 100, + cursor: null, + }, + { + kind: "diff", + comparison: { compare: "unstaged" }, + paths: null, + offset: 0, + limit: 4096, + cursor: null, + }, + { kind: "file", commit: fixture.c1, path: "src/a.ts", offset: 0, limit: 4096 }, + ]; + + for (const [index, query] of queries.entries()) { + const own: ReplObservationRequest = { ...request, query }; + const digest = yield* replObservationFingerprint(own); + const payload = replObservationRecordJson({ + fingerprint: digest, + request: own, + checkout, + outcome: Ok(pageOf(pages[index])), + }); + const parsed = parseReplObservationRecord(payload); + expect(`${query.kind}: ${parsed === undefined ? "unreadable" : "read"}`).toBe( + `${query.kind}: read`, + ); + expect(parsed?.request.query).toEqual(query); + expect(parsed?.outcome.ok).toBe(true); + // And the name is the digest of the request, so the record and the + // question it answers cannot be separated. + expect(replObservationRecordName({ operation: query.kind, fingerprint: digest })).toBe( + replObservationRecordName({ + operation: query.kind, + fingerprint: yield* replObservationFingerprint(own), + }), + ); + } + + const refused = replObservationRecordJson({ + fingerprint, + request, + checkout, + outcome: Err( + new GitObservationError( + elementOf("files"), + "unusable-repository", + " could not read its tracked files.", + ), + ), + }); + const parsedRefusal = parseReplObservationRecord(refused); + expect(parsedRefusal?.outcome.ok).toBe(false); + expect(parsedRefusal?.outcome.ok === false ? parsedRefusal.outcome.error.message : "").toBe( + " could not read its tracked files.", + ); + }); + + // Over a real production history rather than a hand-built record, because the + // route check below is a relationship *between* records: an observation names + // the checkout it read, and what authorizes that is a placement the same + // history made. A lone observation has no history to be routed by. + it("refuses a version, a fingerprint, a name, a page and a route it cannot read", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const live = yield* runEntry( + place, + '\n', + new InMemoryStream(), + entryCounters(), + ); + expect(live.refusal).toBe(undefined); + const recorded = observations(live.events); + expect(recorded).toHaveLength(1); + const name = recorded[0]?.name ?? ""; + const sound = soundPayload(live.events); + + // The control: this history is admitted as it stands, so every refusal + // below is about the one thing it changed. + yield* admitReplRepositoryHistory(live.events); + + const broken: [string, string, Json][] = [ + ["a stated version this host does not implement", name, { ...objectOf(sound), version: 2 }], + [ + "a fingerprint that is not the request's digest", + name, + { ...objectOf(sound), fingerprint: "0".repeat(64) }, + ], + ["a name its own request does not produce", `${name} `, sound], + [ + "a page holding more records than the limit asked for", + name, + { + ...objectOf(sound), + outcome: { + ok: true, + value: { paths: Array.from({ length: 101 }, (_, i) => `p${i}`), next: null }, + }, + }, + ], + ["an operation the request's query is not", name, { ...objectOf(sound), operation: "diff" }], + [ + "a checkout route this version did not write", + name, + { ...objectOf(sound), checkout: { root: fixture.root } }, + ], + // Fully shaped, and wrong. Every required member is present, the request, + // the fingerprint and the journaled name are untouched, and only the root + // names a checkout no placement in this history selected. + [ + "a complete checkout no placement in this history selected", + name, + { + ...objectOf(sound), + checkout: { ...objectOf(soundCheckout(sound)), root: "/unregistered/other-checkout" }, + }, + ], + [ + "a complete checkout under another repository's name", + name, + { + ...objectOf(sound), + checkout: { ...objectOf(soundCheckout(sound)), repositoryName: "elsewhere" }, + }, + ], + [ + "a complete checkout claiming to be a worktree", + name, + { + ...objectOf(sound), + checkout: { ...objectOf(soundCheckout(sound)), worktreeName: "side" }, + }, + ], + [ + "a complete checkout under another Git directory", + name, + { + ...objectOf(sound), + checkout: { + ...objectOf(soundCheckout(sound)), + commonDirectory: "/unregistered/other-checkout/.git", + }, + }, + ], + ]; + + for (const [named, recordName, payload] of broken) { + let refusal: unknown; + try { + yield* admitReplRepositoryHistory(replacingObservation(live.events, recordName, payload)); + } catch (error) { + refusal = error; + } + expect(`${named}: ${refusal === undefined ? "admitted" : "refused"}`).toBe( + `${named}: refused`, + ); + } + }); + + it("refuses a well-shaped wrong route before reusing a completed or unfinished page", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const source = ['', "", "", ""].join("\n"); + + // A settled entry, and the same entry left at a question: the route has to + // be refused before either is reused, and a completed terminal is reused + // without the body that checks the live registry ever being entered. + const settled = yield* runEntry(place, source, new InMemoryStream(), entryCounters()); + expect(settled.refusal).toBe(undefined); + const unfinished = yield* leftEntry( + place, + [source, 'Carry on?', ""].join("\n"), + entryCounters(), + ); + + for (const [named, history] of [ + ["a completed history", settled.events], + ["an unfinished history", unfinished], + ] as const) { + const elsewhere = replacingObservation(history, observations(history)[0]?.name ?? "", { + ...objectOf(soundPayload(history)), + checkout: { + ...objectOf(soundCheckout(soundPayload(history))), + root: "/unregistered/other-checkout", + }, + }); + const counted = entryCounters(); + const replayed = yield* runEntry(place, source, new InMemoryStream([...elsewhere]), counted); + // Refused through the production trusted admission, so no page is bound + // and the original output is not reproduced. + expect(`${named}: ${replayed.refusal === undefined ? "reused" : "refused"}`).toBe( + `${named}: refused`, + ); + expect(`${named}: ${replayed.refusal ?? ""}`).toContain( + "names a checkout no placement in this history selected", + ); + expect(replayed.output).not.toContain("src/a.ts"); + // And decided from the history alone: nothing opened a checkout, resolved + // a path or ran `git` to find out. + expect(`${named}: ${entrySubcommands(counted).join(",")}`).toBe(`${named}: `); + expect(counted.directories).toBe(0); + } + }); + + it("admits a completed page whose checkout has been deleted since", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const source = ['', "", "", ""].join("\n"); + const live = yield* runEntry(place, source, new InMemoryStream(), entryCounters()); + expect(live.refusal).toBe(undefined); + + // The control the route check must not break: a cold reuse whose checkout + // is gone. The route is authorized by the history's own placement, and + // nothing consults a filesystem to decide it. + yield* rm(fixture.root, { recursive: true, force: true }); + yield* repoint(place, place.root); + const counted = entryCounters(); + const again = yield* runEntry(place, source, new InMemoryStream([...live.events]), counted); + expect(again.refusal).toBe(undefined); + expect(again.output).toBe(live.output); + expect(entrySubcommands(counted)).toEqual([]); + }); + + it("refuses a failed envelope under this type, and leaves older histories alone", function* () { + let refusal: unknown; + try { + yield* admitReplRepositoryHistory([ + { + type: "yield", + coroutineId: "root", + description: { type: REPL_REPOSITORY_OBSERVATION, name: "x" }, + result: { status: "err", error: { message: "boom" } }, + }, + ]); + } catch (error) { + refusal = error; + } + expect(refusal === undefined ? "admitted" : "refused").toBe("refused"); + + // A history with none of this type in it is unchanged by the new reader. + yield* admitReplRepositoryHistory([ + { + type: "yield", + coroutineId: "root", + description: { type: "some_other_feature", name: "y" }, + result: { status: "ok", value: { anything: true } }, + }, + ]); + }); +}); + +describe("ROB3 — ordinary middleware keeps its ordinary meaning", () => { + it("creates no record when a replacement answers above the provider", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + const answered: string[] = []; + + const run = yield* runEntry( + place, + ['', "", "", ""].join("\n"), + new InMemoryStream(), + counted, + { + *between(): Operation { + yield* GitQuery.around({ + // deno-lint-ignore require-yield + *diff([invocation]: [GitDiffInvocation]): Operation { + answered.push(invocation.request.kind); + return { + comparison: { compare: "unstaged" }, + text: "answered above", + offset: 0, + length: 14, + next: null, + }; + }, + }); + }, + }, + ); + + expect(run.refusal).toBe(undefined); + expect(answered).toEqual(["diff"]); + expect(run.output).toContain("answered above"); + // Answered entirely above the provider, so no provider was reached: no + // native read, no scratch directory and no observation record. + expect(observations(run.events)).toEqual([]); + // Ambient discovery still happened — the component asks which repository is + // in scope before it asks anything of it — but not one native *read* ran. + expect(reads(counted)).toEqual([]); + }); + + it("records the request a delegating replacement actually passed down", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + + const run = yield* runEntry( + place, + ['', ""].join("\n"), + new InMemoryStream(), + counted, + { + *between(): Operation { + yield* GitQuery.around({ + diff: ([invocation]: [GitDiffInvocation], next): Operation => + next({ ...invocation, request: { ...invocation.request, limit: 7 } }), + }); + }, + }, + ); + + expect(run.refusal).toBe(undefined); + const recorded = observations(run.events); + expect(recorded).toHaveLength(1); + // The request the provider received, not the one the document wrote. + expect(recorded[0]?.record.request.query).toEqual({ + kind: "diff", + comparison: { compare: "unstaged" }, + paths: null, + offset: 0, + limit: 7, + cursor: null, + }); + }); +}); + +describe("ROB4 — a saved refusal replays, and a new read sees today", () => { + it("retains an ordinary bad-revision refusal and raises it again with no git", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const first = entryCounters(); + const source = [ + "", + '', + "Source: {source}", + "", + "", + ].join("\n"); + + const live = yield* runEntry(place, source, new InMemoryStream(), first, {}); + expect(live.refusal).toBe(undefined); + expect(live.output).toContain("names no commit in the selected repository"); + const recorded = observations(live.events); + expect(recorded).toHaveLength(1); + expect(recorded[0]?.record.outcome.ok).toBe(false); + + const replayed = entryCounters(); + const again = yield* runEntry(place, source, new InMemoryStream([...live.events]), replayed); + expect(again.output).toBe(live.output); + expect(entrySubcommands(replayed)).toEqual([]); + }); + + it("restores a prefix and performs the first unrecorded read against new data", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const first = entryCounters(); + const source = [ + '', + "", + "", + "", + 'Carry on?', + "", + '', + "", + "", + "", + ].join("\n"); + + const prefix = yield* leftEntry(place, source, first); + expect(observations(prefix)).toHaveLength(1); + + // New data behind the question: a tracked path the saved page never held. + yield* writeFixtureFile(fixture.root, "src/late.ts", "export const late = true;\n"); + fixture.run("add", "--", "src/late.ts"); + + const resumed = entryCounters(); + const again = yield* runEntry(place, source, new InMemoryStream([...prefix]), resumed, { + *between(): Operation { + yield* Elicitation.around({ + // deno-lint-ignore require-yield + *elicit(): Operation { + return {}; + }, + }); + }, + }); + + expect(again.refusal).toBe(undefined); + const pages = observations(again.events); + expect(pages).toHaveLength(2); + // The prefix restored the page it saved, and the suffix read live. + const before = pages[0]?.record.outcome; + const after = pages[1]?.record.outcome; + expect( + before?.ok === true && "paths" in before.value ? [...before.value.paths] : [], + ).not.toContain("src/late.ts"); + expect(after?.ok === true && "paths" in after.value ? [...after.value.paths] : []).toContain( + "src/late.ts", + ); + }); +}); + +describe("ROB5 — cancellation, persistence and ownership", () => { + it("joins an interrupted read's child and removes its scratch before settling", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 11;\n"); + const watched = watchChildren(); + const marker = join(place.root, "read-ran"); + const script = join(place.root, "waiting-fsmonitor"); + yield* waitingScript(script, marker); + + // The comparison refreshes the index before it answers, and a file-system + // monitor is consulted while it does. This one reports that it ran and then + // waits, holding the pipes the parent inherited — which is what makes the + // cut land on a real child rather than between commands. + const intercept = (invocation: GitInvocation): Partial | undefined => + invocation.args.includes("diff") + ? { + args: ["-c", `core.fsmonitor=${script}`, ...invocation.args], + observe: watched.observe, + } + : undefined; + + const events: DurableEvent[] = []; + const journal = collecting(events); + yield* scoped(function* () { + const task = yield* spawn(() => + runEntry( + place, + [ + '', + 'sentinel', + ].join("\n"), + journal, + counted, + { intercept }, + ), + ); + // The marker says the descendant really ran and is holding the pipes. + yield* appears(marker); + yield* watched.spawned; + yield* task.halt(); + }); + + // The child is gone and both its pipes have ended; the scratch the read + // owned is gone with it; and nothing after the read ran. + expect(watched.observed() > 0).toBe(true); + expect(watched.closed()).toBe(true); + expect(watched.alive()).toBe(false); + expect(counted.scratch.length > 0).toBe(true); + for (const directory of counted.scratch) { + expect(yield* exists(directory)).toBe(false); + } + expect(observations(events)).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + }); + + it("keeps a failed page append fatal, with no binding and nothing after", function* () { + const fixture = yield* useObservationFixture(); + const place = yield* useEntryPlace(fixture); + const counted = entryCounters(); + yield* writeFixtureFile(fixture.root, "src/a.ts", "export const a = 19;\n"); + const events: DurableEvent[] = []; + const rejecting = collecting(events, function* (event) { + if (event.type === "yield" && event.description.type === REPL_REPOSITORY_OBSERVATION) { + throw new Error("the journal refused this event"); + } + }); + + const performed = yield* runEntry( + place, + [ + "", + '', + "Diff: {diff.length}", + "", + 'sentinel', + ].join("\n"), + rejecting, + counted, + ); + + // The read happened; the durability failure is fatal; no success binding + // was offered; and nothing after it ran, printing region or not. + expect(reads(counted)).toContain("diff"); + const reported = performed.refusal ?? performed.output; + expect(reported).not.toContain("sentinel"); + expect(observations(events)).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + }); + + // The two halves of the same ownership rule, in one entry and then across + // two. Inside one entry a read owns the native turn and a mutation waits; + // between entries a mutation lease is a different mechanism and a read does + // not wait for it. +}); diff --git a/packages/git/tests/support/observations.ts b/packages/git/tests/support/observations.ts new file mode 100644 index 000000000..db5da3471 --- /dev/null +++ b/packages/git/tests/support/observations.ts @@ -0,0 +1,470 @@ +/** + * The real repository the local Git read suites observe. + * + * Everything here is a real checkout, real commits and real `git`. The fixture + * is deliberately awkward in the ways a read has to survive: a binary file, a + * mode change, a rename, a deletion, a type change, a tracked gitlink whose + * pointer moves, a filename holding a space, a tab, a newline and a character + * outside ASCII, a filename holding the characters a glob would expand, an + * untracked file, an untracked nested checkout and a sibling checkout of the + * same origin. + * + * Nothing here is substituted. A suite that wants to know whether a read ran a + * repository-defined program, or changed a byte of the repository it read, + * needs a repository that really holds one and really could. + */ + +import { until } from "effection"; +import type { Operation } from "effection"; +import { ensureDir, exists, lstat, readdir, readTextFile, rm } from "@effectionx/fs"; +import { chmod, readFile, readlink, symlink, writeFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import { join } from "node:path"; +import { realpath } from "node:fs/promises"; +import type { ComponentRegistration } from "@executablemd/core"; +import type { Json } from "@executablemd/durable-streams"; +import { useTempDirectory } from "@executablemd/test-support/temp"; +import { git, gitOutput } from "./git-remotes.ts"; + +/** A filename holding a space, a tab, a newline and a non-ASCII character. */ +export const AWKWARD_PATH = "uni/é space\ttab\nnewline.md"; + +/** A filename holding every character a glob would have expanded. */ +export const MAGIC_PATH = "star*name[1]?.md"; + +/** The 256 bytes a binary file holds at the first commit. */ +export const BINARY = new Uint8Array(Array.from({ length: 256 }, (_, index) => index)); + +/** The same bytes reversed, which is what the second commit holds. */ +export const BINARY_CHANGED = new Uint8Array([...BINARY].reverse()); + +export interface ObservationFixture { + /** The canonical root of the checkout a read is run in. */ + readonly root: string; + /** The home every fixture command runs with. */ + readonly home: string; + /** The first commit's full object id. */ + readonly c1: string; + /** The second commit's full object id. */ + readonly c2: string; + /** The nested repository's two commits, which the gitlink points at. */ + readonly gitlink: { readonly before: string; readonly after: string }; + /** The sibling checkout of the same repository, outside this root. */ + readonly sibling: string; + /** Run one Git command in the checkout, trimmed. */ + run(...args: string[]): string; + /** Run one Git command in the checkout, exactly as it printed. */ + output(...args: string[]): string; +} + +function* write(root: string, path: string, content: string | Uint8Array): Operation { + const target = join(root, path); + const separator = target.lastIndexOf("/"); + if (separator > 0) { + yield* ensureDir(target.slice(0, separator)); + } + yield* until(writeFile(target, content)); +} + +/** + * The fixture, built commit by commit so each status has a real cause. + * + * The gitlink is added with `update-index --cacheinfo` rather than through + * `submodule add`, which would write a `.gitmodules` file and ask Git to clone + * over a protocol this suite does not enable. What a read has to survive is a + * tracked gitlink whose recorded commit moves, and that is exactly what this + * produces. + */ +export function* useObservationFixture(): Operation { + const home = yield* useTempDirectory("xmd-observations-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-observations-"))); + const root = join(parent, "checkout"); + yield* ensureDir(root); + git(["init", "--initial-branch=main", "."], root, home); + git(["config", "user.name", "Fixture"], root, home); + git(["config", "user.email", "fixture@example.invalid"], root, home); + + // The repository the tracked gitlink points into, with two commits of its own. + const nested = join(root, "nested"); + yield* ensureDir(nested); + git(["init", "--initial-branch=main", "."], nested, home); + git(["config", "user.name", "Fixture"], nested, home); + git(["config", "user.email", "fixture@example.invalid"], nested, home); + yield* write(nested, "inner.txt", "inner one\n"); + git(["add", "inner.txt"], nested, home); + git(["commit", "-m", "inner one"], nested, home); + const gitlinkBefore = git(["rev-parse", "HEAD"], nested, home); + + yield* write(root, "src/a.ts", "export const a = 1;\n"); + yield* write(root, "src/bin.dat", BINARY); + yield* write(root, "mode.sh", "#!/bin/sh\necho hi\n"); + yield* write(root, "doc/orig.md", `${"x".repeat(200)}\n`); + yield* write(root, AWKWARD_PATH, "unicode\n"); + yield* write(root, MAGIC_PATH, "literal\n"); + yield* write(root, "gone.ts", "removed later\n"); + yield* write(root, "link-me.txt", "a file for now\n"); + yield* until(chmod(join(root, "mode.sh"), 0o644)); + git(["add", "--", "src", "mode.sh", "doc", "gone.ts", "link-me.txt"], root, home); + git(["add", "--", AWKWARD_PATH, MAGIC_PATH], root, home); + git(["update-index", "--add", "--cacheinfo", `160000,${gitlinkBefore},nested`], root, home); + git(["commit", "-m", "first"], root, home); + const c1 = git(["rev-parse", "HEAD"], root, home); + + yield* write(nested, "inner.txt", "inner two\n"); + git(["add", "inner.txt"], nested, home); + git(["commit", "-m", "inner two"], nested, home); + const gitlinkAfter = git(["rev-parse", "HEAD"], nested, home); + + git(["mv", "doc/orig.md", "doc/renamed.md"], root, home); + yield* write(root, "src/bin.dat", BINARY_CHANGED); + yield* until(chmod(join(root, "mode.sh"), 0o755)); + yield* write(root, "src/added.ts", "export const added = true;\n"); + git(["rm", "--quiet", "--", "gone.ts"], root, home); + git(["rm", "--quiet", "--", "link-me.txt"], root, home); + yield* until(symlink("src/a.ts", join(root, "link-me.txt"))); + git(["add", "--", "src", "mode.sh", "link-me.txt"], root, home); + git(["update-index", "--cacheinfo", `160000,${gitlinkAfter},nested`], root, home); + git(["commit", "-m", "second"], root, home); + const c2 = git(["rev-parse", "HEAD"], root, home); + + // A sibling checkout of the same repository, outside this root, holding + // content of its own so a read that answered from the wrong one would say so. + const sibling = join(parent, "sibling"); + git(["clone", "--quiet", "--", root, sibling], parent, home); + + // An untracked nested checkout, which no listing may traverse into. + const scratch = join(root, "scratch-repo"); + yield* ensureDir(scratch); + git(["init", "--initial-branch=main", "."], scratch, home); + git(["config", "user.name", "Fixture"], scratch, home); + git(["config", "user.email", "fixture@example.invalid"], scratch, home); + yield* write(scratch, "scratch.txt", "scratch\n"); + git(["add", "scratch.txt"], scratch, home); + git(["commit", "-m", "scratch"], scratch, home); + + return { + root, + home, + c1, + c2, + gitlink: { before: gitlinkBefore, after: gitlinkAfter }, + sibling, + run: (...args: string[]) => git(args, root, home), + output: (...args: string[]) => gitOutput(args, root, home), + }; +} + +/** A checkout whose branch has no commit yet, with one path staged. */ +export interface UnbornCheckout { + readonly root: string; + readonly home: string; + run(...args: string[]): string; +} + +export function* useUnbornCheckout(): Operation { + const home = yield* useTempDirectory("xmd-unborn-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-unborn-"))); + const root = join(parent, "checkout"); + yield* ensureDir(root); + git(["init", "--initial-branch=main", "."], root, home); + git(["config", "user.name", "Fixture"], root, home); + git(["config", "user.email", "fixture@example.invalid"], root, home); + yield* write(root, "first.ts", "export const first = true;\n"); + git(["add", "--", "first.ts"], root, home); + return { root, home, run: (...args: string[]) => git(args, root, home) }; +} + +/** A checkout stopped in the middle of a merge, with one unmerged path. */ +export function* useConflictedCheckout(): Operation { + const home = yield* useTempDirectory("xmd-conflict-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-conflict-"))); + const root = join(parent, "checkout"); + yield* ensureDir(root); + git(["init", "--initial-branch=main", "."], root, home); + git(["config", "user.name", "Fixture"], root, home); + git(["config", "user.email", "fixture@example.invalid"], root, home); + yield* write(root, "shared.txt", "base\n"); + git(["add", "--", "shared.txt"], root, home); + git(["commit", "-m", "base"], root, home); + git(["checkout", "--quiet", "-b", "other"], root, home); + yield* write(root, "shared.txt", "other\n"); + git(["commit", "--quiet", "-am", "other"], root, home); + git(["checkout", "--quiet", "main"], root, home); + yield* write(root, "shared.txt", "main\n"); + git(["commit", "--quiet", "-am", "main"], root, home); + // A conflicted merge left in place: the index holds several stages for the + // path, which is the one state a comparison reports as unmerged. + try { + git(["merge", "--no-edit", "other"], root, home); + } catch { + // The conflict is the fixture. + } + return { root, home, run: (...args: string[]) => git(args, root, home) }; +} + +/** Write one file inside a fixture, creating the directories it needs. */ +export function writeFixtureFile( + root: string, + path: string, + content: string | Uint8Array, +): Operation { + return write(root, path, content); +} + +/** A checkout holding `count` generated tracked files, for the paging cases. */ +export interface GeneratedFixture { + readonly root: string; + readonly home: string; + /** Every tracked path, in the order a listing answers them. */ + readonly paths: readonly string[]; + run(...args: string[]): string; + /** Run one Git command in the checkout, exactly as it printed. */ + output(...args: string[]): string; +} + +export function* useGeneratedFixture(count: number): Operation { + const home = yield* useTempDirectory("xmd-generated-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-generated-"))); + const root = join(parent, "checkout"); + yield* ensureDir(root); + git(["init", "--initial-branch=main", "."], root, home); + git(["config", "user.name", "Fixture"], root, home); + git(["config", "user.email", "fixture@example.invalid"], root, home); + const paths: string[] = []; + for (let index = 0; index < count; index += 1) { + const path = `gen/file-${String(index).padStart(5, "0")}.txt`; + paths.push(path); + yield* write(root, path, `generated ${index}\n`); + } + git(["add", "--", "gen"], root, home); + git(["commit", "-m", "generated"], root, home); + return { + root, + home, + paths: Object.freeze([...paths].sort((left, right) => (left < right ? -1 : 1))), + run: (...args: string[]) => git(args, root, home), + output: (...args: string[]) => gitOutput(args, root, home), + }; +} + +/** The musical symbol whose single code point is two UTF-16 code units. */ +export const ASTRAL = "\u{1D11E}"; + +/** + * A blob longer than the old proposed cap, with the astral character placed so + * that a window counted in code units would cut it in half. + * + * 4095 one-unit characters, then the astral one, so the default 4096-code-point + * window ends exactly after it — and a naive `slice(0, 4096)` would end exactly + * inside it. + */ +export const LONG_BLOB = `${"a".repeat(4095)}${ASTRAL}${"b".repeat(61442)}\n`; + +/** Bytes that are not valid UTF-8 and hold no NUL. */ +export const MALFORMED = new Uint8Array([0x61, 0xff, 0xfe, 0x62]); + +/** Bytes holding a NUL, which no text window can carry. */ +export const WITH_NUL = new Uint8Array([0x61, 0x00, 0x62]); + +export interface FileFixture { + readonly root: string; + readonly home: string; + /** The commit holding the long blob, which every window reads from. */ + readonly f1: string; + /** The commit after it, which replaced the long blob. */ + readonly f2: string; + run(...args: string[]): string; +} + +/** A checkout whose first commit holds exactly the blobs a file read needs. */ +export function* useFileFixture(): Operation { + const home = yield* useTempDirectory("xmd-file-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-file-"))); + const root = join(parent, "checkout"); + yield* ensureDir(root); + git(["init", "--initial-branch=main", "."], root, home); + git(["config", "user.name", "Fixture"], root, home); + git(["config", "user.email", "fixture@example.invalid"], root, home); + yield* write(root, "big.txt", LONG_BLOB); + yield* write(root, "empty.txt", ""); + yield* write(root, "bad.bin", MALFORMED); + yield* write(root, "nul.bin", WITH_NUL); + yield* write(root, "dir/inside.txt", "inside\n"); + yield* until(symlink("big.txt", join(root, "points-at-big"))); + git(["add", "--", "."], root, home); + git(["commit", "-m", "first"], root, home); + const f1 = git(["rev-parse", "HEAD"], root, home); + yield* write(root, "big.txt", "replaced\n"); + git(["add", "--", "big.txt"], root, home); + git(["commit", "-m", "second"], root, home); + const f2 = git(["rev-parse", "HEAD"], root, home); + return { root, home, f1, f2, run: (...args: string[]) => git(args, root, home) }; +} + +/** + * Configure this checkout so that every program a read could be made to run is + * one that leaves evidence. + * + * External diff, text conversion, a clean filter, a process filter and a + * file-system monitor. Each is a script that appends its own name to a marker + * file, so "no sentinel executed" is an observation rather than an assumption: + * if a read consulted the checkout's configuration, the marker would say which + * one it reached. + */ +export function* useSentinelConfiguration( + root: string, + home: string, +): Operation<{ readonly marker: string; fired(): Operation }> { + const marker = join(root, "..", "sentinel.log"); + const script = join(root, "..", "sentinel.sh"); + yield* until( + writeFile( + script, + [ + "#!/bin/sh", + `printf '%s\\n' "$SENTINEL_NAME" >> ${JSON.stringify(marker)}`, + "exit 0", + "", + ].join("\n"), + ), + ); + yield* until(chmod(script, 0o755)); + const named = (name: string): string => `env SENTINEL_NAME=${name} ${script}`; + git(["config", "diff.external", named("diff-external")], root, home); + git(["config", "diff.sentinel.textconv", named("textconv")], root, home); + git(["config", "filter.sentinel.clean", named("filter-clean")], root, home); + git(["config", "filter.sentinel.smudge", named("filter-smudge")], root, home); + git(["config", "filter.sentinel.process", named("filter-process")], root, home); + git(["config", "core.fsmonitor", named("fsmonitor")], root, home); + git(["config", "core.hooksPath", join(root, "..", "hooks")], root, home); + git(["config", "credential.helper", named("credential")], root, home); + git(["config", "gpg.program", named("gpg")], root, home); + yield* write(root, ".gitattributes", "*.txt diff=sentinel filter=sentinel\n"); + git(["add", "--", ".gitattributes"], root, home); + git(["commit", "-m", "sentinels"], root, home); + // The fixture's own commands ran with these settings in place, so whatever + // they fired is the baseline. It is removed here, before any read, so what + // the marker holds afterwards is what a read did. + yield* rm(marker, { force: true }); + return { + marker, + *fired(): Operation { + if (!(yield* exists(marker))) { + return []; + } + const content = yield* readTextFile(marker); + return content.split("\n").filter((line) => line !== ""); + }, + }; +} + +/** A linked checkout of the fixture's repository, beside its primary one. */ +export function* useLinkedWorktree(fixture: ObservationFixture): Operation { + const root = join(fixture.root, "..", "linked"); + fixture.run("worktree", "add", "--quiet", "-b", "linked", root); + const resolved = yield* until(realpath(root)); + return { + root: resolved, + home: fixture.home, + run: (...args: string[]) => git(args, resolved, fixture.home), + }; +} + +/** + * Everything one checkout holds, as one digest. + * + * Every file beneath the root, the Git administration directory included, by + * path, mode and bytes. A symbolic link contributes its target rather than + * whatever it points at. Access times are not read at all, so a read that only + * looked at a file cannot move this digest — and a read that wrote one byte + * anywhere cannot help moving it. + */ +export function* fingerprintCheckout(root: string): Operation { + const digest = createHash("sha256"); + for (const path of (yield* collectPaths(root, "")).sort()) { + const absolute = join(root, path); + const info = yield* lstat(absolute); + digest.update(path); + digest.update(String(info.mode)); + if (info.isSymbolicLink()) { + digest.update(yield* until(readlink(absolute))); + continue; + } + if (info.isFile()) { + digest.update(yield* until(readFile(absolute))); + } + } + return digest.digest("hex"); +} + +function* collectPaths(root: string, prefix: string): Operation { + const found: string[] = []; + for (const name of yield* readdir(join(root, prefix))) { + const path = prefix === "" ? name : `${prefix}/${name}`; + const info = yield* lstat(join(root, path)); + found.push(path); + if (info.isDirectory()) { + found.push(...(yield* collectPaths(root, path))); + } + } + return found; +} + +/** The exact bytes of the primary index, a linked index and every shared one. */ +export function* indexFingerprints( + fixture: ObservationFixture, + linked: UnbornCheckout, +): Operation> { + const common = join(fixture.root, ".git"); + const found: Record = {}; + const digest = function* (path: string): Operation { + if (!(yield* exists(path))) { + return; + } + found[path.slice(common.length)] = createHash("sha256") + .update(yield* until(readFile(path))) + .digest("hex"); + }; + yield* digest(join(common, "index")); + for (const name of yield* readdir(common)) { + if (name.startsWith("sharedindex.")) { + yield* digest(join(common, name)); + } + } + for (const name of yield* readdir(join(common, "worktrees"))) { + yield* digest(join(common, "worktrees", name, "index")); + } + return found; +} + +/** A component that performs one host action where a document writes it. */ +export function actionProbe(name: string, run: () => Operation): ComponentRegistration { + return { + name, + origin: `test://${name.toLowerCase()}`, + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* run(); + return ""; + }, + }; +} + +/** A component that records exactly the value a document bound and projected. */ +export function capturingProbe(seen: Json[], name = "Captured"): ComponentRegistration { + return { + name, + origin: `test://${name.toLowerCase()}`, + props: { + type: "object", + properties: { value: {} }, + required: ["value"], + additionalProperties: false, + }, + // deno-lint-ignore require-yield + *fn(props: Record): Operation { + seen.push(props.value ?? null); + return ""; + }, + }; +} diff --git a/packages/git/tests/support/repl-observations.ts b/packages/git/tests/support/repl-observations.ts new file mode 100644 index 000000000..ee02255da --- /dev/null +++ b/packages/git/tests/support/repl-observations.ts @@ -0,0 +1,307 @@ +/** + * One REPL entry, over one journal, reading a real repository. + * + * The same assembly the REPL host installs: core initialization establishes the + * entry's recording inside the scope the invocation owns, the Git Plugin's own + * trusted admission is attached, and the components are the shipped ones. What + * a suite substitutes is only what it has to observe — the Git host, so native + * commands can be counted, barriered and watched, and the filesystem, so the + * document's own writes can be. + */ + +import { + ensure, + race, + resource, + scoped, + sleep, + spawn, + suspend, + until, + withResolvers, +} from "effection"; +import type { Operation } from "effection"; +import { exists, rm, writeTextFile } from "@effectionx/fs"; +import { mkdtemp, realpath, symlink, unlink } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { API, useHostFiles } from "@executablemd/runtime"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, DurableStream } from "@executablemd/durable-streams"; +import { Execution } from "@executablemd/core/api"; +import { executeInstalled } from "@executablemd/core/host"; +import { collect, Elicitation } from "@executablemd/core"; +import { denoRepositoryHost } from "../../src/deno/composition/host.ts"; +import type { + GitBytes, + GitInvocation, + GitOutcome, + RepositoryHost, +} from "../../src/deno/composition/host.ts"; +import { UNAUTHENTICATED } from "../../src/deno/composition/authentication.ts"; +import type { GitAuthenticationSession } from "../../src/deno/composition/authentication.ts"; +import { useReplRepositoryRecording } from "../../src/deno/retained-repository/provider.ts"; +import { admitReplRepositoryHistory } from "../../src/retained-repository/admission.ts"; +import { useCompositionComponents } from "../../src/composition/installation.ts"; +import type { ObservationFixture } from "./observations.ts"; + +/** What the host beneath one entry was asked to do. */ +export interface EntryCounters { + /** Every `git` invocation this entry ran, as its argument list. */ + commands: string[][]; + /** How many scratch directories this entry's reads acquired. */ + directories: number; + /** Every scratch directory, by path, so its removal is observable. */ + scratch: string[]; + /** Every authentication locator a session was opened for. */ + sessions: string[]; +} + +export function entryCounters(): EntryCounters { + return { commands: [], directories: 0, scratch: [], sessions: [] }; +} + +/** The Git subcommands one entry issued, in order. */ +export function entrySubcommands(counted: EntryCounters): string[] { + return counted.commands.map((args) => args.find((arg) => !arg.startsWith("-")) ?? ""); +} + +/** What a row may do to one `git` invocation before the real one runs. */ +export type GitBarrier = (invocation: GitInvocation) => Operation; + +/** + * The production host, counted, and interruptible at a named subcommand. + * + * Every leaf is wrapped rather than replaced: what a row needs to know is + * whether a read ran at all, and the only honest way to answer is to let the + * real `ls-files`, the real `diff` and the real `cat-file` happen and watch. A + * barrier runs *before* the real invocation is delegated to, so a row that + * halts there halts with the native work still to come — and one that delegates + * first halts with a real child in flight. + */ +export function countingEntryHost( + counted: EntryCounters, + barrier?: GitBarrier, + intercept?: (invocation: GitInvocation) => Partial | undefined, +): RepositoryHost { + const host = denoRepositoryHost(); + const watched = (invocation: GitInvocation): GitInvocation => ({ + ...invocation, + ...(intercept?.(invocation) ?? {}), + }); + return { + *git(invocation: GitInvocation): Operation { + counted.commands.push([...invocation.args]); + if (barrier !== undefined) { + yield* barrier(invocation); + } + return yield* host.git(watched(invocation)); + }, + *gitBytes(invocation: GitInvocation): Operation { + counted.commands.push([...invocation.args]); + if (barrier !== undefined) { + yield* barrier(invocation); + } + if (host.gitBytes === undefined) { + throw new Error("the production host lends no byte capture"); + } + return yield* host.gitBytes(watched(invocation)); + }, + *useDirectory(): Operation { + counted.directories += 1; + const directory = yield* host.useDirectory(); + counted.scratch.push(directory); + return directory; + }, + *useAuthentication(locator: string): Operation { + counted.sessions.push(locator); + return host.useAuthentication === undefined + ? UNAUTHENTICATED + : yield* host.useAuthentication(locator); + }, + }; +} + +export interface EntryRun { + readonly output: string; + readonly events: readonly DurableEvent[]; + /** What ended the entry, when something did. */ + readonly refusal: string | undefined; +} + +export interface EntryRunOptions { + /** The directory the entry is opened in. Defaults to the fixture's link. */ + readonly cwd?: string; + /** What this row installs between the host and the recording. */ + readonly between?: () => Operation; + /** A barrier inside one `git` invocation. */ + readonly barrier?: GitBarrier; + /** What this row changes about one invocation before the real host runs it. */ + readonly intercept?: (invocation: GitInvocation) => Partial | undefined; + /** Whether this entry installs a managed root at all. */ + readonly root?: string | undefined; + /** The host this entry runs through, when a row supplies its own. */ + readonly host?: RepositoryHost; +} + +/** Where one scenario's entries are opened, and what they can reach. */ +export interface EntryPlace { + /** The scenario's own directory, which is also the include root. */ + readonly root: string; + /** Where this scenario's managed checkouts live. */ + readonly managed: string; + /** + * A symbolic link an entry is opened through. + * + * Pointed at the fixture's checkout to begin with, and re-pointed by a row + * that needs "nothing resolved this directory" to be a positive fact. + */ + readonly link: string; +} + +export function useEntryPlace(fixture: ObservationFixture): Operation { + return resource(function* (provide) { + const root = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "repl-observe-"))))); + yield* ensure(() => rm(root, { recursive: true, force: true })); + const link = join(root, "entered"); + yield* until(symlink(fixture.root, link)); + yield* provide({ root, managed: join(root, "repositories"), link }); + }); +} + +/** Point the link an entry is opened through at another directory. */ +export function* repoint(place: EntryPlace, target: string): Operation { + yield* until(unlink(place.link)); + yield* until(symlink(target, place.link)); +} + +/** One entry, over one journal, in one checkout. */ +export function runEntry( + place: EntryPlace, + source: string, + stream: DurableStream, + counted: EntryCounters, + options: EntryRunOptions = {}, +): Operation { + return scoped(function* () { + const path = join(place.root, "entry.md"); + yield* writeTextFile(path, source); + const working = options.cwd ?? place.link; + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd() { + return working; + }, + }, + { at: "min" }, + ); + // What a runtime entrypoint installs beside the provider: `API.Files` has + // no host default, and a document that writes `` must reach the + // caller's own filesystem exactly as the REPL leaves it. + yield* useHostFiles(); + yield* useCompositionComponents(); + if (options.between !== undefined) { + yield* options.between(); + } + const managed = "root" in options ? options.root : place.managed; + // Registered the way the REPL host registers it: core initialization is + // what establishes the recording and the handlers, inside the scope the + // invocation owns. + yield* Execution.around({ + *init(_args, next): Operation { + yield* useReplRepositoryRecording({ + host: options.host ?? countingEntryHost(counted, options.barrier, options.intercept), + ...(managed === undefined ? {} : { root: managed }), + }); + yield* next(); + }, + }); + let refusal: string | undefined; + let output = ""; + try { + output = String( + yield* collect( + yield* executeInstalled({ path, stream, includes: [place.root] }, [ + // What the Git Plugin contributes for `repl`, attached directly: + // this suite assembles no command Plugins. + { admissions: [(retained) => admitReplRepositoryHistory(retained)] }, + ]), + ), + ); + } catch (error) { + refusal = error instanceof Error ? error.message : String(error); + } + return { output, events: yield* stream.readAll(), refusal }; + }); +} + +/** + * The prefix an entry left behind when it was left at a question. + * + * A *real* unfinished history, not a complete one with its closes removed: a + * journal whose run finished replays every position it holds and runs no + * component, so a continuation with a new suffix would never reach the suffix. + * What a row needs is the other case — somebody answered nothing, the entry is + * still standing there, and the work after the question has never run. + */ +export function leftEntry( + place: EntryPlace, + source: string, + counted: EntryCounters, + options: EntryRunOptions = {}, +): Operation { + return scoped(function* () { + const stream = new InMemoryStream(); + const reached = withResolvers(); + const task = yield* spawn(() => + runEntry(place, source, stream, counted, { + ...options, + *between(): Operation { + if (options.between !== undefined) { + yield* options.between(); + } + yield* Elicitation.around({ + *elicit(): Operation { + reached.resolve(); + // Nobody answers, so this is where the entry is when it is left. + return yield* suspend(); + }, + }); + }, + }), + ); + yield* race([ + reached.operation, + (function* (): Operation { + yield* sleep(30_000); + throw new Error("the entry never reached its question"); + })(), + ]); + const prefix = [...(yield* stream.readAll())]; + yield* task.halt(); + return prefix; + }); +} + +/** Wait until this path is there, or say what never appeared. */ +export function* appears(path: string): Operation { + const deadline = Date.now() + 30_000; + while (!(yield* exists(path))) { + if (Date.now() > deadline) { + throw new Error(`${path} never appeared`); + } + yield* sleep(25); + } +} + +/** Whether this process is still running, asked of the operating system. */ +export function running(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch { + return false; + } +} diff --git a/packages/git/tests/support/run-composition.ts b/packages/git/tests/support/run-composition.ts index a180aefdc..e1ac9b684 100644 --- a/packages/git/tests/support/run-composition.ts +++ b/packages/git/tests/support/run-composition.ts @@ -29,7 +29,12 @@ import { useCompositionComponents } from "../../src/composition/installation.ts" import { RepositoryContext } from "../../src/composition/context.ts"; import type { RepositorySelection } from "../../src/composition/selection.ts"; import { denoRepositoryHost } from "../../src/deno/composition/host.ts"; -import type { GitInvocation, GitOutcome, RepositoryHost } from "../../src/deno/composition/host.ts"; +import type { + GitBytes, + GitInvocation, + GitOutcome, + RepositoryHost, +} from "../../src/deno/composition/host.ts"; import { UNAUTHENTICATED } from "../../src/deno/composition/authentication.ts"; import type { GitAuthentication, @@ -146,6 +151,13 @@ export function countingOrdinaryHost( counters.commands.push([...invocation.args]); return yield* inner.git(invocation); }, + *gitBytes(invocation: GitInvocation): Operation { + counters.commands.push([...invocation.args]); + if (inner.gitBytes === undefined) { + throw new Error("the counted host wraps one that lends no byte capture"); + } + return yield* inner.gitBytes(invocation); + }, useDirectory: inner.useDirectory, *useAuthentication(locator: string): Operation { counters.sessions.push(locator); diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 739afb3c1..c31df3d7c 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -688,6 +688,18 @@ const DENO_ONLY_REPOSITORY_PROVIDER: RuntimeExclusion[] = [ "a nested execution's own provider instance, which needs the same kernel-released advisory lock the root's does, and real subprocesses to prove where a child stands", issue: DERIVED_SCOPE, }, + { + path: "packages/git/tests/git-observations-deno.test.ts", + reason: + "the subject is a managed selection answering a local Git read: it clones a remote into a managed root and adds a worktree, held under a kernel-released exclusive advisory lock taken through the Deno runtime, so under Node and Bun the document fails on the lock before a read happens. Everything else about which checkout answers is provider-neutral and runs on every runtime in packages/git/tests/git-observations.test.ts", + issue: DERIVED_SCOPE, + }, + { + path: "packages/git/tests/retained-repository-observations-deno.test.ts", + reason: + "the subject is the native side of a REPL entry's repository host: cancelling a finished read's append unwinds a Deno-backed Files provider, a held read excludes the same entry's mutation only once that mutation reaches a real `git` child, and another holder's mutation lease is the same kernel-released advisory lock. The record format, its admission, every refusal and the rest of cancellation and persistence run on every runtime in packages/git/tests/retained-repository-observations.test.ts", + issue: DERIVED_SCOPE, + }, { path: "packages/git/tests/retained-repository-local.test.ts", reason: diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index e4be12a3f..d026a7679 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -3053,8 +3053,9 @@ completes its teardown and commits no record. #### The run profile's repository declarations -Twelve names — `Repository`, `Worktree`, `Git.Switch`, `Git.Add`, `Git.Commit`, -`Git.Push`, `PullRequest`, `PullRequest.Reviews`, `PullRequest.Comments`, +Sixteen names — `Repository`, `Worktree`, `Git.Switch`, `Git.Add`, +`Git.Commit`, `Git.Push`, `Git.Files`, `Git.Changes`, `Git.Diff`, `Git.File`, +`PullRequest`, `PullRequest.Reviews`, `PullRequest.Comments`, `PullRequest.Checks`, `IssueTracker` and `Issue` — are ordinary [registered defaults](#registration-and-resolution-order). A repository-local Markdown or TypeScript component of any of those names is chosen ahead of them, @@ -10259,7 +10260,7 @@ calling a selector, cloning, adding a worktree, reading a sidecar, opening a credential or taking a lock, and only genuinely new work meets the current filesystem. -It answers three local Git operations on the same terms — ``, +It answers three local Git transitions on the same terms — ``, `` and `` — by calling the *ordinary* performers and recording what they produced. A completed one is answered from its record with no `git` invocation of any kind, and the object id a completed `` @@ -10274,6 +10275,27 @@ one. A live operation takes the selected placement's managed slot and then the hold on the repository it changes, keeps both until the entry is torn down, and takes neither to answer a completed one. +It answers the four local reads on the same terms — ``, +``, `` and `` — through the same contextual +read Api an ordinary run answers, at the same implementation slot, and records +the page each produced. A completed read is answered from its record with no +`git` invocation of any kind, and the page names the repository as it was then. +What the record carries is the authenticated Repository identity, the checkout +the selection named and its common Git directory, the received working directory +as it arrived, the normalized query, the checkout the route reached, and the +exact page including its complete `next` — never the ephemeral selection +identifier, and never anything a Git command printed. Trusted admission holds +that route to the entry's own history before either a completed or an unfinished +page is reused: the selection and the checkout have to be ones a preceding +placement in that same history made, decided from parsed historical facts +alone. A record naming a checkout nothing placed ends the attempt, rather than +answering from a route this entry never took. A read takes the selected +placement's managed slot and no mutation lease, because it changes nothing: an +entry mutating a checkout and an entry reading it do not wait for each other. +A continuation check belongs to a live read rather than to replay — a saved +`next` used in a new read refuses when the answer has changed, and reopening the +saved page itself asks the repository nothing. + It answers one publication beside them. `` publishes the branch the selected checkout is on to that Repository's origin through the shared Git-host reconciliation, and what the entry adds beneath it is the request it asked for. diff --git a/specs/repl-spec.md b/specs/repl-spec.md index 5d8de5d63..7cf4f18f3 100644 --- a/specs/repl-spec.md +++ b/specs/repl-spec.md @@ -1434,6 +1434,72 @@ completed Git operation takes neither. A no-op Add, a same-branch Switch and a Commit that will refuse take them too: whether `git` will change anything is not predicted. +### Reading the repository an entry is standing in + +Inside the same checkout, type this: + +```xmd + + + + + +``` + +Four reads and nothing changed. `` lists the tracked paths the index +holds, relative to the repository root, excluding untracked files and the +contents of any checkout nested beside them. `` reports each +changed path with Git's own letter for what happened to it, for the index +against the working tree, HEAD against the index, or two commits you name +directly. `` answers the same comparison as patch text. `` +answers the text one commit holds at one path — the commit's bytes, not the +working copy, so it still answers after you have edited the file. + +A page is bounded: 100 records, or 4096 characters of text, unless you ask for +more. Ask for more and you get more; nothing caps it beyond what a number can +represent. When something remains, `next` holds the complete arguments of the +read that continues it, so a later entry follows it by passing the value back: + +```xmd + +``` + +Follow one after the answer has changed and the read refuses rather than +stitching two repositories together: *The Git observation changed. Restart +`` without cursor.* A change elsewhere that does not alter this +answer needs no restart, and raising `limit` between pages is safe — a limit is +how much of an answer to return, not which answer it is. `` carries +its resolved commit in `next`, so moving the branch under it changes nothing +about the file you are reading. + +Leave an entry at a question after these reads, change every file behind it, and +come back. The saved pages answer again: the same paths, the same statuses, the +same patch window, the same commit — and not a single `git` command. The entry +after it inherits those bound values and can render members of them without +reading anything. Reads after the question run for the first time, against the +repository as it is now. Delete the checkout outright and the saved pages still +answer; reading one consults nothing. + +A refusal you asked for is retained as a refusal, exactly as a transition's is. +A revision that names no commit, a path the commit does not hold, a directory or +a nested checkout's recorded commit where a file was asked for, and a file that +is not text each say which it was and what to write instead. A directory is a +directory under either spelling: `src` and `src/` both refuse, and neither +answers with something found inside it. None of them quotes +anything Git printed. + +These reads change nothing, so they take no hold on the repository you are +changing: another entry committing in the same checkout does not wait for a +read, and a read does not wait for it. Inside *one* entry it is the other way +round, and deliberately: one read runs several Git commands against one private +index copy, so that entry's own ``, `` or `` +waits for the read to finish and be written down before it runs. Nothing moves +under a sequence already in progress. The first new read inside a restored +managed placement takes that placement's ownership and nothing else; a completed +read takes neither. Hooks, signing, filesystem monitors, external diff programs, +text conversion, content filters and repository credential helpers stay +disabled, and no read reaches a remote — a missing object refuses instead. + ### The branch an entry published Inside a lexical `` — or a `` of one, including a @@ -2169,11 +2235,23 @@ resolved against the frame that published its coordinates. neither a retained local result nor a retained publication authorizes one. What an entry retains is `` and `` selection, the file and directory work inside those checkouts and the one it was started in, - ``, `` and `` in them, the branch - `` published, and the issues it filed and read. A runtime that - installs no repository provider — Node and Bun — refuses the Git components, - publication included, before any native operation or remote contact, while - ordinary `` and `` work keeps working there. + ``, `` and `` in them, the page each of + ``, ``, `` and `` answered with, + the branch `` published, and the issues it filed and read. A + runtime that installs no repository provider — Node and Bun — refuses the Git + components, the four reads and publication included, before any native + operation or remote contact, while ordinary `` and `` work keeps + working there. +- A read confers nothing. Nothing a page holds authorizes a mutation, a + publication, a pull request or an Agent, and the opaque `cursor` a `next` + carries is continuation data rather than permission: a follow-up read + authenticates and routes on its own terms and then checks that the cursor + describes the checkout it reached. +- A read is not a snapshot. One observation may run several Git commands against + one private index copy and one pair of resolved commit ids, and that is as much + as it promises: nothing locks the repository, watches it or compares every + write that happened while it ran. What continuation checks is whether the + *answer* changed, not whether the filesystem did. - `` is an ordinary shadowable default here, as it is under every other host. A repository component of that name keeps its ordinary precedence over it, and what such a component does not get is the identity an entry's own diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index 2e3cb6580..9290fdd9f 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -179,6 +179,19 @@ no transaction, no frontier and no logical filesystem — what it has is a historical record of each completed operation, which is neither of the other two. +``, ``, `` and `` are available to +a REPL entry and are retained on the same terms, with one difference that +follows from what they are: a read changes nothing, so it takes the selected +placement's ownership of a managed checkout where it has one and no hold on the +repository at all. An entry committing in a checkout and an entry reading it do +not wait for each other. Within one entry they do: every operation of that entry +that runs `git` against one shared Git directory — the four reads, the three +transitions and the publication's own observation — takes the same turn, and a +read keeps it until its children have joined and its page is acknowledged. A completed read is answered from its record with no +`git` invocation, and the page it answered with describes the repository as it +was then — a saved `next` used in a *new* read is the one place a changed answer +is noticed, and it refuses there rather than combining two observations. + `` is available to a REPL entry, and what it retains is `specs/repl-spec.md`'s subject rather than this document's. The reconciliation is the same one §7.4 and the Git-host effect contract describe — observe, adopt, @@ -1221,7 +1234,8 @@ contextual way `` writes one. ## 7. Git operations -The four Git components are likewise one language behind several providers. +The four Git transitions and the four Git reads are likewise one language behind +several providers. Which checkout one acts on is decided the same way everywhere: by the Repository in scope and by the contextual working directory, neither of which admits anything. @@ -1880,6 +1894,93 @@ not name its own number and repository are all refused. A well-formed answer to another question is still the wrong answer. +### 7.8 Local repository reads + +Four read-only components answer questions about the checkout a Repository +selected: + +```md + + + + +``` + +Each is self-closing, binds JSON through a required ordinary `as`, and renders +nothing. Which checkout answers is decided exactly as it is for the four +transitions — the Repository in scope and the contextual working directory, +neither of which admits anything — so a `` inside a selected `` +reads that worktree, and a read written outside every Repository has nothing to +read. + +`` lists the tracked paths the selected checkout's index holds, +sorted, deduplicated and relative to its repository root. An untracked file is +not in it, the contents of a nested or sibling checkout are not in it, and a +tracked gitlink is one path rather than a traversal. The optional `prefix` is +literal string-prefix matching: it may stop inside a final segment, and `*`, `?` +and `[` match those characters. + +`` and `` take the same three comparisons. `unstaged` is +the index against the working tree, excluding untracked files; `staged` is the +HEAD tree against the index, and the empty tree on an unborn branch; `commits` +compares the trees of the required `base` and `head` commits directly, never +their merge base. Each change is a path, Git's own letter for what happened to +it — `A`, `M`, `D`, `R`, `T`, `U` — and the path a rename came from, or `null`. +`` answers the same comparison as unified patch text, with Git's +binary marker, mode metadata and gitlink pointer changes intact: it is a text +window rather than a structured hunk graph. The optional `paths` filter names +literal repository-relative files and directory prefixes, matches a rename on +either of its paths, and selects nothing at all when it is empty — which is +different from omitting it. + +`` resolves `commit` once and reads the blob that commit's tree holds +at `path`. The result names the full resolved commit id, so it answers the same +bytes after the working copy has changed and after the branch has moved. A +symbolic link is its own stored link text and is never followed. + +A page is bounded by the caller's own limit: 100 records or 4096 code points by +default, and whatever larger positive safe integer a caller asks for otherwise. +No arbitrary maximum is imposed; the default bounds what a document binds and a +journal retains rather than what Git had to read. `offset`, `limit`, `length` +and a next offset count Unicode code points, so no window splits a character, +and nothing is trimmed. When records or characters remain, `next` holds the +complete arguments of the read that continues it, and `null` says the +observation is complete. + +Continuation identifies one complete filtered observation and grants nothing. +The opaque `cursor` a list or text `next` carries is data: a follow-up read +authenticates its Repository and routes to a checkout on its own terms, then +checks that the cursor describes that checkout, that same query and that same +answer. An answer that has changed refuses with restart guidance rather than +combining two observations; a change that does not alter this answer needs no +restart; and raising `limit` between pages is safe. `` needs no cursor, +because its input is already immutable. + +Availability is the installed provider's. The shipped Deno and compiled +`xmd run` and `xmd repl` providers answer; a host that installs no +selected-repository provider refuses, and nothing falls back to the caller's own +filesystem. A logical workflow filesystem is not a native checkout: there is no +provider for one, so the reads refuse there as they do under Node and Bun. The +vocabulary itself is never hidden or banned by profile. + +A read changes nothing in the repository it reads, and nothing it returns +authorizes anything else: no mutation, no publication, no pull request, no +Agent, no Push evidence and no broader command capability. The existing +`Permissions` surfaces mediate what they already mediate and do not reach these +reads by implication. + +What a read may not do, it may not be made to do by the repository's own +configuration. A read runs against isolated Git administration and a contained, +read-only view of the authenticated object database, so the selected checkout's +`.git/config` supplies no external diff program, no text conversion, no content +filter, no file-system monitor, no hook, no credential helper and no transport. +Index-based reads use a private copy of the checkout's own index — discovered +through Git's path query, because a linked worktree has one of its own, and +copied with its own modification time, because that time is what Git's +racily-clean rule compares against. Nothing refreshes, expands or repairs the +source index, and lazy fetching is disabled even while a revision is resolved: +a missing object refuses rather than reaching a remote. + ## 8. Agents inspect; XMD mutates ### 8.1 No directory registration