development.md

Development

The gate, the formatters, the hooks, the generated files, and where a slow run's time actually goes. make targets and the one-line summary of each are in README.md; everything here is the detail behind them.

Selection admission and refresh behavior are specified in interview selection. The Strict browser regression tests use the existing real media preflight with a fake media grant and room transport; they do not exercise paid LiveKit, Gemini or Egress services.

Account database writes

Each server keeps one SQLite connection behind a mutex and runs account queries on the blocking pool. The mutex serializes that server's queries; it does not protect against another server using the same database.

Read-then-write account operations use Accounts::transaction, which reserves SQLite's writer with BEGIN IMMEDIATE before reading. Report saves enforce the MAX_REPORTS_PER_USER quota and ownership check in the same transaction as the upsert. Interview creation reuses pending consent before applying the MAX_INTERVIEWS_PER_USER quota, with no interleaving claim or insert from another connection. Session creation, expired-account sweeps, delivery claims and retry scheduling, and recording tombstones use the same commit-or-rollback helper. Single-statement room claims, releases and deletes remain atomic without it.

Replay ingestion keeps its ownership precheck outside the writer transaction; replay snapshots use a deferred read transaction so they do not reserve the writer.

A competing writer waits up to the connection's five-second busy timeout, then returns a database error. These transactions contain only local database work; network calls must stay outside them. No schema migration is required. Regression checks live in tests/unit/accounts/schema.migration_tests.rs.

Dependency security checks

A new RustSec advisory must reach maintainers without making an unrelated feature look broken. The independent audit job uses scripts/audit.py:

• Pull requests compare the tested merge commit against its first parent, the event's base tip. Merge groups compare against merge_group.base_sha. Never use the old branch's merge-base: that can hide a reintroduced weakness.
• Both lockfiles use one freshly fetched advisory database snapshot. Findings are compared by advisory ID, crate and version, so replacing one vulnerable dependency with another cannot pass by keeping the same count.
• An unchanged lockfile gets one informational scan. Inherited vulnerabilities and scan errors appear in the job summary without failing the PR. With a changed lockfile, new findings or an incomplete scan fail the audit. Removing a finding passes; upgrading to another affected version remains blocking.
• Pushes to main run a full audit. The rolling release waits for it (the builds run beside it), so a known vulnerability or scan error blocks publication. This can delay unrelated fixes while the previous release remains vulnerable; prioritize the dependency repair rather than bypassing the audit.

Checkout and tool-installation failures still fail the job. The informational exception covers scanner and database errors after the tool is installed.

The separate security audit workflow runs daily and on manual dispatch from main. It maintains one bot-authored issue titled Dependency security audit, updating its findings or closing it after a clean scan, and reopening the same issue when findings return so its subscribers keep following it. Changes to the findings also leave a comment to notify subscribers, once even when a retry follows a failed update; a database refresh alone does not. An incomplete scan fails without closing the issue. Only this trusted-main job has issues: write; fork PRs use read permissions and job summaries, without comments or secrets. Maintainers should subscribe to the tracking issue and workflow failures, assign an owner and a repair deadline, and link the repair PR in the issue. A recurring scan is not a repair deadline.

Enable Dependabot alerts and security updates in repository settings; the existing .github/dependabot.yml groups coupled security updates. RustSec findings may precede GitHub's advisory import, so the daily audit remains necessary. Review security PRs promptly and link them to the audit issue. Do not add blanket advisory ignores. Any proposed exception needs a maintainer, reason, expiry and tracking issue; no exception is implemented by this change.

./scripts/test.sh retains the full local cargo audit when installed. Its three CI lanes omit that gate because the separate audit job owns CI policy. To reproduce CI locally, run python3 scripts/audit.py for a full scan or python3 scripts/audit.py --base <base-tip-sha> on the tested merge tree. The scanner tests run with python3 scripts/run-python-tests.py tests/test_audit.py.

What the gate checks, and what it will not

./scripts/test.sh finishes by printing what it did not check, because several lanes are optional and a run that skips them still exits 0. A checkout with none of the tools below is a green gate over roughly a tenth of the browser suite and no static analysis of the shell, the workflows, the Python, or the dependency tree. The summary names each one; these are what they need.

npm ci                            # ESLint and Prettier, pinned
npx playwright install chromium   # the browser tests, the whole lobby flow
cargo install cargo-audit         # the dependency advisory scan
cargo install cargo-deny          # dependency licenses and sources, deny.toml
sudo apt install libwebkit2gtk-4.1-dev   # Linux: clippy and tests of desktop/

CI runs scripts/test.sh rust, scripts/test.sh portable and scripts/test.sh browser in parallel, with fail-fast disabled so a failure in one lane does not hide the others. The browser lane runs as three jobs, each with a BROWSER_SHARD slice (1/3, 2/3, 3/3) that splits the suites by source size; set it locally to run one slice. Clippy and Rust tests share the Rust lane's build cache; the desktop app's clippy and tests run in the same lane against their own lockfile and cache. The Rust lane installs both crates' system packages in one apt transaction. Formatting, Python and shell checks run in the portable lane; Node browser tests have their own lane, except the face-detector suite, which needs the Rust binary and runs after its build. All lanes fetch and verify vendor assets; each restores the vendored-asset cache, and only the portable lane saves it, after its gates pass. Each lane has its own job timeout: 45 minutes for Rust, 20 for the others. The Rust and browser lanes clone only HEAD; the portable lane keeps full history for the commit-message check. The Rust lane deletes the runner's unused toolchains only when less than 40 GB is free. Running ./scripts/test.sh locally still runs every gate.

On a busy local host, RUST_TEST_THREADS=4 ./scripts/test.sh rust limits concurrent server launches in the Rust suites. The browser compiler fixtures also bound their concurrent jobs so candidate programs get CPU before their execution deadlines. Editor-history fixtures set selections directly and use the browser's platform modifier; native dragging includes a press-and-hold before moving selected text.

check is the only status a branch rule for main should require. It needs every lane and the independent security audit to succeed, and on a pull request the mutation plan and every shard too; shards may be skipped only when the plan found no mutants. A push to main or a merge queue entry plans no mutants, so check reports as soon as the lanes finish. The workflow also runs on merge_group, so once main has a merge queue, it tests each pull request on top of the main it will land on; two pull requests that pass against their own bases can still break main together. A merge group saves no caches, since its queue ref is gone once it lands, and the cargo build caches are saved from main only: a pull request's copy is readable by that pull request alone, and the repository's 10 GB limit was full of them. Mutation planning checks the Rust diff before installing its tools, so a PR with no Rust changes starts no mutation shards and pays no mutation-tool setup cost. Rust diffs still use the complete diff-selected mutant set and the existing exclusions.

Every skip the gate sees joins its not checked by this run: summary: a gate's own skip, a Node browser test's skip or todo, a missing formatter in scripts/indent.sh --skip-missing, and a skipped Python case. A Rust test that skips itself when sqlite3 or Playwright is missing is not in it, because cargo test hides a passing test's output. With CI=true the summary fails the run, and those Rust tests fail rather than skip. Optional tools still skip on a local run. Browser files run two at a time with a ten-minute timeout per file (Node also applies that limit to individual tests). The editor history suite bounds each interaction at thirty seconds separately, so adding passing cases cannot consume a single interaction's budget. Local Node versions without the timeout or concurrency option still run the browser tests. npm test uses the same browser lane as CI, including vendor verification and skip reporting; the server-tier face detector suite stays with the Rust lane, which builds the server it needs. Browser tests that wait out a timer jump the fake clock with page.clock.fastForward rather than runFor, which also steps every frame and interval in between and once made a single test take eighteen minutes.

The weekly avatar check is the avatar job in the same workflow, started only by the schedule or a manual dispatch, and neither starts anything else. It restores the Cargo, vendored-asset and Chromium caches through the same composite actions under .github/actions/ as the lanes, and saves none of them, so the weekly job keeps no second copy. Chromium's system dependencies are still installed on every fresh runner.

The gate picks its own compiler and Python, so running it directly builds with the same Clang make would. On Linux x86_64 with no CXX set, it uses the checksum-pinned Clang that make fetches into target/clang, fetching it just before the first lane that compiles. It runs the Python lanes with the first Python 3.11 or newer it finds, because the Rust style checker uses the standard-library TOML parser; set PYTHON to name one. An explicitly named older interpreter is refused before any gate runs, except for the browser lane, which runs no Python gate. The Rust lane requires Python for its workspace inheritance audit but never installs the instructor tool's packaging environment. Recreate an older target/task-tools environment with Python 3.11+ before using it for the gate; existing environments are never cleared automatically. ruff.toml pins the rule set the gate was written against, so a newer ruff with wider defaults does not fail an unchanged tree.

Lint levels live in the workspace tables in Cargo.toml rather than only in the gate's command line. The server and the Rust hook checker both inherit them through [lints] workspace = true. The Rust lane's workspace-lints gate obtains actual membership from offline Cargo metadata and refuses a member that opts out, including implicit path dependencies. The portable source checker needs no Cargo metadata. The Rust lane also builds public documentation with cargo doc --no-deps; shared rustdoc lints reject broken links and links from public documentation to private items. [workspace.lints.clippy] all = "deny" is why an editor running a bare cargo clippy agrees with make check instead of reporting warnings for what the gate calls an error. Ignored must-use results, stale lint expectations, discarded futures, locks held across await and mem::forget are explicit denials too. [workspace.lints.rust] unsafe_code = "forbid" is what keeps this crate's unsafe count at zero — forbid rather than deny, so a local #[allow] cannot reintroduce it. scripts/check-rust-style.py holds what a lint cannot ask: that this table stays strict, that no allow or bare #[ignore] appears, that no marker or CJK text lands in Rust or web/, and that no file passes the line ceiling in scripts/rust-style-ceilings.json, which --ratchet only lowers. The rules and the lints left off on purpose are in the codetrial-rust skill.

The compiler those lints run on is pinned in rust-toolchain.toml, so a new Rust release cannot fail an unchanged tree: Dependabot proposes each bump as its own pull request. The pin is the gate's compiler; rust-version remains the oldest release the crate builds on, and the Linux release leg builds on it.

shellcheck and ruff come from the system package manager. actionlint is not packaged as widely, so the gate falls back to its container image, pinned to the version the workflow installs, whenever the binary is absent and a docker daemon answers. CI installs all three, so what is optional locally is enforced on a pull request — the summary exists so that a contributor knows which of the two they are looking at. One lane needs working javac and java; the Java class-harness fixture uses Java 8 syntax, and nothing else depends on it.

Where the time goes, measured on this repo rather than guessed, because the answer is not the one a first look gives. Warm, the two Rust lanes are seconds: cargo clippy --all-targets takes about five and building and linking the eleven test binaries about nine, and the whole browser suite runs in under twenty. What costs minutes is any change that invalidates the dependency graph, a lockfile bump, a toolchain bump, a profile edit or a fresh checkout, because rebuilding it means building the LiveKit SDK and libwebrtc from source. That is why CI caches target keyed on the lockfile and links with mold, and it is why a local run that suddenly takes twenty minutes has usually just had its cache invalidated rather than got slower.

Trimming debug info, which is what buys CI 40% per relink, was measured here and buys nothing: 7.5s against 8.1s on macOS, where the system linker rather than mold does the work. It stays a CI setting for that reason.

Formatting

scripts/indent.sh holds the formatter chain: comment reflow with commentflow, then rustfmt, ruff format and shfmt, in that order because the formatters have to have the last word over the reflow. make indent runs it with --write, make indent-check and the gate with --check, against a copy of the tree so a check never rewrites what it is judging, and the pre-commit hook with --staged, against a checkout of the index. The file lists, the vendor exclusion and the order live only there. shfmt takes its style from .editorconfig and is passed no style flags anywhere.

A formatter that the selected files need is required: make indent, make indent-check and the hook fail on a missing one and say how to install it, so a commit that touches only Rust does not need Prettier, but one that touches JavaScript does. Only the gate passes --skip-missing, which skips the lane, ends with Formatting clean for the lanes that ran; not checked: and the tools it skipped instead of Formatting clean., and joins the gate's summary, which CI fails on. CI reports rather than fails formatting drift on a pull request Dependabot opened and wrote, because it can bump Prettier but cannot run make indent. Nothing fails it after that until main has a merge queue, so whoever merges such a pull request runs make indent on it first, or the push to main and every later pull request fail on the drift.

npm ci installs the pinned Prettier version used for HTML, JavaScript (.js, .mjs and .cjs) and the issue forms under .github/ISSUE_TEMPLATE/. The forms ride along because Prettier is the only lane that parses their YAML, for the reason scripts/indent.sh gives. It checks syntax and not the form schema, so a mistyped key or an unknown element type formats clean and still breaks the issue chooser: only GitHub, or the published schema at json.schemastore.org, judges that.

Prettier's style lives in .prettierrc.json; .prettierignore excludes vendored and generated files. Prettier shares no file with the rest of the chain, so the check runs it in place rather than on the copy, alongside the chain, with --cache so a rerun skips unchanged files. The generated problem-card section in web/index.html has a prettier-ignore marker so the rest of the page can be formatted without changing the generator's output. As with the other formatter lanes, an absent Prettier is reported and skipped in the gate; CI installs it through npm ci.

The commits that only reformatted are listed in .git-blame-ignore-revs, so git blame skips them. GitHub reads the file on its own; a local blame needs git config blame.ignoreRevsFile .git-blame-ignore-revs once per clone.

Comments that count things

Comments here carry the reasoning, deliberately, and that is not the part worth economising on. What has gone wrong repeatedly is narrower: a comment that states a count of code, "all three routes", "written out twice", "five caveats". Nothing checks those, the code moves, and the comment is then a confident false statement sitting next to what it describes. One recent branch spent twelve of its eighteen commits correcting claims an earlier commit's prose had made.

Two shapes are safe and are the ones to reach for. A count in the past tense is history and cannot go stale: "seven modules were each restating this" stays true after an eighth arrives. A count a test already pins is checked by that test, so ReplayKind::ALL being [Self; 6] makes "six kinds" safe to write.

A present-tense count of anything else is a liability. Either drop the number, naming the things instead, or move it somewhere a run can fail on it.

.cargo/mutants.toml is the second shape. Its header declares how many functions the mutation gate has been told not to judge, and scripts/test.sh compares that against the list, so an entry added without moving the number fails the gate. The number is not the point: each exclusion is defensible on its own and the list only ever grows, and a gate that judges steadily less is the thing worth putting in front of a reviewer. Removing an entry, by making the function reachable from cargo test, is the direction it should travel.

Git hooks

scripts/test-git-hooks.sh drives all four hooks against a scratch repository and runs in the gate, so a hook that stops rejecting fails here rather than on somebody's next commit.

make hooks installs a wrapper in .git/hooks for each scripts/git-*.sh that resolves the active worktree's copy, then verifies each is the current wrapper and executable, and exits nonzero otherwise. A wrapper names itself on its second line, so one an older install wrote is reported STALE and rewritten. A hook it did not write is left in place and reported as FOREIGN, and the install exits nonzero. Once you move it to <name>.local and run make hooks again, it runs after ours with the same arguments and stdin, and make uninstall-hooks moves it back. A core.hooksPath outside the repository is reported as SHARED and nothing is written there. A checkout from before the scripts existed has nothing to run and commits as before. ./scripts/install-git-hooks.sh --check runs the verification alone.

Changes to the hook scripts, installer, message checker or hook suite run scripts/test-git-hooks.sh during pre-commit. Changes under .claude/skills/, scripts/, docs/, src/, web/, tests/, problem-bank/, .github/ or to root files run the skill reference checker when the verification skill exists. Both checks read the staged snapshot, including deletions. The hook suite suppresses recursive suite invocation in its nested test commits. Checks run before formatting and never stage their output.

When a push goes to origin, the pre-push hook first fetches the rolling release tags latest and desktop-latest using forced exact refspecs. Discovery and fetch share a one-minute deadline, with TERM at 55 seconds and a forced kill five seconds later, and never prompt. Synchronization needs timeout or gtimeout; without either tool (stock macOS has neither) the hook says so and skips it. An unreachable origin refuses that push; a push to another remote, or one that itself updates those tags, does not fetch, and an origin without the tags (a fork) is not an error. A repository without origin skips this. The installer also adds the glob refspec +refs/tags/*latest:refs/tags/*latest to the clone's remote.origin.fetch (git fails a fetch whose exact refspec names a tag the remote lacks), so a plain git pull or git fetch tracking origin refreshes the tags even when the branch is already up to date. A missing tag is not an error there; the glob also takes any other tag ending in latest from origin. Explicit branch refspecs, as in git pull origin main, do not request those extra tags. Pulling another remote does not fetch origin. Git has no pre-pull hook. Installing replaces the exact refspecs an earlier installer wrote. Uninstalling removes the refspec; other fetch settings stay in place.

Offline credential checks and message guidance

scripts/check-secrets.py checks the changed index blobs before pre-commit materializes the snapshot. It also checks the outgoing commit range in pre-push, including merge resolutions and an earlier commit whose secret was removed at the tip. Commits the destination remote already holds are not rescanned, and neither is history another remote already holds. Pull-request CI repeats the history check. Git inspection errors refuse the operation rather than reporting a clean scan.

The checker uses Python's standard library. It detects private-key headers, Google API keys and GitHub token formats, and literal credential values from .env, .env.local and config/codetrial.env.local, for variables whose names contain KEY, TOKEN, PASSWORD or SECRET. Those local files are read as data, never sourced or interpolated; symbolic-link sources are refused. Values shorter than eight bytes, obvious placeholders and whole-value variable or command references are excluded; a password that merely contains $ is kept. Paths under tests/, which carry throwaway private keys, are exempt from the private-key rule only. Changed live .env paths are refused; .example, .sample and .template paths are allowed but their contents are still scanned. Binary blobs are not exempt, and submodule commits are not read as files. Findings name the path, rule and commit, with matching values redacted even from filenames. No snippet, credential value or provider request is logged or transmitted. This is a bounded set of checks, not a claim to detect every secret format or encoding. The regression suite is tests/test_check_secrets.py.

The prepare-commit-msg hook preserves supplied messages and adds its guidance only once. It shows at most forty staged filenames and a short change summary above the scissors line, without generating a subject or sending a diff to an AI service. The existing message rules remain authoritative.

Build-time installation

Every cargo build, cargo check, cargo run and rust-analyzer pass in a clone or worktree runs the install through scripts/hook-check, a workspace member whose build script reruns on every build, so a deleted or non-executable hook and a core.hooksPath set in a global config are all caught on the next one. Anything the install cannot settle, a FOREIGN hook, a SHARED hooks directory or no sh to run it with, fails the build with the installer's own instructions. CODETRIAL_SKIP_HOOKS=1 skips the check: CI sets it, since it replays every commit message anyway, and so can a checkout that cannot take hooks. It is also what keeps make uninstall-hooks in effect, since the next build otherwise puts the hooks back. cargo test leaves the check out, since it would link an empty test binary for it each time, and so does a command that names its targets: -p codetrial, --bin, --test or cargo install --path ..

The hooks run the scripts/ of the checkout they fire in, as scripts/hook-check does on a build, so review a branch before committing on it as you would before building it. No script of ours runs while a merge is concluded: by the time its commit-msg or prepare-commit-msg fires the incoming tree is written, and running it would make git pull execute what it pulled, so the wrapper skips it, and there is no pre-merge-commit hook. Post-commit also checks the resulting commit's parent count, since Git has already removed MERGE_HEAD at that point. A user-owned .local hook still runs. CI checks a pull request's content.

The pre-commit hook runs ESLint, ruff check and shellcheck, then scripts/indent.sh --staged, over a checkout of the index, so an unstaged edit neither fails a commit nor passes one, and it never stages anything itself. The formatting check is what make indent would change in the staged files, and in every file when the commit changes a formatter setting (.editorconfig, rustfmt.toml, ruff.toml, the Prettier config or the version pinned in package.json, or scripts/indent.sh). The edition comes from Cargo.toml for both, and rustfmt.toml, empty of settings, keeps a ~/.rustfmt.toml out. Paths are read with core.quotePath off, so a name past ASCII is checked under its own name; one git still quotes, or one with a blank, is refused by name rather than skipped. A file added with --intent-to-add carries no content into the commit and is not judged. A missing formatter refuses the commit; a missing linter is skipped and named in the closing line. The commit-msg hook holds the subject to 50 characters and the body to 72, imperative and ASCII, with a link or file name longer than a line, or a trailer, allowed past the limit. It refuses a subject that only names a file, an AI attribution and a Refs #N on the last line, and says why (see CONTRIBUTING.md). The rules run in scripts/git-commit-msg.awk, one process per message, which is also what scripts/check-commit-log.sh and the PR sweep's hygiene check ask, with --stored for a message a commit already holds. The pre-push hook replays the same message rules over commits a rebase or an amend rewrote after the fact, and CI runs them over a pull request's own commits. On a topic branch small changes get one advisory line when the head has no /codetrial-review record. A topic branch with any commit at least 200 added plus deleted text lines, or a net PR diff over 500, is refused without matching durable code-review and interactive-explanation evidence. The hook uses raw text counts conservatively, without automatically exempting generated or vendored paths; record any exclusion evidence during the review. It resolves the PR base from the canonical upstream's local main tracking ref, falling back to the push remote's main when no canonical upstream is configured. Missing history is refused when there is no remote-tracking main to measure against (for example a push to a bare URL from a clone that never fetched it): skipping the check would let any large change through, so fetch the base and push again. No browser or model starts in a hook. Deleting the local skill file does not disable a review policy present at the pushed tip or the upstream PR base.

scripts/hashchain-base holds the configurable chain anchor. Python's standard library implements scripts/hashchain.py: pre-commit records the checked index, commit-msg generates the tree-linked Change-Id, and post-commit repairs amend inference only for a matching checked candidate. A skipped commit without that claim is not repaired automatically. Pre-push checks every pushed commit tip, including tags and main, independently of the checked-out branch. PR CI checks the actual PR head rather than GitHub's synthetic merge; push CI checks the pushed head. Merge-queue CI also verifies the actual group head against its event base policy.

A commit authored by dependabot[bot] continues the chain without a Change-Id trailer, because the bot cannot run the hooks; the author line is unauthenticated, so locally this is a convenience and not a trust boundary. scripts/hashchain.py is the only place that decides who Dependabot is: the commit-log check asks it for the commits to skip, and the description check excuses a pull request only when GitHub reports dependabot[bot] (account type Bot) as its opener and every commit in it is Dependabot's. A forged author address therefore buys nothing there for a contributor who cannot push to Dependabot's branches, and a person's commit on the branch is read like any other. The hashchain and commit-log checks in CI still go by the author line alone, so a forged address skips them. Its link is derived from the preceding link and its tree, including a consecutive run of bot commits; any trailer in a bot message is ignored consistently. A normal hooked commit or amend can follow such a parent directly. Rechain preserves accepted bot commits when their parent is unchanged; when an earlier repair changes their parent, it preserves their message rather than imposing contributor message rules. Any other commit made without the hooks (a web edit, --no-verify) fails verification until rechain restamps it.

scripts/test-interactive-review.sh runs every test that covers this workflow (chain, thresholds and evidence, the workbench server and worker, the hooks in a scratch repository, and the page in a browser) without the rest of the gate.

post-commit never runs scripts that the commit itself brought in: it is skipped for a commit that changes the hook scripts or the review skill, and during a merge, rebase or cherry-pick sequence. Such a commit keeps the Change-Id its commit-msg hook computed, which is correct unless it was amended with a fresh message; run python3 scripts/hashchain.py rechain before pushing if pre-push reports a broken link.

What cannot be enforced locally

Nothing on a contributor's machine can make the hooks or the interactive review impossible to skip, for a person or for an agent. git commit --no-verify, core.hooksPath, building a commit with commit-tree, and rechain all work for anyone with a shell, and a Change-Id is a hash of public inputs, so it proves that a commit is consistent and not that a hook ran. What this tree does instead is layer three things, each closing a different gap:

• Hooks do the work at commit and push time, which is when it is cheapest to fix, and catch the honest mistake.
• scripts/agent-guard.py, registered in .claude/settings.json as a Claude Code PreToolUse hook, refuses an agent's shell command that skips the hooks (--no-verify, core.hooksPath, commit-tree, update-ref, editing .git/hooks or a hook script) and a shell reader pointed at a per-developer credential file, a glob over them or the directory itself under config/. It stops an agent that follows its permissions from doing it by habit or to get past an error; it does not stop a determined one, and it does not cover another agent runner, which should be given the same rule in its own configuration.
• CI recomputes everything it can and is the only part a contributor cannot edit locally: the Change-Id chain, the size of every commit and of the whole pull request, and that a change over the review thresholds carries a maintainer's explained label. It reads nothing the author writes in the description. It runs the base branch's checker, so a pull request cannot weaken it. It holds only if the repository requires it: make check and interactive-review required status checks, merge only through the merge queue, let only the maintainer bypass them, from a pull request and never by pushing to main, and require maintainer review for changes to scripts/hashchain.py, scripts/check-interactive-review.py, scripts/check-queue-review.py, .claude/skills/codetrial-review/commit-sizes.py, scripts/agent-guard.py, .claude/settings.json and .github/workflows/.

pr-description.yml runs on pull_request_target, so the workflow that runs is the base branch's and a pull request cannot edit or remove it. That trigger runs with the base repository's token, which is why the job asks for read access only, checks the pull request out as data and runs only the base branch's checker. It is also why apply-pr-actions.sh otherwise sends every approval of a first-time contributor's run to the confirmed list while a workflow uses that trigger: pr-description.yml is the one workflow it names as reviewed, and only for pull_request_target. A new workflow on either base-token trigger needs the same review, and the script's tests fail until it has one.

The required job is named interactive-review (the workflow is pr-description); require that job and check in the branch ruleset. The workflow also handles merge_group: using the base branch's checker, it lists open PRs targeting that branch, selects heads present in the group but absent from its base, and rechecks each one's size against its own head and its group base, and its labels. Every ordinary commit in the group must belong to one of those heads; missing PRs, a missing label and API errors fail the check. Merge commits are verified by the separate hashchain gate. This requires the merge method described below; squash and rebase queues are unsupported. Queue checks use a read-only token with pull-requests: read to retrieve the open PRs and their labels, never local answers. The explained label is not bound to a head, so a push keeps it; the ruleset's dismissal of stale approvals is what asks for a fresh review.

Use GitHub's required-review rules as the independent approval boundary: require at least one non-author maintainer approval, require code-owner review for the policy files and all their checker dependencies, dismiss stale approvals when new commits are pushed. Require these rules for everyone except the maintainer, who may bypass them from a pull request only. That bypass skips the review, the required checks and the queue for the maintainer's own pull requests, so the gate binds contributors, not the maintainer. A merge group runs the workflow file as the queue ref has it, edits from the pull request included, and a job its own if: skips counts as passing, so loading the checker from base is not enough. The pull-request run is the base branch's workflow, so it is where such an edit is stopped: the --guard step of scripts/check-queue-review.py fails when the pull request changes pr-description.yml from where its branch began, or adds another workflow that mentions interactive-review anywhere, or removes one of the checker scripts the workflow runs (it skips itself while the base has none of them, so a deletion would switch the check off). The workflow has no job-level condition for the same reason. A maintainer approves a deliberate change by setting the repository variable CODETRIAL_POLICY_APPROVED_HEAD to the exact head after reading it (one head at a time); Dependabot gets no exception, since its action bumps edit this file. The guard is a text check on one file. It does not protect check.yml, or a workflow that posts a status by API, so code-owner review of .github/workflows/ and the organization's required-workflow rule remain the boundary for those. Repository owners must configure these settings; adding workflow code does not enable them. Land the workflow and the checkers on the protected base before enabling the new required check. Approval means a maintainer accepted the reviewed change; it does not certify that the author understood it.

What remains open is proof that a person did the interactive review: CI cannot see the local evidence, and anything the page records, a script on the same machine can record too. A WebAuthn assertion can bind a credential holder's confirmation to a server challenge containing the diff identity. It does not prove reading or understanding, and the author can confirm an unreviewed diff. It must not be presented as proof that the hooks or review ran.

If independently recorded interaction becomes necessary, use a separate trusted service to record the exchange and issue a signed receipt containing the repository identity, PR number, exact head and base SHAs, canonical diff hash, policy version and a unique session ID. Keep its signing key outside contributor machines, and have CI verify the signature, issuer, policy, repository and every snapshot field against current data. Reject receipts copied between PRs or replayed after any snapshot change. Keep answers private. Such a receipt proves what the service observed, not comprehension. This service and receipt verifier are future work; the current explained label remains a maintainer's word.

The chain is a checksum of each parent link and tree, not proof of review: a commit made with --no-verify can be restamped by rechain and still verify. It catches accidental history edits. Interactive-review evidence lives under .git/codetrial-review/ on the contributor's machine, so the pre-push check is local deterrence and CI does not see it.

Each ordinary commit continues the link of its own parent, including commits on both sides of a merge. GitHub's two-parent integration commits need no Change-Id trailer: their tree must equal a successful git merge-tree --write-tree result for those exact parents. Conflicts, extra edits and octopus merges fail. Resolve conflicts on the topic branch and rebase before integration. Repository and merge-queue settings must retain the reviewed topic commits under a merge commit; squash and server-side rebase do not preserve these parent-linked IDs. Verification requires Git 2.45 or newer for --no-lazy-fetch and merge-tree with a pinned attribute source. Each merge is recomputed in a temporary bare object context using its first parent's attributes, independent of the current checkout, index, local attributes or configured merge drivers. The original object database is read through alternates; computed objects stay in the temporary context. A merge's descendant link is SHA-256 of codetrial/merge:<full merge SHA>, truncated to 40 hex digits, so changing its tree, parents or message also changes subsequent links.

After a rebase, explicitly run python3 scripts/hashchain.py rechain on a work branch, never stability. Repair verifies and preserves the most recent integration on the first-parent path, then repairs only the linear topic suffix. It refuses to repair an invalid integrated prefix rather than rewriting main. The repair preserves trees and author metadata, removes invalidated Git signatures, changes only necessary parent/message headers, and adds no repair trailer. It does not change the index or worktree. The chain detects missing, copied or stale links; a reproducible hash is not authentication that a human actually ran the hooks. CI independently verifies the chain and code gates.

The anchor is public policy, not a secret. Keeping its commit ID private does not turn an unkeyed hash into proof of hook execution: anyone able to create commits can recompute the same links, including with rechain. Standalone verification defaults to policy in the submitted history; use verify --tip <head> --anchor <full-trusted-commit-sha> to pin the boundary independently. A symbolic ref or abbreviated anchor is refused. Pre-push pins it to the canonical upstream main policy when available. CI uses the PR base, merge-group base or previous push tip policy and verifier, so the outgoing branch cannot reset its own policy or verifier. Pull-request jobs run the workflow from the PR's merge ref, so steps that run before the verifier are contributor-controlled: protect the required workflow as described below, and treat this as a guard against accidental drift, not against a hostile author. Missing trusted verifier code fails the check rather than falling back to the candidate's code. Once the policy is installed, trusted verification rejects a submitted tip that changes or deletes its anchor file, even if its links still match the old anchor. A changed seed also changes the links, so a future migration needs coordinated maintainer-controlled policy installation and history repair; the ordinary candidate cannot authorize that transition. Protect the required CI workflow against contributor changes through repository rules; local hooks and private review records are not tamper-proof remote attestations.

The first policy installation has no trusted verifier in its base. CI refuses this case unless a maintainer sets the repository Actions variable CODETRIAL_HASHCHAIN_BOOTSTRAP_HEAD to the exact full reviewed PR head SHA. Only that committed verifier is loaded, never the working-tree copy, and the initial anchor remains b6eb8ba9bfbb14ff8d8a7dee920d29590ee89f25. The approved head may be wrapped in one clean two-parent integration whose first parent is the event's trusted base/previous push tip and whose second parent is exactly that approved head. The resulting merge must retain the approved verifier and policy blobs; the approved verifier checks its entire graph and merge tree. This covers the first queue integration and the corresponding main push without guessing or pre-approving their newly generated SHAs. New topic heads need new approval. For initial installation, a batched queue wrapping several PRs is refused unless the exact tested group head is explicitly approved instead. Remove the variable after installation. This exception is an explicit approval of verifier code, not evidence that hooks or interactive review ran; it grants no approval of answers. No repository variable is set by the local installation or tests.

Review patches and line counts use one canonical object-based diff. Textconv, external drivers, current checkout/index attributes and local diff configuration cannot change evidence for a committed base/head. Attributes come from the target head, or from each measured commit for parent-relative sizes. Dirty previews use a copied index and the existing worktree while retaining HEAD's attribute policy; they remain provisional. A corrected canonical format may require a fresh bundle for older previews; retain their author records instead of silently rebinding them. Certificate revocation syncs the certificate directory before changed answers are saved, so an acknowledged edit cannot depend on an unsynced deletion.

Hashchain and size inspection ignore Git replacement objects and refuse local history grafts or a shallow repository. Fetch complete history and remove grafts before retrying; a truncated or rewritten local graph cannot clear the large-change requirement.

To inspect the interactive requirement without pushing, run:

python3 scripts/check-interactive-review.py status --base origin/main --head HEAD

Use the intended upstream PR base when it differs from origin/main. The JSON names every triggering full commit SHA and its added plus deleted text lines, the final PR text total, and not_required, pending, or complete. A large commit remains a trigger when later commits undo it. coverage_complete is false and unmeasured_commits names the SHAs and paths when Git cannot report text line counts. These changes also require interactive review: binary attributes such as -diff cannot turn a large text change into an automatic pass. Inspect real binaries separately during that review. Missing history is an error. complete requires current matching evidence, not a commit-message label or a line in the reviewed-head file. The evidence binds the whole reviewed branch head and base; it does not certify a separate per-commit review. Rebase, amend, base changes, or edited answers require revalidation. No Reviewed: yes or Rechained: yes trailer is generated.

Interactive review events are displayed in local Git notes under refs/notes/codetrial-review. Post-commit marks individual large/unmeasured commits, and pre-push includes the full PR aggregate. The head's note names its Change-Id, exact head/base, triggering commit SHAs, pending/complete state and whether the matching workbench server started. View it with git log -1 --notes=codetrial-review; starting a server is not completion. Only certificate verification can produce the generated complete state, and the push hook never trusts the note as proof. Editing answers revokes the certificate and refreshes the note to pending. The note contains no private explanation or feedback. Notes preserve commit identities and are not pushed automatically. See the marker lifecycle.

Releases and the embedded web tree

The prerelease is published by the build and release jobs in .github/workflows/check.yml, which run only on a push to main. No repository secrets are involved: the macOS binary ships with the linker's ad-hoc signature and nothing else, so a fork builds the same artifacts this repository does. Developer ID signing and notarization would need a paid Apple Developer Program membership, and the download instructions clear quarantine instead.

The desktop app is released separately, from .github/workflows/desktop.yml when a desktop-vX.Y.Z tag is pushed, and never touches the CLI's latest; desktop.md has the procedure and what that workflow checks.

The embed falls back per file rather than wholesale, so a partially populated web tree is filled in from the built-in copy rather than 404ing. Assets on disk win where they exist. That root is web/ relative to the working directory unless CODETRIAL_WEB_DIR names another, so running the binary from a checkout picks up edits with no environment variable set.

Static assets use ETags for revalidation. Ordinary files require revalidation with no-cache; vendored assets may be cached for one day. Conditional GET and HEAD accept If-None-Match: *, tag lists, repeated header fields, and weak or strong spellings of the same tag. A matching asset returns a bodyless 304 with its cache policy, ETag when available, and Vary: accept-encoding. Unknown or stale tags return the current representation.

Agent working method

.claude/skills/codetrial-change/SKILL.md is the order an agent works in on a change (failing test first, smallest edit, root cause, small ordered commits) and the limits it keeps inside the task it was given. It restates no gate; the gate is scripts/test.sh, and the layers that hold when an agent does not follow it are under "What cannot be enforced locally".

The skill's "What is automated" table names the hooks and checks behind it and what each cannot see. Gate receipts, one file per lane with the verdict, skipped count and scripts/tree-fingerprint.sh of the tree, live under .git/codetrial-review/gate-receipts/. A receipt shows that a run happened on this tree, not that it was complete; CI recomputes the verdict.

Review memory and the review skill

/codetrial-review in Claude Code, and $codetrial-review in Codex (which reads the same skill through .agents/skills), review a diff against the project's own history instead of from a blank slate. That history lives in memory.db, a SQLite file under the git common directory (<git-dir>/codetrial-review/memory.db), or wherever CODETRIAL_MEMORY points. It holds two kinds of data with different trust:

• Public project memory: every pull request and issue discussion of this repository, searchable, with author, date, trust level and source. Review preparation draws the entries relevant to the diff into history.md for the reviewers, and memory.py search, why, fixes and show open a decisive source.
• Private review experience: the findings a reviewer confirmed, the claims it refuted, the questions left unproven, and later fixes, recorded by this clone's own reviews. Preparation retrieves the cases that touch the same files into experience.md. It is a lead to recheck against the current code, never a maintainer ruling. It stays on the machine that wrote it: it is excluded from every published snapshot and the file is owner-only.

Each CodeTrial skill ends with a closeout step. Review closeout runs review-lifecycle.py finish, which checks that the report still matches the bundle it started from, records the report, and marks the head reviewed only after the record succeeded. A failed write is kept and retried with review-lifecycle.py --retry. Other skills record through memory.py experience record. The confirmed and refuted cases of complete runs are then retrieved for later reviews that touch the same files.

A new clone does not collect the history itself. memory.py ensure downloads the latest verified memory.db.gz from a review-memory-* release, so the first review costs one download and an incremental poll rather than a backfill. The Review memory workflow (.github/workflows/review-memory.yml) builds and publishes that release from main every six hours; the file is checked against the signed manifest and the publisher's provenance before it is installed, and installing it keeps any private experience and local decisions already present. Until a release exists, or when the download cannot be verified, ensure reports a degraded status and the review continues without history. A full backfill from GitHub is slow and runs only when asked.

The snapshot contract is in .claude/skills/codetrial-review/snapshots.md, and the experience contract in .claude/skills/codetrial-review/experience.md.

Generated files

web/problems/, web/judges/, and the problem cards in web/index.html are generated from problem-bank/. Regenerate them after editing that source, or the gate fails on the drift:

python3 scripts/gen-problems.py
python3 scripts/gen-problem-cards.py

The generated Rust tables include scenario metadata, topics, private guides, and the private rubric in src/agent/problem_rubrics.rs. Do not edit a generated table directly. Adding a problem describes the source files, required checks, and the difference between an imported and an original exercise.

scripts/top-interview-150.json records which problems the study plan asks for. Refresh it from LeetCode with:

python3 scripts/gen-problems.py --sync-study-plan

The sync refuses to write when the plan and problem-bank/ disagree, naming the imported problems each side is missing. Port those first. Original exercises are deliberately outside the plan, so sync and drift checks leave them alone. --check holds the committed manifest to the same imported set, so drift fails the gate offline.

Two commands cover the porting. --plan-drift asks LeetCode what changed without writing anything, and --scaffold SLUG prints the problems.json and judges.json entries for one problem, filled in as far as LeetCode's metadata reaches:

python3 scripts/gen-problems.py --plan-drift
python3 scripts/gen-problems.py --scaffold reverse-linked-list-ii

It stops where it has to. The statement, examples and constraints stay empty because the fetcher never asks LeetCode for problem prose, and each judge case carries its input without an expected value, because exampleTestcases is inputs only. Both are written by someone who has read the problem.

A ported problem also needs its entry in problem-bank/variants.json, and the generator refuses to run without one. The page shows a scenario written around the same contract instead of the published problem, and the interviewer holds the rest. Each variant has a title and a brief, which the page shows; a private contract the interviewer judges against; one or two examples, each naming a judge case by index and not all of them published ones; clarifications answered only when asked, with the constraints and edge-case policies the brief leaves out; followUps for after a tested solution; and three hints, a nudge, a direction and the key step.

A function problem also declares a new entry and a class problem a new className, since the published name is as recognisable as the title. Either may declare parameters where a parameter name gives it away, and terms for any other word a starter snippet carries, such as a comment naming the published node type. The bank keeps LeetCode's names; the generator puts the variant's in place in the judge and starter code it writes, and refuses a variant whose title, brief, contract, examples, constraints, starter code or judge cases still name the published problem.

The browser knows a problem only by its page name, made from the scenario title: the interview URL, the page and judge files, the token request and saved history all use it, and the server resolves it to the problem. Each page names its published title once, as source, which the interview shows in small print beside the scenario so the problem can be found again afterwards. The published id is in one file, web/problem-pages.json, which the browser fetches only to open a link that still carries a published id, to read history saved before pages had names, when the candidate asks the lobby to show the titles on the cards, or to show or export a completed report. Completed reports, including those opened from Past attempts or a replay, and downloaded Markdown keep the scenario title and add an optional Original problem link with the published title and LeetCode URL. Collapsed attempts do not request the map for this link; opening or exporting their report does. Successful lookups are cached for the page's lifetime. Missing or malformed mappings leave the link out, and a failed lookup or a wait longer than ten seconds does not block a Markdown download: it proceeds without the link.

None of this is secrecy. The page map, each page's source and every judge are served to anyone who asks, and a candidate with developer tools can map a scenario back to its published problem and read its test cases. The disguise exists so an honest candidate meets the problem the way an interview poses it; nothing that must hold against a determined one, integrity checks included, may rely on it. Keep a variant's title stable once it ships: a retired page name opens the default exercise, with a note saying so.

problem-bank/guides.json holds optional solution notes for the report reviewer, with the license they are used under. The generator refuses a note for a problem the bank does not have, or one still carrying page furniture from the import, and writes src/agent/problem_guides.rs. Only the report prompt reads them.

Checks outside the gate

Whether the interviewer actually follows the live prompt, rather than whether the prompt says the right things, needs a Gemini key. The check scripts a candidate through three problems against a text model given the same instructions, greeting and tools, and fails on a named source, a volunteered limit, an unanswered size question, or a hint that goes past the rung it was served:

scripts/interview-behavior-check.sh
BEHAVIOR_PROBLEMS=3sum,lru-cache scripts/interview-behavior-check.sh

A free key allows fifteen requests a minute, so the check waits out rate limits; three problems take about two minutes.

BEHAVIOR_CHECK_MODE=injection runs only the instruction-injection probe. It takes an explicit call cap and a token ceiling that can overshoot by one call, and needs an isolated config and a journal path outside the repository; tests/fixtures/report-eval/README.md describes it.

The end-to-end browser check additionally needs Playwright and Chromium:

npm ci && npx playwright install chromium
scripts/browser-check.sh

BROWSER_CHECK_FLOW=soak is the long-conversation acceptance, and the only check that covers an interview outliving one Gemini socket. It needs real credentials and runs for eleven minutes:

BROWSER_CHECK_FLOW=soak BROWSER_CHECK_AGENT=dispatch scripts/browser-check.sh

The duration defaults to 660 seconds, past the Live API's ten-minute connection cap, because a soak that stops short of the cap crosses no handover and proves only that the room stayed up. BROWSER_CHECK_SOAK_SECONDS lengthens it and belongs to this flow alone; naming it on another one fails rather than being ignored, and so does a value under the cap.

Presence is the weakest of the things it judges, because the interviewer stays in the room through every way this can go wrong. So the flow also requires, from the runtime log and the transcript panel:

• Gemini asked for the restart, which is the proactive path. Reaching a new socket by way of the close resumes too, so a check that asked only whether it resumed would pass with that path dead.
• The replacement resumed rather than restarting cold. A cold one keeps the interviewer in the room having lost the conversation.
• No degraded restart at any point, reported where it happened.
• Every replacement episode is held to the reply it owed, matched by ids and not by the transcript panel. An episode that began with a reply owed passes only when that same reply (same pid, session and obligation) was answered with audio first accepted on a socket at or after the one the episode installed; audio taken on the replaced socket, an earlier recovery's answer and another room's or session's reply satisfy nothing. An episode that began with nothing owed is classified idle, and one whose reply a hold, cut or barge-in ended before any audio is classified held; neither is made to speak. A start with no end, a swap whose reply never finished and a cut log fail. Each episode is printed with its status and reason, and BROWSER_CHECK_SOAK_RESULT names a file that receives the same as JSON.
• Every swap that was answered with audio is followed by decoded audio on the track. The track is reused across Gemini sockets, so the sample taken when it was first subscribed is older than the socket under test and covers nothing. The page marks each installed swap, waits for the first Speaking transition after the mark (audio already playing when it was marked is the old socket's), and samples the track again from a fresh counter baseline, within a fixed number of samples. A swap passes with such a sample of its own that has received packets and a decoded signal, or when a later swap's sample covers it after the later mark replaced a sample that had not opened its window. A missing, stale, silent or unavailable sample fails, as does one whose window did not open after its mark. If the track is resubscribed before the sample is taken, it is taken again on the new track from the same mark. A swap that owed no audio, or whose reply was held, is not made to speak. The sample proves decoded audio on the receiving track after the mark, ordered by the page's Speaking transitions; it is not tied to the swap's own reply, which the reply check above settles, and a reply that began between the install and the mark (the log is read every quarter second) is missed, so the next transition is used and may not come within the grace. It is not evidence that anything was physically audible and it does not time when the audio began.

BROWSER_CHECK_SOAK_VERDICT_MODE is strict (the default) or pilot, and belongs to the soak alone. Strict is the qualification and keeps the cap of one rebuild from local state after a reply timeout. Only a degraded watchdog rebuild whose fallback is no_handle or resume_suppressed counts toward the cap; a natural GoAway swap never does, and a watchdog swap that resumed the session is reported apart and counts for nothing. Pilot leaves that cap alone out of its verdict, so its verdict is the structural one: a reply, audio, GoAway or handover failure still exits 1, and a soak without a GoAway still exits 77. A pilot prints qualification=not_evaluated and both outcomes, so a pilot that completed is not a pass of the strict check.

BROWSER_CHECK_SOAK_RESULT receives mode, verdict, structural_verdict, strict_verdict, qualification, watchdog_rebuilds_total (every degraded watchdog rebuild), strict_cap_eligible_total (those that count toward the cap), watchdog_resumed_total, goaway_swaps_total, exposure_s, the per-episode reply and audio results, and the capture file holds the same under soak. exposure_s is the largest assessment clock of the interview clock: samples when they are anchored and in order; when they are missing or inconsistent it is null (with exposure_basis saying why), never zero. A rate per hour of any of these totals is meaningful only with a known positive exposure_s.

Checks that need credentials or a running service stay out of the gate and are run on their own: scripts/server-check.sh, gemini-check.sh, parity-check.sh, report-parity-check.sh, visual-parity-check.sh, recording-integration.sh, and recording-provision-check.sh.

Instructor task packaging

The instructor tool and its acceptance tests use the pinned Python dependency in scripts/requirements-task.txt. scripts/task-package.py finds its own interpreter: it needs Python 3.9 or newer (the test gate needs 3.11, for its Rust style checker), and reruns itself under a newer python3.x on PATH when started on an older one; build also needs the dependency, and reruns under target/task-tools (CODETRIAL_TASK_TOOLS names another place), or else under a python3.x on PATH that already has it, and only then sets up target/task-tools from that file. It creates that directory and never clears one, so a broken one has to be removed by hand. ./scripts/test.sh uses the same environment and sets it up the same way, in a full run or the portable lane, the only one with packaging tests. Given a PYTHON without the dependency, it skips those tests instead. To set it up by hand:

python3 -m venv target/task-tools
target/task-tools/bin/pip install -r scripts/requirements-task.txt

scripts/task-package.py --help lists references, new, validate, render-prompt, preview, build and publish-scan; task-mode.md says what each does. The tooling never uploads or pushes files.

Run budget for live validation

A live validation run spends provider money from the moment a candidate joins, and a worker that keeps its own count can lose it in a page reload. The budget lives in a separate process instead: scripts/interview-run-budget.py launches the worker, owns the count of scripted actions, samples the clock and reaps what the run started. Nothing here calls a provider; the fixture suite tests/test_interview_run_budget.py runs in the gate with fake workers and a Unix socket.

The manifest is a JSON file with schema_version 1. It names the run, pins every executable by absolute path and sha256 (the candidate, and a control or a test worker if the run uses one), and records the settings and fixture hashes, selected_duration_s, outer_wall_cap_s, cleanup_reserve_s (at least 60), end_mode, the integer limits, the declared scopes and the exact actions list. The list holds the nonfinal action IDs, common setup such as candidate_join included. action_ceiling must equal that count plus one: the extra permit is the reserved end, which finalization spends whether or not the scenario displays an explicit End. A duplicate ID, a listed end, or a count that does not match is refused, and so is a list without candidate_join, the action that gates the re-check of the pinned executables. Executables are built and hashed before the run; the launcher re-hashes them before the join permit, and browser-check.sh runs the pinned candidate, for the server and for the agent, instead of building one. Each run needs an empty artifact directory, so a rerun cannot start from a fresh count.

Proposed launcher command, run from the repository root:

python3 scripts/interview-run-budget.py validate \
    --manifest run/manifest.json --verify-binaries
python3 scripts/interview-run-budget.py run \
    --manifest run/manifest.json --artifact-dir run/artifacts \
    [--resource-cleaner PATH] -- \
    scripts/browser-check.sh

validate is free and should always precede run. run passes the worker BROWSER_CHECK_RUN_MANIFEST, BROWSER_CHECK_RUN_ID, BROWSER_CHECK_RUN_MANIFEST_SHA256 (the hash of the manifest bytes), BROWSER_CHECK_BUDGET_SOCKET, BROWSER_CHECK_BUDGET_TELEMETRY and BROWSER_CHECK_BUDGET_STOP. scripts/browser-check.sh refuses a partial set, a run ID or hash that disagrees with the manifest, a binary whose sha256 changed, and CODETRIAL_WEB_URL, since an external server is shared and was not built from the pinned executable. With BROWSER_CHECK_VALIDATE_ENV_ONLY=1 it stops after those checks, so they are testable without a browser.

BROWSER_CHECK_FLOW=long-interview holds one real interview for a duration the manifest names and judges that it ran that long. It needs a budgeted run and BROWSER_CHECK_AGENT=rust or dispatch. The manifest adds run_kind (smoke, pilot, qualification, experiment or clip), duration_min and observation_target_s, and end_mode gains manual, server_deadline, absence, room_closed, terminal_cancel beside native_close; explicit_end is the older spelling of manual and is not a long interview. interview-run-budget.py checks only their shape. scripts/long-interview-plan.cjs owns what they mean for a browser run, and browser-check.sh and browser-check.cjs both call it under BROWSER_CHECK_VALIDATE_ENV_ONLY=1, before anything is built or started:

• BROWSER_CHECK_DURATION_MIN is harness-only, 20 when unset, and must be 10, 20, 30, 60 or 90, inside the product's 10 to 90 minute range. It sets the page URL, the token request, the agent's bootstrap and the countdown, and the run fails if the token, the bootstrap line or the countdown disagrees. A recording cap (CODETRIAL_RECORDING_MAX_MINUTES, from the environment or the config file) below the duration is refused rather than clamped. Every other flow refuses the variable, so a 660 second soak never becomes a long interview by selecting 60 or 90.
• A pilot runs 60 minutes and a qualification 90, both with a manual End and observation_target_s equal to duration_min * 60. An experiment names its frozen length in experiment_profile_min (20 or 30), which must equal duration_min; no document in this repository defines that profile, so it is a manifest field instead of a constant. A smoke may observe a shorter positive target. A server_deadline smoke sends nothing, so the real route ends the interview, and observes the duration plus the agent's 120 second grace.
• clip and native_close belong to the native worker and are refused for a browser run, as are absence, room_closed and terminal_cancel, which this flow does not drive. A clip emits no assessment report and satisfies no report gate.
• A run with no bootstrap line, token response, countdown or interview clock sample (for a target of two minutes or more) fails instead of passing on nothing, and so does a clock whose origin is missing, moves or lies outside the time the run was alive.

A long interview's wall windows are separate and sit outside the selected duration: 300 seconds of setup, 300 of end and report, 60 of teardown, so outer_wall_cap_s is selected_duration_s plus 660 and cleanup_reserve_s is 360. These are harness ceilings, not production latency guarantees, and the report generator's own 125 second limit bounds one step of the end window, not evidence settlement, the farewell, delivery or cleanup. scripts/long-interview-timing.cjs names three moments that must not be read as one: the last-stimulus boundary (the duration less a 120 second reserve, the 90 seconds of serial handover and progress grace the harness already allows plus a 30 second margin; the final question starts 30 seconds before it), the end of the full interview observation (the whole selected duration, which is the only thing that may be called a full 60 or 90 minute interview), and the completion of the end and report work. Stimuli stop at the boundary, the connected session and the observation continue to the end of the duration, and then the harness deliberately ends. The run is anchored to the interview clock's start_unix_ms, or failing that the first token request, so a page reload cannot reset it, and the exposure it actually had is recorded. The reserve is revalidated before launch against the resolved handover grace (BROWSER_CHECK_HANDOVER_GRACE_S, 90 when unset). A GoAway rotation that begins at or after the boundary is late_idle or late_terminal_preempted and never counts toward the two completed rotations a run needs.

run also samples the owned process tree (RSS, open descriptors and threads for every process, keyed by PID and start time so a reused PID is another process) every 30 seconds into resource-samples.jsonl, for either arm of a comparison (--arm candidate|baseline). The file is appended as the run goes, so nothing depends on a final tail. A value that could not be read is null, a tree total is the sum of the values that were read with the number of processes that gave none, and a tree with none readable has a null total: unknown, never zero. The manifest's max_archive_bytes bounds the whole artifact directory. It must lie between a floor (the sampler's worst case, floor(wall_cap / 30) + 2 samples of at most 4096 bytes, plus 256 KiB for the controller's own files) and that floor plus 256 MiB, so the bound is finite and derived, and a manifest that names a duration_min must carry one. A run that would exceed it, or whose sampler would outgrow its own bound, stops, writes incomplete_artifact.json and exits 6; nothing is evicted to make room. Exit codes: 1 a structural failure, 4 incomplete_budget, 5 incomplete_cleanup, 6 incomplete_artifact, 77 no GoAway (inconclusive). When several apply the exit names the most serious in that order (5, 6, 4, 1, 77) and each class still has its own artifact.

A worker asks for each scripted action by ID and sequence before dispatching it (scripts/run-budget-guard.cjs for the browser, WorkerGuard in the launcher script for Python). The launcher writes the grant to budget-ledger.jsonl before it replies, so an action is granted at most once and a lost reply still counts as spent. A duplicate, unknown, out-of-sequence or malformed request, a reply that is not an exact grant, a lost connection and a timeout over one second all deny and are never retried. Counts live in the launcher, so a page reload or a reconnecting worker asks for the current count instead of restarting it. The launcher stops granting new work at outer_wall_cap_s minus cleanup_reserve_s and grants only the reserved end after that, until the reap starts; it then sends TERM and, if needed, KILL to the processes it started, recorded at each sample, and to nothing else. A SIGTERM, SIGINT or SIGHUP to the launcher takes the same path at once. A killed launcher leaves the worker unable to get a permit, and WorkerGuard.watch_launcher lets it run its own bounded cleanup; a dead worker leaves the launcher to reap what it leaked.

What is enforced and what is only observed:

• Hard: scripted actions (permits), wall time (the stop-new-work deadline is also checked in each permit, so a stalled sampler cannot open the cleanup window to new work), and agent-alive exposure when the manifest declares it hard. Exposure is wall time measured from the worker's agent_alive telemetry, a conservative bound under the single-socket invariant and not a bill. If its heartbeat stops, the run fails rather than continuing blind.
• Observed: model turns, tool calls, HTTP requests, audio frames and restarts, reported from the worker's telemetry as a count, or as unknown when none arrived, never as zero. A manifest may not declare any of these hard, and provider_invoice may only be not_measured; the preflight refuses a manifest that asks for a limit nothing can enforce, as it does a run with named resources and no --resource-cleaner, a sampler slower than one second or a pin too large to re-hash inside the permit timeout.

Exit codes: 0 complete, 1 the worker failed inside its budget, 2 preflight refused the run before anything started, 4 incomplete_budget, 5 incomplete_cleanup (a surviving owned process, a failed resource cleanup or a reap over its budget). The artifact directory keeps budget-result.json, budget-ledger.jsonl, and incomplete_budget.json or incomplete_cleanup.json when either applies.

Limits of the process tracking: it reads /proc, or ps elsewhere, and on Linux marks the launcher a child subreaper so orphans stay in the tree; without that mark a daemon that reparents and changes session between two samples is not found. The KILL stage cannot remove a process stuck in uninterruptible sleep; it is reported as a survivor.

Native SDK recovery fixture

tests/unit/livekit/recovery_live.rs holds the one credentialed recovery check that needs no browser: an ignored test, live_sdk_control_presence_and_report_delivery, that hosts the real room loop in the test process with a real LiveKit SDK candidate and the configured Gemini. It is a worker for the controller above and refuses to start without the four budget variables. Real: the LiveKit room and transport on both ends, the candidate's data packets, the room loop and recovery driver, Gemini Live and the report call. Not real: speech (nothing is spoken; the editor state is authored text on the code topic), the page (the report receipt is the digest the page sends), and the recovery trigger, which ends the first Live reader through a test-only inspection so the loop takes its own Close path and reaches the prepared recovery write that a test-only gate holds. Every effect, including starting the room loop, is taken behind a permit; reads and waits are not actions. The candidate token is minted by the token endpoint's own function, so durations follow the same 10 to 90 minute and recording-cap rules.

scripts/live-sdk-fixture.sh build DIR            # once; pins DIR/live-fixture
CODETRIAL_LIVE_CONFIG=PATH scripts/live-sdk-fixture.sh run a DIR

CODETRIAL_LIVE_CONFIG is a config file for an isolated LiveKit project and a Gemini key with an agreed quota. run takes one scenario letter, writes that scenario's manifest from the test binary, validates it with the pinned hash, and launches the controller; a scenario runs once per DIR, and nothing reruns it. The log is DIR/worker-<letter>.log with URL credentials redacted, and the fixture's observations are its LIVE_OBS lines. The controller re-hashes the pin inside the join permit, and by default refuses a pin that takes over 0.5 seconds to hash. A stripped release pin is large enough that a shared host under load can hash it slower than that, and a debug pin is far over, so build makes a release build, strips the copy and prints the hash time. run and run-clip pass --max-join-hash-s 0.7 (the ceiling below) and print the hash time they measure; nothing else raises the limit.

The run subcommand takes --max-join-hash-s SECONDS (default 0.5). A value that is not finite and positive, or is over 0.7, is refused at preflight. The ceiling is the permit timeout (1 s) minus a 0.3 s margin for everything else a join permit does (reading the request, one fsynced ledger line, the reply), so a hash at the ceiling still leaves the client 0.3 s of its 1 s. A result whose limit differs from the default carries "overrides": {"max_join_hash_s": N} in budget-result.json; a default run has no overrides key. At the join permit each pinned binary is hashed once, and the whole hash is timed: one over the limit is denied as join_hash_too_slow and moves the run to draining, never granted after the client has given up.

Letter	Script	Terminal	Report
a	join, seed, trigger, leave, return, End	manual End while recovery pends	one
b	join, seed, trigger	the loop's own 12 minute deadline (10 + 120 s)	one
c	join, seed, trigger, leave	absence grace (90 s) expires	none
d	join, seed, Thinking hold, trigger, release, End	manual End while the recovery write is still held	one
e	join, seed, trigger, room_delete	the server closes the room while the recovery write is held: the agent's SDK reports Disconnected and the loop takes Terminal::RoomClosed	none

Case e creates its room through create_interview_room, as production does, and its room_delete action deletes that room with the existing RoomService helper only after the prepared recovery write is held; the End permit stays reserved and unspent. Besides the conditions below, its verdict requires that the loop's own recovery terminal, read from a test-only probe keyed by the room, was RoomClosed (not End, Absent or Deadline) and arrived after the delete. A participants listing that finds no room is acceptable, since the room is already gone. The recovery matrix gate (scripts/check-recovery-matrix.py) accepts run e-room-closed-during-recovery (end mode room_closed, with controlled_recovery_trigger and room_delete granted and no End permit) as the representative RoomDisconnected observation, and RoomEventsEof's SDK column stands only on a validated RoomDisconnected claim, plain or representative.

The pass conditions are the fixture's judge: the gate was entered, the loop returned, exactly the declared number of final report receipts (a second copy counts), the End or the automatic terminal came while recovery was pending and not early, no participant is in the room before the fixture deletes its own named room, the room is gone afterwards, and every Live reader has finished. A passing run shows one crossing for one session. It does not show that a recovery-time cancel is correct for any other timing, and what it did not reach stays uncovered.

Native speech-continuity arms

The same file family holds a second ignored fixture, live_sdk_clip_continuity_arm in tests/unit/livekit/recovery_continuity.rs. It asks whether the SDK's bounded microphone queue keeps what a candidate said while the room loop is not reading it. A real SDK candidate publishes two authored, numbered speech clips (c01_say_plan and c01_say_loop of the 10-minute coding smoke schedule, six numbered turns) through a real audio track into the real room loop and the configured Gemini. A test-only gate in front of the loop's microphone dequeue (src/livekit/probe.rs, compiled only for tests; there is no production fault flag and no network rule) holds that one dequeue from about the clip's first frame (a consumer already waiting on the stream may take one more frame) while the room controls, the Gemini reader and the keepalive polling keep running, and releases it after the arm's stall. The queue holds 4000 ten-millisecond frames (40 s) and drops the oldest when full.

Arm	Dequeue held	Expected if the SDK keeps its documented rule
s0	never (matched control)	everything arrives as it is spoken
s20	20 s	all frames retained
s39	39 s	all frames retained
s41	41 s	the oldest 1 s of audio is dropped
s60	60 s	the oldest 20 s is dropped, so the first clip is lost

Each arm is one invocation under the B0.1 manifest: run_kind clip, end_mode native_close, 60 s of setup, 180 s for the clip, the hold and the drain, 60 s of cleanup (a 300 s cap), the planned actions candidate_join and clip_start, and the reserved close permit as the third. The fixture closes the candidate's connection and the interviewer's task itself and deletes its own room; it sends no End, so no assessment report is generated, and the arm cannot certify end or report behavior. The spoken clips live outside the checkout; the answer key is authored on the smoke-coding set, and the loader refuses a set whose clip text differs (the whiteboard set does).

scripts/live-sdk-fixture.sh build DIR
CODETRIAL_LIVE_CONFIG=PATH CODETRIAL_LIVE_CLIPS=/path/to/clips/out/smoke-coding \
    scripts/live-sdk-fixture.sh run-clip s0 DIR      # then s20, s39, s41, s60
scripts/live-sdk-fixture.sh clip-report DIR          # reads DIR/worker-s*.log

Each arm runs once per DIR and nothing reruns it. grep LIVE_OBS DIR/worker-<arm>.log shows the raw clip and turn IDs, the times the frames were actually taken from the SDK queue (a per-second tally, the first frame after the release), the Gemini input transcript lines as the room loop received them, and the room loop's own retention counters (ingress dropped_bytes, quarantined bytes). clip-report compares each arm's transcript with the answer key and counts missing, repeated and delayed turns per arm; a miss, repeat or delay the unstalled control shows too is reported as inconclusive, not as a loss, and an arm with no result is listed as not run. One clip per arm is descriptive evidence only: it is no estimate of a loss rate, and exact SDK drops are not inferred from the transcript, because the SDK counts them nowhere a caller can read and a transcript can lack words that were sent. The local recording-peer test (the_recording_peer_sees_nothing_while_held_and_the_retained_frames_after) runs the same gate against a fake Gemini socket offline and asserts the wire: nothing during the hold, then one message per retained frame, in order.