From cdd9b150ddb0d27776983fc213e18b2220026897 Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 12:50:49 -0400 Subject: [PATCH 1/6] docs: trim AGENTS.md to rules no tool enforces and drop vendor names AGENTS.md loads into every agent session. Keep only rules that change behavior and that no lint, test or CI job already enforces; point to the docs that hold procedures. Describe the independent high-risk auditor without naming a vendor, in AGENTS.md, CONTRIBUTING.md and doc/testing_matrix.md, and say which audit_kind each auditor records. --- AGENTS.md | 533 +++++++-------------------- CONTRIBUTING.md | 5 +- doc/high_risk_pre_merge_readiness.md | 5 +- doc/testing_matrix.md | 12 +- 4 files changed, 152 insertions(+), 403 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4572e6667..dbc239341 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,406 +1,151 @@ # AGENTS.md -Guidance for agentic coding assistants working in this repository. Keep this -file as the compact entrypoint for durable repo rules; put detailed matrices and -long procedures in discoverable docs. +Open a linked doc before working in its area; it holds the procedure this file +leaves out. -## Quick Commands +## Build and test ```bash dart run tool/prepare_workspace.dart -dart format . dart format --output=none --set-exit-if-changed . -dart analyze +dart analyze # CI's Pana step also fails on any info in lib/ +dart run tool/testing/check_platform_boundaries.dart dart test -p vm -j 1 --exclude-tags local-only dart test -p chrome --exclude-tags local-only ``` -The workspace preparation command resolves the root package and every -maintained example, and fails if any example or companion package is missing or -unclassified. Run it from a clean checkout before the root format/analyze -gates; CI uses the same entry point and fails if it leaves tracked files -modified. Never hand-edit a `pubspec.lock`: pin the offending dependency in -`pubspec.yaml` and let pub regenerate the lock, so its `sdks:` block stays one -pub can actually produce. Companion packages retain their own dependency, -analyze, test, SwiftPM, and publish-validation lanes. -Run repository-wide quality gates with the exact Flutter SDK CI pins: `3.47.1`, -recorded in `.flutter-version` at the repo root and in every -`subosito/flutter-action` step. Older Dart formatters produce different source -layouts, so a different SDK makes `dart format --set-exit-if-changed` disagree -with CI in both directions. Bumping the pin is its own PR: update -`.flutter-version` and the workflow steps together, and carry any reformatting -the new `dart_style` demands in that same PR. - -Use the testing matrix to choose validation for non-trivial changes: - -```bash -dart run tool/testing/test_matrix.dart --list -dart run tool/testing/test_matrix.dart --pr-template -dart run tool/testing/test_matrix.dart --tier essential -dart run tool/testing/run_local_e2e.dart --list -dart run tool/testing/run_local_e2e.dart --scenario --dry-run -``` - -For docs and release-sensitive snippets: - -```bash -./tool/docs/build_site.sh -./tool/docs/validate_links.sh -dart run tool/testing/verify_release_docs_versions.dart -``` - -For mixed chat-template handler changes, run the full parity suite below. -Rendering-only, parser-only, and grammar-only changes use the applicable checks -in `doc/high_risk_pre_merge_readiness.md`; compiled grammar checks are required -when grammar/schema enforcement is affected: - -```bash -tool/testing/run_template_parity_suites.sh -``` - -For coverage when `lib/` behavior changes or coverage is in doubt: - -```bash -dart test -p vm --coverage=coverage -dart run tool/testing/format_lcov.dart --lcov --in=coverage/test --out=coverage/lcov.info --report-on=lib --check-ignore -dart run tool/testing/check_lcov_threshold.dart coverage/lcov.info 70 -``` - -Heavy or device/model-backed scenarios stay out of default CI. Discover them -through `tool/testing/test_matrix.dart` and `tool/testing/run_local_e2e.dart`, -then record row ID, command or CI workflow, platform/device, model, backend, -PASS/FAIL/N/A, and log/artifact notes in the PR. - -Do not add one-off repro scripts under `tool/testing/` unless they are wired -into `tool/testing/run_local_e2e.dart`, represented in -`tool/testing/test_matrix.dart`, and documented where contributors can find -them. Prefer durable assertions in `test/unit/`, `test/integration/`, or -`test/e2e/`. - -For Web chat app validation after bridge/runtime or UI changes, use the local -E2E runner instead of ad hoc build/server steps. Preserve the runner's -`--base-href`, static headers, Playwright options, and macOS Chromium ANGLE -settings when debugging helpers directly. - -For llama.cpp n-gram speculative output-hash mismatches, compare the same prompt -and sampling settings against upstream `llama-server` before classifying the -mismatch as Dart-only. Use the benchmark runner's `--include-output` flag when -the exact generated text matters. - -## Code Style - -### Design Principles - -- Favor SOLID, DRY, and KISS: keep responsibilities focused, remove meaningful - duplication, and choose the simplest design that satisfies the current - requirement. -- Add abstractions only when they reduce real complexity, protect a stable - boundary, or match an established repo pattern. -- Prefer small, reviewable changes over broad refactors unless the broader - cleanup is necessary for correctness or maintainability. - -### Dart Style - -- Use `dart format` defaults. -- Keep imports ordered as SDK, package, then relative imports, separated by blank - lines. -- Avoid `show`/`hide` unless needed for deconfliction. -- Public APIs need explicit parameter, return, and field types. -- Local `var`/`final` inference is fine when the type is obvious. -- Public members require `///` Dartdoc with useful Markdown. -- Avoid new TODO/FIXME comments in source. Existing generated/template platform - files may contain upstream TODOs; do not copy that pattern into maintained - Dart or workflow code. - -### Names And Structure - -- Classes: `PascalCase`. -- Functions, methods, variables, parameters, and constants: `lowerCamelCase`. -- Files and directories: `snake_case`. -- Public exports live in `lib/llamadart.dart`; implementation stays under - `lib/src/`. -- Keep `LlamaEngine` free of `dart:ffi` and `dart:io` so web support remains - viable. -- Use conditional imports for platform-specific backends. -- Behavior tests under `test/unit/` mirror the source structure of `lib/src/`; - a source file with no behavior of its own (bare enum, marker interface, - `external` interop declarations) needs no test file and is listed in the - `behaviorlessSources` set of - `test/unit/test_structure/mirrored_unit_structure_test.dart`. Never assert - something that is true by construction (`expect(SomeType, isNotNull)`, - `Enum.values` containing a declared member); assert wire values where they - are consumed instead. See `test/README.md`. - -### Error Handling - -- Use the `LlamaException` hierarchy from `lib/src/core/exceptions.dart`. -- Throw `LlamaUnsupportedException` or a typed subtype for unsupported public - paths instead of silently reporting success. -- Include actionable diagnostics: missing capability, platform/runtime - condition, and version requirement where known. - -### Compatibility - -- New public APIs require tests and Dartdoc. -- Capability-dependent behavior needs both happy-path and unsupported or - version-skew tests. -- Generated files that should not count against coverage must include +- Run `prepare_workspace.dart` first, on a clean checkout; CI fails if it leaves + tracked files modified. Never hand-edit a `pubspec.lock`: constrain + `pubspec.yaml` and let pub regenerate it. +- Use the Flutter SDK in `.flutter-version`; other SDKs format differently from + CI. A pin bump is its own PR that updates `.flutter-version`, every workflow + `flutter-version:`, and the reformat it causes. +- The root gates skip `packages/`: companion packages keep their own + dependency, analyze, test, SwiftPM and publish-validation lanes (`ci.yml`). +- Chat app changes: `(cd example/chat_app && flutter test)`, plus + `flutter test --platform chrome test/chat_generation_service_test.dart` there + for Web-only paths such as LiteRT-LM generation settings. +- Pick further validation from `dart run tool/testing/test_matrix.dart --list` + (`doc/testing_matrix.md`) and put its `--pr-template` rows in the PR. +- Heavy, device- or model-backed checks stay out of default CI: tag such tests + `local-only`, or wire the scenario into `run_local_e2e.dart` and + `test_matrix.dart` and document it. Run one with `run_local_e2e.dart --list`, + then `--scenario --dry-run` before a real run. No one-off repro + scripts in `tool/testing/`; prefer durable tests under `test/`. +- Build, serve or Playwright-smoke the Web chat app only through + `run_local_e2e.dart` web scenarios or `scripts/build_chat_app_web.sh`; a plain + `flutter build web` omits the gitignored bridge assets. Check a reused build + with `scripts/validate_chat_app_web_build.sh`. To run a helper by hand, copy + what `--dry-run` prints. Keep the required `Web Chat Contract` CI job free of + large or availability-sensitive remote models. +- Before calling a llama.cpp n-gram speculative output-hash mismatch Dart-only, + compare the same prompt and sampling settings against upstream `llama-server`. +- Docs: `./tool/docs/build_site.sh` fails on broken links. + +## Code + +- Prefer small, reviewable changes and simple, focused designs without + meaningful duplication. Add an abstraction only to cut real complexity, + protect a stable boundary or match a repo pattern; refactor beyond the task + only when correctness or maintainability needs it. +- Order imports SDK, package, then relative, with a blank line between groups. + Avoid `show`/`hide` on imports unless resolving a name clash. +- No new TODO/FIXME comments in maintained Dart or workflow code. +- Export new public API from `lib/llamadart.dart` with explicit parameter, + return and field types (locals may infer), and test it; implementation stays + in `lib/src/`. +- Throw the `LlamaException` hierarchy (`lib/src/core/exceptions.dart`). An + unsupported platform or option combination throws `LlamaUnsupportedException` + (or a typed subtype) naming the missing capability, the platform/runtime + condition and, where known, the required version, or it is explicitly + disabled and documented. It never reports success. +- When behavior depends on runtime assets, platform, browser APIs or native + features, prefer an explicit capability probe to a structural or interface + check, and test both the supported and the unsupported/version-skew path. +- If `test/unit/test_structure/mirrored_unit_structure_test.dart` demands a test + for a file with no behavior (bare enum, marker interface, `external` interop + only), add the file to its `behaviorlessSources`. Never assert something true + by construction; assert wire values where they are consumed + (`test/README.md`). +- Generated files that should not count toward coverage carry `// coverage:ignore-file`. -- Close ports, streams, and controllers in test cleanup. - -## Architecture Rules - -- Zero-patch strategy: do not patch upstream native or web bridge sources in - this repository. -- Native build/source ownership lives in `llamadart-native`. -- Web bridge source/build ownership lives in `llama-web-bridge`. -- Web bridge runtime asset publishing lives in `llama-web-bridge-assets`. -- Keep local native integration focused on hook/config/bindings consumption. -- Keep local web integration focused on bridge tag pinning, fetch flow, and - runtime wiring. -- Prefer explicit capability probes over structural/interface checks when - behavior depends on runtime assets, platform support, browser APIs, or native - feature availability. -- Keep README, website docs/support matrix, examples, and changelog aligned with - any public capability or platform-support change. - -## Web And WebGPU - -WebGPU bridge features are versioned runtime capabilities. When changing bridge -behavior, verify the pinned asset tag/manifest, direct bridge calls, worker -path, Dart interop wrapper, public engine API, docs, and examples together. -`tool/testing/check_webgpu_bridge_tag.dart` enforces the tag half of that: every -site pinning the tag must match the default in -`scripts/fetch_webgpu_bridge_assets.sh`. Capability floors (`v0.1.30+`) and -changelog entries keep their own values and are not checked. - -Flutter Web builds exclude the gitignored generated bridge assets. Deployment -and real-model Web E2E flows must use `scripts/build_chat_app_web.sh`, which -builds the app, stages the pinned assets into the final output, and validates -the runtime files and checksums. Use -`scripts/validate_chat_app_web_build.sh` when intentionally reusing a build. - -The required `Web Chat Contract` CI job covers chat-app VM/Chrome tests, the -deployable artifact, deterministic UI behavior, and a real worker/WASM/GGUF -smoke. Keep it independent of large or availability-sensitive remote models; -full Gemma 4 GGUF and LiteRT-LM Web smokes remain targeted validation. - -Document browser durability precisely. Web bridge filesystem paths may be -virtual or in-memory unless the active bridge documents durable backing storage; -durable browser storage can require app-level export/import outside Dart file -helpers. - -Unsupported platform or option combinations must fail loudly with typed, -actionable errors or be explicitly disabled/documented. - -LiteRT-LM Web streams directly and rejects native worker batching controls. -Keep `streamBatchTokenThreshold` and `streamBatchByteThreshold` at their -defaults on Web, and run chat-app parameter tests on both VM and Chrome when -changing LiteRT-LM generation settings. - -## Multi-Repo Ownership - -Many maintainer checkouts keep sibling repos one level above this repo: - -```text -../llamadart -../llamadart-native -../llama-web-bridge -../llama-web-bridge-assets -``` - -Verify sibling paths before operating on them. Cross-repo runtime changes should -flow in ownership order: - -1. Change the owning repo (`llamadart-native` or `llama-web-bridge`). -2. Commit and push there. -3. Publish/update owning artifacts. -4. Update pins, tags, hooks, docs, and tests in `llamadart`. -5. Run `dart analyze` and relevant tests here before the final commit. - -## Native And Web Asset Sync - -Apple llama.cpp process lookup requires the resolved companion's package -identity and SwiftPM pin to match the core native pin. Preserve this guard and -its metadata cache dependencies when changing sync or hook behavior; declared -version constraints and core native overrides are not ABI evidence. Unverified -local companion `Artifacts` overrides must fail closed. -Changes to the companion SwiftPM implementation require reviewing/updating the -hook's normalized manifest template contract; tag/checksum-only syncs do not. - -Prefer the repository workflow for native version and binding updates: -`.github/workflows/sync_native_bindings.yml`. - -Stable `llamadart-native` distribution tags use strict -`vMAJOR.MINOR.PATCH`. A wrapper-only rebuild of upstream `vM.m.p` uses -`vM.m.p-N`, preserving the exact upstream prefix; native release ordering treats -`-N` as a forward wrapper sequence despite generic SemVer prerelease ordering. -New wrapper and nightly releases are GitHub prereleases and must be selected -explicitly. Historical `bNNNN` and `bNNNN-llamadart.N` artifacts may retain -older `prerelease=false` metadata, but remain explicit compatibility inputs; -`latest` accepts only unsuffixed `vMAJOR.MINOR.PATCH` regardless of GitHub -metadata. Build-hook overrides must always name an explicit tag; only maintainer -synchronization and header/binding regeneration accept `latest`. Nightly cores -use canonical decimal spelling (`b0` or a nonzero first digit), and rebuild -counters start at 1 without leading zeros. -New nightly wrapper rebuilds use `bNNNN-N`; existing -`bNNNN-llamadart.N` artifacts remain explicit consumption-only compatibility -inputs. New stable or nightly wrapper forms require manifests containing both -`native_release_tag` and the legacy `tag` alias. Stable syncs must validate -manifest provenance, hook contract, bundle coverage, checksums, upstream/native -tag separation, and forward version movement before changing the default pin. -The manual workflow requires its explicit `allow_nightly_channel` input before -moving a stable pin back to the nightly channel. - -For local native regeneration: - -```bash -tool/native/sync_native_headers_and_bindings.sh --tag latest -``` - -For local WebGPU bridge asset refreshes in `example/chat_app/web`: - -```bash -WEBGPU_BRIDGE_ASSETS_TAG= ./scripts/fetch_webgpu_bridge_assets.sh -``` - -See `website/docs/maintainers/native-and-web-sync.md` for the full maintainer -procedure. - -## Changelog And Releases - -- Never add unreleased work to an already-published version section in - `CHANGELOG.md` or `website/docs/changelog/recent-releases.md`. -- If the latest section is a concrete released version, create `## Unreleased` - above it for new PR entries. -- Only move `Unreleased` entries into a numbered version section during an - explicit release/version-bump task. -- Keep changelog entries concise and user-facing: one short bullet per - independently useful change; put implementation details, validation, and - migration notes in PRs or maintainer docs instead. -- Release prep PRs must not publish by themselves. After merge, - `release_on_prep_merge.yml` is the publishing approval boundary. -- Do not manually push companion or core release tags unless release automation - is disabled, blocked, or being repaired. -- Keep release-sensitive paths covered by `.github/CODEOWNERS`; enforcement - depends on GitHub branch protection or rulesets. -- Never log `RELEASE_AUTOMATION_TOKEN` or remotes/URLs derived from credentials. - -Before release prep or current install-snippet changes, run: - -```bash -dart run tool/testing/verify_release_docs_versions.dart -dart run tool/testing/verify_release_docs_versions.dart --release-prep -``` - -The default run reports a companion whose `Package.swift` pin is still only in -its `## Unreleased` section as a pending bump; `--release-prep` fails on it. -Resolve every pending bump in the release-prep PR, because the post-merge -workflow skips a companion version that already exists on pub.dev. - -After release automation runs, verify the release workflow, package publication, -GitHub Release, docs cut, docs pages deployment, and docs version selector -before calling the release complete. Use -`website/docs/maintainers/release-workflow.md` for the detailed checklist. - -## PR Readiness - -Treat `main` as production-ready. Before opening or updating a non-trivial PR, -make sure the PR template can honestly state: - -- User-facing scope and explicit out-of-scope behavior. -- Platform matrix rows and validation evidence. -- Unsupported combinations fail loudly or are clearly documented. -- README, website docs, examples, support matrices, and changelog match public - behavior. -- Regression coverage covers the issue plus important negative/version-skew - paths where applicable. -- Logs, cache keys, metadata, errors, and snapshots avoid credentials, bearer - tokens, signed URLs, and raw secret-bearing paths. -- Useful non-blocking follow-ups are tracked in GitHub Issues before merge. - -Before merging, reply to and resolve every review thread. For comments that are -outdated, superseded, or intentionally non-actionable, record the concrete -rationale in the thread before resolving it; classification alone is not a -substitute for closing the thread. Do not merge while any review thread remains -unresolved. - -Treat parser, grammar, streaming-protocol, backend/runtime-routing, capability- -probe, artifact-consumer, release-automation, and regression-policy changes as -high risk. Before mark-ready, run a blocking-only review pass that is independent -from the implementation task and reviews the exact PR head against the current -base. Use an independent operator-owned or fresh Codex adversarial audit identity -(the standalone qa profile is retired and prohibited; author self-approval is -rejected). -Classify changed paths with `tool/testing/classify_high_risk_changes.dart`. -Evaluate pre-merge readiness evidence with `tool/testing/high_risk_readiness.dart` -against `tool/testing/high_risk_readiness_evidence.schema.json`, supplying the -repository, PR, author, exact head, and exact base from an independent source. -The evaluator derives the rename-aware changed-file inventory from Git; it does -not accept a caller-supplied inventory. Inspect production call sites and -require issue-specific positive and negative tests that fail if the relevant -branch is deleted, bypassed, or miswired. -Record zero known PR-caused P1 regressions and zero unresolved review threads -in the PR's high-risk block and evidence payload. -For core-patch release metadata only, use the bounded -`release-metadata-verification` evidence route described in -`doc/high_risk_pre_merge_readiness.md`. It checks exact Git blobs and existing -release tests; do not fabricate changed tests or reclassify the PR as standard. - -Structured-output evidence v2 separates input rendering/history, output -parsing/streaming, and grammar/schema enforcement. Require the applicable axes; -unknown or mixed effects stay conservative. The same independent exact-pair -review must justify exclusions with inspected production callsites. Changed and -existing tests both need production reachability and causal before-fix or -mutation failure evidence; changing a test filename is not proof. See -`doc/high_risk_pre_merge_readiness.md` for the axis mapping and schema migration. -Use the closest affected-family model or artifact. If exact weights are -unavailable, name every unavailable family and use primary upstream emissions -plus durable fixtures; an unrelated representative model is pipeline-only evidence. - -Post-merge QA remains mandatory, but it must not be the first adversarial pass. -If it finds a PR-caused P1, stop lower-priority merge work, file a causally -accurate issue, and prepare one cohesive recovery before resuming feature work. -Security-sensitive required-check publication and repository settings are -governed by `doc/high_risk_pre_merge_readiness.md`; checkout-local verification -can return only an unverified-prerequisites result and fails closed until the -dedicated GitHub App, authenticated auditor boundary, and conditional ruleset -enforcement are separately implemented and configured. The default-branch -workflow is a non-required advisory: it warns on high-risk paths instead of -failing, so high-risk readiness must be established by the repository-local -review and evidence. Do not configure it as a required readiness check. - -For docs-only PRs, state that runtime behavior is unchanged and list docs -validation. If implementation scope changed, reduce and state the scope rather -than merging incomplete behavior. - -### PR Branch Integrity and Mutation Contract - -- Repository-local mutations to open PR branches must enforce an expected-head - compare-and-swap (CAS) plus fast-forward-only contract (`tool/git/safe_pr_head_update.dart`). - Immediately before mutation, the remote head must match the caller's expected - head, and the proposed new head must equal that head for an idempotent retry or - descend from it. -- The helper uses a normal push guarded by the remote OID advertised to a - one-shot `pre-push` hook. Stale ancestor restorations, force options, - `+refspec` updates, and blind overwrites are strictly rejected. Read the exact - target ref back before claiming success. -- Consequence of unexpected transitions (#434 incident): Any unexpected - branch-head transition invalidates prior CI runs, review approvals, and - independent QA evidence recorded against that head. If the remote head moves, - all validation and QA blocks must be re-executed against the exact new head. -- Residual governance boundaries: checkout-local tooling governs only writers - that invoke it. It cannot govern GitHub web UI updates, Dependabot, installed - apps, or other server-side writers. The repository relies on branch protection - and rulesets to block force pushes and enforce linear history, status checks, - and CODEOWNERS reviews across those paths. See the bounded writer inventory - and incident evidence in `doc/pr_branch_writer_inventory.md`. - -## AGENTS.md Maintenance - -- Update this file when a task reveals a durable workflow rule, command, - ownership boundary, release gate, or validation expectation. -- Remove or replace outdated guidance when repo workflows change. -- Keep this file concise. Prefer pointers to `doc/testing_matrix.md` and - maintainer website docs over copying long checklists here. -- Avoid session logs, one-off incident details, and duplicated advice. +- Tests close the ports, streams and controllers they open. +- A public capability or platform-support change updates README, website + docs/support matrix, examples and changelog in the same PR. +- Keep credentials, tokens, signed URLs and secret-bearing paths out of logs, + errors, cache keys, metadata and snapshots. + +## Ownership + +Never patch upstream native or web bridge sources here. Owners: +`llamadart-native` (llama.cpp runtime), `litert-lm-native` (LiteRT-LM runtime), +`llama-web-bridge` (web bridge), `llama-web-bridge-assets` (published assets). +This repo only consumes them: native hook/config/bindings, and bridge tag +pinning, fetch and runtime wiring +(`website/docs/maintainers/runtime-ownership.md`). Checkouts often keep them as +siblings in `..`; verify the path first. Change and release the owning repo +first, then update pins, hooks, docs and tests here. + +- A WebGPU bridge change is verified across the pinned tag/manifest, direct and + worker paths, Dart interop, public API, docs and examples together; capability + floors and changelog entries keep their own versions. Bridge behavior and + browser-storage durability: `doc/webgpu_bridge.md`. +- Native or web pins, bindings, sync scripts, `hook/`, or companion SwiftPM + changes: `website/docs/maintainers/native-and-web-sync.md`. Native release + tags: `latest` accepts only unsuffixed `vMAJOR.MINOR.PATCH`; new nightly + wrapper rebuilds use `bNNNN-N`, while `bNNNN-llamadart.N` is consume-only; + nightly cores use canonical decimal spelling, and rebuild counters start at 1 + without leading zeros. + +## Changelog and releases + +- New entries go under `## Unreleased` in `CHANGELOG.md` and + `website/docs/changelog/recent-releases.md`; add that heading if the top + section is a released version. Never add to a released section. Move + `Unreleased` into a version only in an explicit release task. +- One short user-facing bullet per change; implementation detail and migration + notes go in the PR or maintainer docs. +- Never push release tags by hand unless release automation is disabled, + blocked or being repaired. Give each new release-sensitive path a + `.github/CODEOWNERS` entry. Release work follows + `website/docs/maintainers/release-workflow.md`. + +## Pull requests + +- `main` stays production-ready. The PR body follows + `.github/pull_request_template.md`; fill every section or mark it N/A with a + reason. +- Update an open PR branch only through `tool/git/safe_pr_head_update.dart` + (expected-head CAS, fast-forward only); writers it cannot govern are in + `doc/pr_branch_writer_inventory.md`. Whenever the head moves, earlier CI runs, + approvals, audits and matrix evidence no longer count: rerun or re-request + them against the new head and update the PR body. +- Before merge, reply to every review thread, with concrete rationale when not + acting on it, and resolve it. +- Classify every PR with + `git diff --name-only --no-renames ...HEAD | dart run tool/testing/classify_high_risk_changes.dart`. + Before a high-risk PR is marked ready, an independent auditor runs a + blocking-only adversarial review of the exact head against the current base. + The auditor is an operator or a fresh agent session that took no part in the + implementation, never the PR author. + Build the evidence per `doc/high_risk_pre_merge_readiness.md` and check it + with `tool/testing/high_risk_readiness.dart` against + `tool/testing/high_risk_readiness_evidence.schema.json`; a valid local run + exits 2 (`unverifiedPrerequisites`). Mark ready only with + zero known PR-caused P1 regressions and zero unresolved review threads, + recorded in the PR's high-risk block and the evidence. +- Use the closest affected-family model or artifact; if its weights are + unavailable, name each unavailable family and use primary upstream emissions + plus durable fixtures. An unrelated model is pipeline-only evidence. +- Post-merge QA is still required but is never the first adversarial pass. If + it finds a PR-caused P1, stop lower-priority merge work, file a causally + accurate issue, and prepare one cohesive recovery before resuming feature + work. + +## This file + +When a task reveals a durable rule that changes agent behavior and that no +tool, test or linked doc covers, add it here. Point to docs instead of copying +them, delete stale rules, and leave out incident history. Tests pin some +phrases here, and docs-only CI does not run them: after editing, run +`dart test -p vm test/unit/tooling/`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1540ee8f0..0065710e2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -310,8 +310,9 @@ Changes touching parsers, grammars, streaming, backend routing, capabilities, artifact consumers, release automation, or regression policy require an independent blocking review pass bound to the exact candidate head SHA and base SHA (`tool/testing/high_risk_readiness.dart`, `doc/high_risk_pre_merge_readiness.md`). -Self-approval is prohibited, and the standalone `qa` profile is retired in favor -of an independent operator-owned or fresh Codex adversarial audit identity. +The auditor runs a blocking-only adversarial review and is an operator or a +fresh agent session that took no part in the implementation, never the PR +author or the retired standalone `qa` identity. The local evaluator derives the changed-file inventory from Git and never emits operational readiness. Until the separately reviewed GitHub App, auditor authentication, and conditional ruleset boundary exist, a valid high-risk run diff --git a/doc/high_risk_pre_merge_readiness.md b/doc/high_risk_pre_merge_readiness.md index 0a643934d..1f0c4cf43 100644 --- a/doc/high_risk_pre_merge_readiness.md +++ b/doc/high_risk_pre_merge_readiness.md @@ -170,7 +170,10 @@ authentication/publication/ruleset boundaries below are unchanged. High-risk evidence requires an audit whose head and base exactly match the PR context, whose decision is `accepted`, and whose known PR-caused P1 regression and unresolved-thread counts are both zero. The auditor must differ from the PR -author and must not use a retired `qa` identity. +author and must not use a retired `qa` identity. A Codex session records +`audit_kind: codex-adversarial`; an operator or any other fresh agent session +records `operator-owned`. `auditor_identity` and `summary` name the reviewer +that actually ran. These repository-local checks establish internal consistency only. They do not authenticate the auditor. That requires the missing external boundary described diff --git a/doc/testing_matrix.md b/doc/testing_matrix.md index 8ab3d065c..ed69cf705 100644 --- a/doc/testing_matrix.md +++ b/doc/testing_matrix.md @@ -77,12 +77,12 @@ git diff --name-only --no-renames "$PR_BASE_REF"...HEAD | \ dart run tool/testing/test_matrix.dart --tier high-risk ``` -Before mark-ready, assign an independent blocking review pass that did not -implement the change, using an independent operator-owned or fresh Codex -adversarial audit identity (the standalone qa profile is retired; author -self-approval is rejected). It must review the exact head against the current -base, inspect the actual production call sites, and verify positive and negative -tests that fail if the relevant branch is deleted, bypassed, or miswired. +Before mark-ready, assign an independent blocking-only adversarial review pass: +an operator, or a fresh agent session that took no part in the +implementation, never the PR author or the retired `qa` identity. It must +review the exact head against the current base, inspect the actual production +call sites, and verify positive and negative tests that fail if the relevant +branch is deleted, bypassed, or miswired. Validate the evidence payload with `tool/testing/high_risk_readiness.dart`, supplying repository, PR, author, exact head, and exact base values from an independent source as documented in `doc/high_risk_pre_merge_readiness.md`. From 1251a2f9668dc5c3ac21bfea89b9854e7c7b3893 Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 13:14:03 -0400 Subject: [PATCH 2/6] docs: restore naming, Dartdoc and conditional-import rules in AGENTS.md --- AGENTS.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index dbc239341..2b54f57c4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,8 +20,9 @@ dart test -p chrome --exclude-tags local-only - Use the Flutter SDK in `.flutter-version`; other SDKs format differently from CI. A pin bump is its own PR that updates `.flutter-version`, every workflow `flutter-version:`, and the reformat it causes. -- The root gates skip `packages/`: companion packages keep their own - dependency, analyze, test, SwiftPM and publish-validation lanes (`ci.yml`). +- Root preparation, analysis and tests skip `packages/` (root format does + not): companion packages keep their own dependency, analyze, test, SwiftPM + and publish-validation lanes (`ci.yml`). - Chat app changes: `(cd example/chat_app && flutter test)`, plus `flutter test --platform chrome test/chat_generation_service_test.dart` there for Web-only paths such as LiteRT-LM generation settings. @@ -50,10 +51,13 @@ dart test -p chrome --exclude-tags local-only only when correctness or maintainability needs it. - Order imports SDK, package, then relative, with a blank line between groups. Avoid `show`/`hide` on imports unless resolving a name clash. +- Names: `PascalCase` types; `lowerCamelCase` members, variables and + constants; `snake_case` files and directories. - No new TODO/FIXME comments in maintained Dart or workflow code. - Export new public API from `lib/llamadart.dart` with explicit parameter, - return and field types (locals may infer), and test it; implementation stays - in `lib/src/`. + return and field types (locals may infer), useful `///` Dartdoc, and tests; + implementation stays in `lib/src/`. Select platform-specific backends with + conditional imports or exports. - Throw the `LlamaException` hierarchy (`lib/src/core/exceptions.dart`). An unsupported platform or option combination throws `LlamaUnsupportedException` (or a typed subtype) naming the missing capability, the platform/runtime From 8e903d5cd2c1a2e65590d4d6bc28c78430845ae0 Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 13:35:01 -0400 Subject: [PATCH 3/6] docs: keep the two-mode release-docs check and pre-commit validation in AGENTS.md --- AGENTS.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 2b54f57c4..696da7a62 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -88,7 +88,8 @@ This repo only consumes them: native hook/config/bindings, and bridge tag pinning, fetch and runtime wiring (`website/docs/maintainers/runtime-ownership.md`). Checkouts often keep them as siblings in `..`; verify the path first. Change and release the owning repo -first, then update pins, hooks, docs and tests here. +first, then update pins, hooks, docs and tests here, and run analyze and the +relevant tests before the final commit. - A WebGPU bridge change is verified across the pinned tag/manifest, direct and worker paths, Dart interop, public API, docs and examples together; capability @@ -109,6 +110,9 @@ first, then update pins, hooks, docs and tests here. `Unreleased` into a version only in an explicit release task. - One short user-facing bullet per change; implementation detail and migration notes go in the PR or maintainer docs. +- Before release prep or a change to current install snippets, run + `dart run tool/testing/verify_release_docs_versions.dart`, then again with + `--release-prep`; resolve pending companion bumps in the release-prep PR. - Never push release tags by hand unless release automation is disabled, blocked or being repaired. Give each new release-sensitive path a `.github/CODEOWNERS` entry. Release work follows From b7519af2c862f708f3060e11f51df9f158a216c6 Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 15:24:58 -0400 Subject: [PATCH 4/6] docs: restore the source-wide TODO rule and the owner-repo push step --- AGENTS.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 696da7a62..7f4437488 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -53,7 +53,7 @@ dart test -p chrome --exclude-tags local-only Avoid `show`/`hide` on imports unless resolving a name clash. - Names: `PascalCase` types; `lowerCamelCase` members, variables and constants; `snake_case` files and directories. -- No new TODO/FIXME comments in maintained Dart or workflow code. +- No new TODO/FIXME comments in maintained source or workflow code. - Export new public API from `lib/llamadart.dart` with explicit parameter, return and field types (locals may infer), useful `///` Dartdoc, and tests; implementation stays in `lib/src/`. Select platform-specific backends with @@ -87,9 +87,9 @@ Never patch upstream native or web bridge sources here. Owners: This repo only consumes them: native hook/config/bindings, and bridge tag pinning, fetch and runtime wiring (`website/docs/maintainers/runtime-ownership.md`). Checkouts often keep them as -siblings in `..`; verify the path first. Change and release the owning repo -first, then update pins, hooks, docs and tests here, and run analyze and the -relevant tests before the final commit. +siblings in `..`; verify the path first. Commit, push and release the change +in the owning repo first, then update pins, hooks, docs and tests here, and run +analyze and the relevant tests before the final commit. - A WebGPU bridge change is verified across the pinned tag/manifest, direct and worker paths, Dart interop, public API, docs and examples together; capability From a214f62ba46a56286336b79c1073c5be75ba54da Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 15:45:57 -0400 Subject: [PATCH 5/6] docs: keep the obvious-type condition on local inference --- AGENTS.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7f4437488..22aad7324 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,9 +55,9 @@ dart test -p chrome --exclude-tags local-only constants; `snake_case` files and directories. - No new TODO/FIXME comments in maintained source or workflow code. - Export new public API from `lib/llamadart.dart` with explicit parameter, - return and field types (locals may infer), useful `///` Dartdoc, and tests; - implementation stays in `lib/src/`. Select platform-specific backends with - conditional imports or exports. + return and field types (locals may infer when the type is obvious), useful + `///` Dartdoc, and tests; implementation stays in `lib/src/`. Select + platform-specific backends with conditional imports or exports. - Throw the `LlamaException` hierarchy (`lib/src/core/exceptions.dart`). An unsupported platform or option combination throws `LlamaUnsupportedException` (or a typed subtype) naming the missing capability, the platform/runtime From 331217e15ad1b506f25f292f8831ceee59287e61 Mon Sep 17 00:00:00 2001 From: Jhin Lee Date: Wed, 23 Sep 2026 16:40:48 -0400 Subject: [PATCH 6/6] docs: require explicit types on all public API again --- AGENTS.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 22aad7324..920f61057 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,10 +54,10 @@ dart test -p chrome --exclude-tags local-only - Names: `PascalCase` types; `lowerCamelCase` members, variables and constants; `snake_case` files and directories. - No new TODO/FIXME comments in maintained source or workflow code. -- Export new public API from `lib/llamadart.dart` with explicit parameter, - return and field types (locals may infer when the type is obvious), useful - `///` Dartdoc, and tests; implementation stays in `lib/src/`. Select - platform-specific backends with conditional imports or exports. +- Public API has explicit parameter, return and field types (locals may infer + when the type is obvious). Export new public API from `lib/llamadart.dart` + with useful `///` Dartdoc and tests; implementation stays in `lib/src/`. + Select platform-specific backends with conditional imports or exports. - Throw the `LlamaException` hierarchy (`lib/src/core/exceptions.dart`). An unsupported platform or option combination throws `LlamaUnsupportedException` (or a typed subtype) naming the missing capability, the platform/runtime