Skip to content

About

Portable code-quality toolkit: atomic ops SDK, plan runner, merge-queue policy templates (work in progress)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

6,275 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

@camerontaylor/cq-toolkit is a portable code-quality toolkit: a TypeScript SDK of atomic code-quality operations, a deterministic plan runner that composes those operations into reproducible execution plans, and adoptable PR and doctrine policy templates that other repositories can take in whole or in part. It is greenfield and self-hosting — the toolkit's own quality gates run on the toolkit itself — and it is a work in progress. See the documentation index for the canonical reading set.

The SDK is the primary interface; the CLI is a secondary interface over the same ops and plans, and no logic lives only in the CLI. Doctrine: see policy/DOCTRINE.md. CodeRabbit is advisory; the merge bar is green static and denylist on the exact PR head, one independent review of that head, and no unaddressed review comments. The independent review is an agent PASS posted with cq-review-post, a human’s PASS, or a crq review with the crq-findings marker, posted by crq under the queue host’s login, whose reviewed range ends at the head and whose findings are all addressed. A clean crq run posts nothing and does not count. Every thread and finding, including Codex, CodeRabbit, bot and minor findings, must be fixed or answered with a reason, and every thread resolved before merge. See docs/coderabbit-review.md. Generated per-op reference: see docs/ops/ (regenerate with pnpm run gen:op-docs).

Install

The first npm release is 0.2.0 (no 1.x will be published). Until @camerontaylor/cq-toolkit@0.2.0 is on the registry — the 0.0.0 version there is a name reservation whose payload is LICENSE + README only — install and build from source:

git clone https://github.com/camerontaylor/cq-toolkit
cd cq-toolkit
pnpm install --frozen-lockfile
pnpm run build

Once 0.2.0 is published, the install is the usual one:

npm install @camerontaylor/cq-toolkit

The package is ESM-only ("type": "module"). Node 24 is the supported/tested runtime floor, matching the CI matrix. Workspace inspection uses fs.promises.glob with pattern exclusions, stable since Node 22.17. On Node 21 or earlier, importing the package fails at link time because the root export reaches the static glob import.

SDK first

Every op is reachable from the package root, and the plan runner composes ops through the frozen kernel contracts. Discover the registry, then invoke an op through its entry:

import { getOp, listOps } from '@camerontaylor/cq-toolkit';

const names = (await listOps()).map((entry) => entry.name); // every registered op

const entry = await getOp('analyze.run');
if (entry === undefined) throw new Error('op not registered');

const op = await entry.importer(); // lazy: no op module loads before this
const result = await op({ dir: '.' }); // OpResult — a status-tagged union
if (result.status === 'ok') {
  // probe → collect → cluster → report: reads ./cq.json (`cq analyze.init` writes one)
  console.log(result.value.failureCount, result.value.clusters, result.value.markdownPath);
}

The same op is composable directly — makeAnalyzeRun(subprocessRunCheck) — with your own check runner injected; analyze.fix previews a remediation for one cluster and applies it only through an attended approval (makeAttendedApprovalAuthority, ADR-0003).

The runner is exported too: runPlan(plan, opts, registry, gov?) returns a serializable RunReport (per-job outcomes, usage rollup, honest-stop counts); its third argument is an OpRegistryView, and budget governance rides the optional fourth — a Governance handle ({ governor, signal?, attended?, optIn? }); inject a clock through createGovernor(config, clock?) when constructing its governor. Caps in opts without a handle throw. See src/kernel/README.md for the composition contract and scripts/smoke-run-plan.mjs for a worked governed run.

CLI (secondary)

One subcommand per registry entry, plus one per shipped plan, plus the built-in run-plan. Flags map onto the op's schema keys; output honors invariant I1 (stdout = one JSON artifact, stderr = cq: narration, machine mode with --json):

For direct ops, stdout = one JSON artifact: the OpResult, optionally with a report field carrying the op's run report on any status. Reports must be JSON-lossless; exit codes follow the result's status.

cq analyze.init --dir=.                   # writes cq.json
cq analyze.run --dir=. --json             # probe → collect → cluster → report
cq analyze.fix --dir=. --clusterId=<id>   # preview
cq analyze.fix --dir=. --clusterId=<id> --apply   # attended apply + recheck

analyze.init detects tsc (a tsconfig.json) and ESLint (a config file) checks. analyze.run writes a Markdown report and its JSON sidecar under .cq/analysis/ and returns the clusters. analyze.fix needs the remediation: an ast-grep rule (--rule='{"id":…,"language":"TypeScript","rule":{…},"fix":"…"}'), or --model='{"provider":…,"model":…}' to ask an agent for a patch; --rule with --files='["src/a.ts"]' works without a cluster. Without --apply no project file is written (the report directory .cq/analysis/ may be created, and a --model preview stores its proposal there). An agent patch is applied only in a second call that names its digest (--previewDigest=<digest> --apply). With --apply the work tree must be a committed, clean git tree (only .cq/ may differ — commit or stash anything else); the edit is applied exactly as previewed (pass --previewDigest=<digest> from the preview to pin it), and the configured checks are re-run to report the failure count before and after. The commands in cq.json run with your privileges, like package.json scripts (as argv, never through a shell), so review the file like code.

cq.json uses schemaVersion: 2. Upgrade a v1 file with cq analyze.init --dir=. --migrate; this preserves its checks and argv and changes only the version. Migration validates the result and replaces the file atomically; running --migrate on a valid v2 file is a no-op. Reading v1 otherwise fails with the migration hint. --force regenerates checks from detection; --migrate upgrades the existing file. Policy options cannot accompany --migrate; a usage error names any supplied policy keys.

The v2 config also accepts packages (name, path), check kind (tests or static), check relatedArgs, roles (worker, rescue, reviewer, each with provider and model), budgets, review, and escalation. Unknown keys fail, including nested keys. relatedArgs is the complete replacement argument vector for a related check: each standalone {files} argument expands into individual file arguments; omitted means append files to the configured command's args. The command executable stays the same.

cq fix.run --source=static groups diagnostics by repository-relative file and rule. --checks=tsc selects a configured adapter family; --checks='["tsc","eslint"]' selects several. Aliases are tsc, eslint, and vitest; unknown or unavailable aliases refuse. Omit --checks to select every check of the requested source. Static ESLint requires an explicit relatedArgs template with standalone {files} before any worker runs. For eslint . --format=json, use ["{files}","--format=json"]; for node scripts/lint.cjs ., use ["scripts/lint.cjs","{files}"]. The replacement must name only task files, while broader validation retains the full configured argv. Existing config remains parseable without the template; invoking static ESLint reports an actionable refusal. Templates retaining literal . or ./ operands also refuse before worker dispatch. Other wrapper arguments and option semantics remain trusted configuration: the engine does not parse arbitrary command grammars or prove that every configured target is scoped. Static tsc always runs the full project check, allowing existing sibling clusters while requiring the assigned cluster to disappear and failure fingerprints to improve. When final related and broader commands are identical, both gates share one fresh base-owned staged-tree probe. The stateless feedback invocation still runs per task.

fix.run retries a task with roles.worker up to escalation.workerAttempts (default 2), then uses roles.rescue. A broader regression routes directly to rescue. Each round uses separate stateless edit and related-check feedback invocations; feedback is mandatory even when it only confirms the edit. Failed edits are pinned under unique refs and restored before retrying. Protected-path, tamper, cancellation and budget refusals retain their gates; retries never bypass validation. Exhausted or unavailable rescue sends that task to human review while siblings continue committing verified fixes.

Every fresh run starts with initialFixedFailures = 0 and the configured rescueBase (default 20). It earns floor(distinct newly verified failure reductions / rescuePerFixed) additional rescue attempts (default one per 10). Unresolved baseline failures earn nothing; repeated sibling rows and feedback calls earn nothing. Fingerprint occurrence reductions are credited once, only after commit verification. The pure SDK allocator can accept an explicit initial fixed-failure count for deterministic allocation math; fix.run has no historical state or input for that count. Run and per-package rescue reservations are shared atomically across units; the package cap defaults to 10. Run tokens and wall time are shared across every tier. Token limits remain landing gates for concurrently in-flight calls: their usage is reported when they settle, and no new invocation starts after exhaustion. Explicit parent cancellation prevents publication; a local budget stop may still publish previously verified commits with --push. Reports retain decisions, reasons, model identity, per-invocation tokens, wall time and available cost with its modeled/billed basis. Unknown cost stays absent. Full JSON evidence is persisted on every exit.

Every planned unit of work appears in the report's planned rows with an honest outcome: fixed, unfixed, escalated-to-human, not-started, cancelled or exhausted. A unit that stopped before deriving its tasks is one branchless row naming the unit and the stop reason (its task list was never known, so none is invented). The failed evidence keeps every retained failed-attempt branch and sha; a task fixed after earlier failed rounds keeps that history under its fixed row. Work with no retained artifact stays branchless — no commit URL is fabricated. Explicit counters carry the denominators (planned, attempted, verified fixed, skipped by an earlier verified sibling, unresolved, plus rescue attempts and verified rescue fixes as separate counts, never a ratio). The opt-in PR summary's Unfixed section names every outstanding unit, branch-bearing rows with their compare link or local branch, branchless rows with their stop reason only, and stays bounded and scan-gated.

analyze.init writes only the version and detected checks unless optional fields are explicitly supplied as flags (for example --budgets='{"run":{"maxTokens":100000}}'). No model is selected by the generated starter config. New agentic compositions use the pure SDK helper resolveCqConfigDefaults(config, { packageName, ...flags }), with the actual package.json name supplied by the caller. Flags override config field by field; packages default to that name at . and check kinds default by adapter (vitest-json means tests, the other adapters mean static).

Effective defaults are run/step wall budgets of 20 minutes, a unit wall budget of 60 minutes, 3 review rounds, blocking critical/major findings, agent high→major and medium→minor, CodeRabbit then agent reviewers, address concurrency 2, bulk concurrency 3, review session input of 60,000 characters and 15 reviewer steps, size tiers of 200 changed lines/5 files (small) and 800 changed lines/20 files (medium, never below small), no review exclusions, prior findings included on re-review, and escalation worker attempts 2, rescue base 20, one rescue earned per 10 fixed failures, per-package rescue cap 10. There is no token budget default: requireAgenticTokenBudget(policy) returns a needs-human result naming budgets.run.maxTokens when absent. Bulk query defaults are computed by the forge consumer from the current user and repository at run time.

Review addressing coordinates each PR and edited checkout with persistent local slot locks. Legacy registry files remain read-only hints; an incomplete checkout requires attended recovery. See the review operations guide for platform and writer-lifetime limits.

Partition review stops dispatching after two failures in completion order with no success between them; already-running sessions finish. With concurrency, the set of later sessions dispatched can therefore vary with provider timing. A deterministic lane or missing-delivery refusal stops dispatch immediately. Session results retain their own verdict when aggregate spend exceeds the run guard. Token reservations limit dispatch allowances, but cannot account for unsettled in-flight usage or prevent one driver step from overshooting. Retry requests and usage cover both attempts; coverage deliveries describe only the final attempt.

This repository's root config declares the compiler check as argv, and runs lint as an oxlint-json check (typed Oxlint with --format json). Tests are omitted until a config check can honour the test:narrow resource preflight and file selection. Consequently cq analyze / analyze.run --dir=. here runs the compiler and lint (plus the other static checks in cq.json); local development still follows pnpm test:narrow and the focused-check policy below. The root role choices are experiment configuration, with no token budget baked in.

cq --help                                        # global help
cq analyze.run --help                            # per-op input schema
cq run-plan --plan=plan.json --json              # a governed plan

Exit codes: 0 clean, 1 thrown/definitive failure, 2 usage error, 3 needs-human or budget-exhausted. An ok result can still carry a failing verdict in its value ({ "status": "ok", "value": { "verdict": "fail", … } }), which exits 0 by default; pass --fail-on-verdict to turn that into exit 1 (stdout is unchanged; only the exact string "fail" counts — a needs-human or failing verdict vocabulary is not mapped). In --json mode an error writes one { "error": { "kind", "subcommand", "message", "exitCode" } } object to stdout (stderr stays empty). CI gate: cq ratchet.checkRatchet … --json --fail-on-verdict. See src/cli/README.md for the full CLI contract.

Drivers

Drivers implement one seam (Driver.run(opInvocation)), so plans are transport-independent: each worker is a fresh isolated invocation, and vendor vocabulary never reaches the kernel or persisted data (I6, I10).

  • AiSdkDriver — in-process provider drivers over the AI SDK (anthropic, openai, zai, deepseek), keys read from the environment at call time.
  • ClaudeCodeDriver — runs the Claude Code substrate on the frozen seam with env-based model routing: the headless CLI over stdio (the primary transport) or the agent-SDK host (the optional sdk transport). The SDK package (@anthropic-ai/claude-agent-sdk) is an optional peer dependency: importing or building the toolkit never requires it, and the sdk transport loads it lazily at run() time. Install it only if you use that transport:
npm install @anthropic-ai/claude-agent-sdk
  • AcpDriver — the Agent Client Protocol lane for operator-installed vendor harness binaries; no vendor package is bundled.

See src/driver/README.md for the seam rules and the .github/workflows/install-matrix.yml workflow for the with-peer/without-peer proof.

Doctrine (I1–I11)

The eleven behavioral invariants are canonical in policy/DOCTRINE.md, each with its Rule, Why, and Enforcement. In brief:

# Invariant
I1 stdout is JSON, stderr is narration; exit codes 0 clean, 1 thrown, 2 arg error, 3 needs-human/budget-exhausted
I2 No-privileged-reviewer acceptance: an independent exact-head agent/human PASS or qualifying crq findings review; all comments addressed and threads resolved; truncated data fails closed
I3 Merge commits directly to main on exact-head CI and independent review — never squash, rewrite, or force-push
I4 Required checks never filter triggers; the only sanctioned skip is a job-level if:
I5 Baselines only tighten; a missing metrics summary is non-passing evidence, never a pass
I6 Every per-file/per-job worker is a fresh isolated invocation; context never leaks
I7 A reused clean worktree re-probes its baselines from scratch; baseline values are never cached
I8 Rescue and escalation policy live in the plan runner, never in a driver
I9 Fleet runs collect everything and never bail mid-fleet; budget stops are honest (budget-exhausted, never fabricated)
I10 The kernel stays vendor-neutral — no vendor SDK vocabulary in kernel types or persisted data
I11 GitHub-API facts are carried as named tests beside the ops that rely on them

Adopting the policy

The PR and doctrine templates under policy/templates/ are adoptable by other repositories. Target main, require static and denylist, obtain an independent review of that head, leave no review comment unaddressed, and merge with a merge commit. The independent review is an agent PASS posted with cq-review-post, a human’s PASS, or a crq review with the crq-findings marker, posted by crq under the queue host’s login, whose reviewed range ends at the head and whose findings are all addressed. A clean crq run posts nothing and does not count. CRQ also reviews merged PRs after merge. See merge policy and the adoption guide.

Self-hosting

Stage 1 reached: CI runs the toolkit from source. The from-source job in .github/workflows/ci.yml builds the package and drives a real governed plan through the built barrel (scripts/smoke-run-plan.mjs — two jobs, claude-code driver (CLI transport), journal + report + I1 output contract asserted), and the pack audit tarballs the package, asserts every shipped path hangs off the files allowlist, and scans the unpacked tree with gitleaks. The required static job regenerates the op reference and fails on drift, so docs/ops/ cannot diverge from the registry.

CI and self-hosted review jobs also require the host rg executable for the harness search tools. Ripgrep 11.0.0 is the runtime floor; CI installs the exact 15.2.0 pin through .github/actions/install-ripgrep from upstream release archives with per-OS/architecture SHA-256 verification. It is a host prerequisite, not an npm dependency; deployers running these tools outside the provided workflows must install rg 11.0.0 or newer.

Local verification

The repository uses pnpm (pinned by packageManager). Its pnpm-workspace.yaml turns on pnpm's global virtual store, so every worktree links one shared, already-materialized dependency graph. CI opts out with PNPM_CONFIG_VIRTUAL_STORE_TYPE=project.

Run pnpm run check:static for the TS7 compiler ratchet and typed Oxlint. pnpm run lint and pnpm run typecheck are compatibility aliases; run only one. Formatting is pnpm run format:check, the tests you need are the ones your diff affects (pnpm test:narrow, the only local test command). The full suite and the build are not local gates: pnpm run build (checked declaration emit) and pnpm run test / test:unit run in CI, and green required CI on the exact candidate SHA is the sole full-gate authority — see docs/focused-checks-contract.md. See the local static policy for tool pins, architecture conformance and the integrated-checker fallback.

Mechanical checks

pnpm run check is a composite of the whole-tree obligations (format check, static gate, the full suite, Knip). It is not a local verification route — its full-suite leg is CI's obligation, so it is CI-only. For local work use the focused set in docs/focused-checks-contract.md. lint and typecheck are compatibility aliases of check:static; run only one. For an inner loop, pass explicit owned files to pnpm lint:fast <file...> or pnpm fix <file...>. The latter applies safe lint fixes and formatting of those files only; it runs no static gate, so run pnpm run check:static for dependents. Build, smoke and denylist remain separate gates. See local static policy for pins, compiler fallback evidence, rule decisions and compatibility changes.

For analyze.run and analyze.fix, use a dedicated report directory for --out (default .cq/analysis). Its entire subtree is exempt from the attended clean-tree check; pointing it at tracked source directories widens that exemption and writes report artifacts alongside source files.

Check configurations also accept ruff-json (ruff check --output-format json .), knip-json (knip --reporter json, Knip 6) and oxlint-json (oxlint --format json, which emits a diagnostics[] envelope rather than ESLint's per-file array). Each adapter requires exit 0 with an empty report or exit 1 with findings; malformed output, operational exits, and contradictory exit/report combinations are indeterminate. oxlint-json also requires a completed nonempty file scan and accepts warning-only reports behind exit 0. Knip's JSON reporter does not retain issue severity, so findings are errors and warning-only reports behind exit 0 are indeterminate. Avoid --exit-zero and mutating fix flags.

About

Portable code-quality toolkit: atomic ops SDK, plan runner, merge-queue policy templates (work in progress)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages