diff --git a/apps/docs/src/components/CliPreview.astro b/apps/docs/src/components/CliPreview.astro index 29a76eeb..61796d31 100644 --- a/apps/docs/src/components/CliPreview.astro +++ b/apps/docs/src/components/CliPreview.astro @@ -22,6 +22,6 @@ ✓ Gate passed. Checkpoint saved. - ⎯ qwen3.6-27b · ctx ~12k/33k 36% · 2 turns · 8s · done · entire workspace + ⎯ deepseek-v4-flash · ctx ~12k/33k 36% · 2 turns · 8s · done · entire workspace diff --git a/apps/docs/src/content/docs/agent/delegation.mdx b/apps/docs/src/content/docs/agent/delegation.mdx index e2a59bf9..f3da8383 100644 --- a/apps/docs/src/content/docs/agent/delegation.mdx +++ b/apps/docs/src/content/docs/agent/delegation.mdx @@ -1,19 +1,19 @@ --- title: Delegate to subagents -description: "tsforge's orchestrator delegates focused, read-only investigation to specialist subagents on its own — you describe a task, it decides what to explore, research, or verify — and renders them as a live tree while they work." +description: "tsforge's orchestrator delegates focused, read-only investigation to specialist subagents on its own. You describe a task, it decides what to explore, research, or verify, and renders them as a live tree while they work." --- -You think in tasks, bugs, and features — never in subagents. So delegation in tsforge is **orchestrator-driven**: the model you're talking to decides, on its own, when a task needs focused investigation, hands it to a specialist subagent, and folds the findings back into its answer. You never name an agent, pass ids, or run a command for it. +You think in tasks, bugs, and features. Never in subagents. So delegation in tsforge is **orchestrator-driven**: the model you're talking to decides, on its own, when a task needs focused investigation, hands it to a specialist subagent, and folds the findings back into its answer. You never name an agent, pass ids, or run a command for it. -Ask it to "figure out how the gate decides a run is done and check there's no race in the scheduler," and it may spawn an `explore` agent for the first and a `verify` agent for the second — in parallel — while it keeps working. +Ask it to "figure out how the gate decides a run is done and check there's no race in the scheduler," and it may spawn an `explore` agent for the first and a `verify` agent for the second in parallel while it keeps working. ## Why delegate -A subagent runs in its **own context window** and returns only a concise, cited summary. That keeps the orchestrator's context small on exploration-heavy work (it doesn't have to read twenty files itself), and — when several run at once — shortens wall-clock. Subagents are **read-only**: they explore, research, and verify; only the orchestrator edits files. +A subagent runs in its **own context window** and returns only a concise, cited summary. That keeps the orchestrator's context small on exploration-heavy work (it doesn't have to read twenty files itself), and when several run at once, it shortens wall-clock. Subagents are **read-only**: they explore, research, and verify; only the orchestrator edits files. ## Built-in specialists -These ship in the binary — delegation works with zero configuration: +These ship in the binary. Delegation works with zero configuration: | Specialist | What it does | | ---------- | ------------ | @@ -22,9 +22,9 @@ These ship in the binary — delegation works with zero configuration: | `verify` | Adversarially checks a specific claim or finding against the real code; returns a verdict. | | `review-lens` | Reviews a change for correctness / regressions from a senior-engineer lens. | -Each is a read-only agent with a prompt that mandates real investigation and cited findings — it must read before it answers. +Each is a read-only agent with a prompt that mandates real investigation and cited findings. It must read before it answers. -**Structured findings.** A specialist doesn't return a wall of prose. It finishes by calling `agent_result` with a **structured** payload — a one-line `summary` plus a list of `findings`, each carrying its own `source` (a `file:line` for code, a URL for external docs) and a `confidence`. The schema *forces* evidence onto every point, which is exactly what separates a grounded answer from a confident hallucination; the orchestrator receives a consistent `summary + cited findings` block it can act on directly. +**Structured findings.** A specialist doesn't return a wall of prose. It finishes by calling `agent_result` with a **structured** payload: a one-line `summary` plus a list of `findings`, each carrying its own `source` (a `file:line` for code, a URL for external docs) and a `confidence`. The schema *forces* evidence onto every point, which is exactly what separates a grounded answer from a confident hallucination; the orchestrator receives a consistent `summary + cited findings` block it can act on directly. ### Add or override a specialist @@ -39,11 +39,11 @@ Drop a JSON file in `.tsforge/agents/.json` (project) or `~/.tsforge/agents/ } ``` -`tools` is intersected with the read-only tool set — a spec can never grant a subagent the ability to write. +`tools` is intersected with the read-only tool set. A spec can never grant a subagent the ability to write. ## The live agent tree -While subagents run, they render as a live tree pinned above the input row — so a delegated run is never a black box: +While subagents run, they render as a live tree pinned above the input row. A delegated run is never a black box: ``` ● agents · 1 running · 1/2 done @@ -51,9 +51,9 @@ While subagents run, they render as a live tree pinned above the input row — s └─ ⠹ verify ``` -Rows appear the moment they're spawned, animate a spinner while running, and finish with their wall-clock and turn count. Beneath the tree, a detail pane streams the **focused** agent's live output (the files it reads, what it finds) — it follows the newest running agent automatically. The tree repaints in place (it never scrolls your transcript) and collapses to `… +N more` past a dozen rows. Ctrl-C cancels the turn and all its subagents. +Rows appear the moment they're spawned, animate a spinner while running, and finish with their wall-clock and turn count. Beneath the tree, a detail pane streams the **focused** agent's live output (the files it reads, what it finds). It follows the newest running agent automatically. The tree repaints in place (it never scrolls your transcript) and collapses to `… +N more` past a dozen rows. Ctrl-C cancels the turn and all its subagents. -**Watch any agent.** Press **↑/↓** while subagents are running to move the detail pane between them — pick one and watch what it's doing, instead of only seeing the newest. This works in both input modes (the default multi-line editor and the basic input row); in the editor it's only bound while the input is empty, so the arrows still edit a message you're composing mid-run. Auto-follow resumes on a fresh turn. +**Watch any agent.** Press **↑/↓** while subagents are running to move the detail pane between them. Pick one and watch what it's doing, instead of only seeing the newest. This works in both input modes (the default multi-line editor and the basic input row); in the editor it's only bound while the input is empty, so the arrows still edit a message you're composing mid-run. Auto-follow resumes on a fresh turn. ## Concurrency @@ -63,7 +63,7 @@ How many subagents run at once is capped by [`agents.concurrency`](/guardrails/c { "agents": { "concurrency": 4 } } ``` -When the orchestrator spawns several agents in one turn, up to this many run in parallel; the rest queue (shown as pending `○` rows) until a slot frees. At `concurrency: 1` they run one at a time — you still get the context-isolation win, just no wall-clock overlap. The config is discovered by walking up from the working directory, so it applies even when tsforge runs from a subdirectory. The REPL prints the resolved setup at startup: +When the orchestrator spawns several agents in one turn, up to this many run in parallel; the rest queue (shown as pending `○` rows) until a slot frees. At `concurrency: 1` they run one at a time. You still get the context-isolation win, just no wall-clock overlap. The config is discovered by walking up from the working directory, so it applies even when tsforge runs from a subdirectory. The REPL prints the resolved setup at startup: ``` ↳ delegation: 4 specialists (explore, research, review-lens, verify) · cap 4 @@ -75,9 +75,9 @@ If you set `concurrency` above 1 but agents still run serially, either the cap i - **Read-only.** A subagent's tools are the read-only set ∩ its spec; the executor hard-rejects any mutation, and the policy layer evaluates every call. - **No recursion.** A subagent is never offered `spawn_agent`, so delegation depth is capped at one. -- **Never overflows.** A subagent auto-compacts its own conversation before a request would exceed the model's context window (and recovers from an overflow rejection by compacting and retrying) — so a long, read-heavy investigation can't fail on length. -- **Policy-aware.** Delegation is its own policy action class — a repo can `deny`/`ask` it via [permissions](/guardrails/policy/), and each subagent inherits the session's policy mode. +- **Never overflows.** A subagent auto-compacts its own conversation before a request would exceed the model's context window (and recovers from an overflow rejection by compacting and retrying). A long, read-heavy investigation can't fail on length. +- **Policy-aware.** Delegation is its own policy action class. A repo can `deny`/`ask` it via [permissions](/guardrails/policy/), and each subagent inherits the session's policy mode. ## Turning it off -Delegation is on by default. Set `TSFORGE_NO_DELEGATION=1` to withhold the `spawn_agent` tool entirely — the orchestrator runs as a single stream with no subagents (the startup line reads `↳ delegation: OFF`). Useful as a control when measuring delegation's effect, or to force a pure single-agent run. +Delegation is on by default. Set `TSFORGE_NO_DELEGATION=1` to withhold the `spawn_agent` tool entirely. The orchestrator runs as a single stream with no subagents (the startup line reads `↳ delegation: OFF`). Useful as a control when measuring delegation's effect, or to force a pure single-agent run. diff --git a/apps/docs/src/content/docs/agent/model-agent.mdx b/apps/docs/src/content/docs/agent/model-agent.mdx index 39f8b1bb..8fc737ee 100644 --- a/apps/docs/src/content/docs/agent/model-agent.mdx +++ b/apps/docs/src/content/docs/agent/model-agent.mdx @@ -1,6 +1,6 @@ --- title: Model agent -description: One turn of the loop — prompt, tools, edits, gate, repair. +description: "One turn of the loop: prompt, tools, edits, gate, repair." --- The **model agent** is what runs when you give tsforge a task. It is not a separate program you invoke. It is the loop inside tsforge: send context to the model, run tool calls, check the gate, feed errors back, repeat. @@ -21,10 +21,10 @@ One approved task can involve many agent cycles until the gate passes or tsforge | --- | --- | --- | | Core | `read`, `run`, `edit`, `create` | always | | Line edits | `edit_lines` | always (line-number edits with hash verification) | -| Script | `script` | always (programmatic tool calling — batch multi-step tool use in one program); withhold with `TSFORGE_NO_SCRIPT=1` for eval | +| Script | `script` | always (programmatic tool calling: batch multi-step tool use in one program); withhold with `TSFORGE_NO_SCRIPT=1` for eval | | Navigation | `search`, `symbol_search`, `find_references`, `type_at`, `diagnostics`, `rename_symbol`, `move_file`, `organize_imports` | existing-code repos | | Git context | `git_context` | existing-code repos (read-only: diff/log/blame/show to scope a change) | -| Conventions | `pull_conventions` | build backends that ship a convention library (BoringStack) — fetch a how-to guide on demand | +| Conventions | `pull_conventions` | build backends that ship a convention library (BoringStack): fetch a how-to guide on demand | | Web research | `package_info`, `package_docs`, `web_fetch`, `web_search`, `web_browse` | when **Web tools** is on in `/config` (no required API keys or paid browser/search service) | On greenfield specs, navigation tools are often withheld so the model focuses on creating files instead of exploring an empty tree. See [TypeScript language server](/lsp/typescript-server/). @@ -35,6 +35,6 @@ On greenfield specs, navigation tools are often withheld so the model focuses on ## File context -The model cannot edit a file it has not read. Specs and tasks should list which files are in scope (`files:` / `context:`). See [Spec format](/spec/format/). Some paths are **always** read-only regardless of scope — see [Vendored & read-only scope](/edit/engine/#vendored--read-only-scope). +The model cannot edit a file it has not read. Specs and tasks should list which files are in scope (`files:` / `context:`). See [Spec format](/spec/format/). Some paths are **always** read-only regardless of scope. See [Vendored & read-only scope](/edit/engine/#vendored--read-only-scope). → [Big picture](/big-picture/) · [When the gate fails](/loop/validation/) diff --git a/apps/docs/src/content/docs/agent/skills-and-harness.mdx b/apps/docs/src/content/docs/agent/skills-and-harness.mdx index 004d36f1..3c2abf72 100644 --- a/apps/docs/src/content/docs/agent/skills-and-harness.mdx +++ b/apps/docs/src/content/docs/agent/skills-and-harness.mdx @@ -1,17 +1,17 @@ --- title: Skills and the harness -description: How IDE agent skills relate to tsforge enforcement — skills guide maintainers; the gate is the runtime oracle. +description: How IDE agent skills relate to tsforge enforcement. Skills guide maintainers; the gate is the runtime oracle. --- tsforge is **enforcement-first**. During a run, the harness does not load `SKILL.md` files. Instead it uses deterministic gates, [rule packs](/guardrails/rule-packs/), and [bad/good feedback cards](/loop/validation/) to tell the model what failed and how to fix it. -**Agent skills** are a separate layer for maintainers and IDE workflows — release, harness audits, skill authoring. They live in the repo under `.claude/skills/` and are discovered by Claude Code / compatible agents at session start. +**Agent skills** are a separate layer for maintainers and IDE workflows: release, harness audits, skill authoring. They live in the repo under `.claude/skills/` and are discovered by Claude Code / compatible agents at session start. ## Two layers | Layer | What it does | Where it lives | | --- | --- | --- | -| **Harness** | Enforces invariants; work is not done until the gate passes | `packages/core/` — rule packs, meta-rules, gate, `rule-docs` | +| **Harness** | Enforces invariants; work is not done until the gate passes | `packages/core/` (rule packs, meta-rules, gate, `rule-docs`) | | **Skills** | Repeatable procedures for humans/agents outside the run loop | `.claude/skills///SKILL.md` | ```mermaid @@ -29,9 +29,9 @@ Skills **inform** how you write rules and docs. They do **not** replace the gate ``` .claude/skills/ -├── harness/ harness-review — adversarial subsystem audit -├── release/ tsforge-release — npm + GitHub release -└── authoring/ effective-tsforge-skills — how to write skills here +├── harness/ harness-review (adversarial subsystem audit) +├── release/ tsforge-release (npm + GitHub release) +└── authoring/ effective-tsforge-skills (how to write skills here) ``` Each skill folder name must match its frontmatter `name`. See `effective-tsforge-skills` for the ship checklist. @@ -50,12 +50,12 @@ Do not put enforceable invariants only in a skill. If the model must not ship wi ## Recipes vs skills -[Recipes](/cli/recipes/) compose **existing CLI options** — gate, scope, models, `profile`, plan mode, greenfield. They are data (`*.json`), not procedures. +[Recipes](/cli/recipes/) compose **existing CLI options**: gate, scope, models, `profile`, plan mode, greenfield. They are data (`*.json`), not procedures. -Skills compose **methodology** — repro-before-report harness reviews, release scripts with failure modes. A recipe might set `profile: "strict"`; a skill teaches how to run `bun run validate` before tagging. +Skills compose **methodology**: repro-before-report harness reviews, release scripts with failure modes. A recipe might set `profile: "strict"`; a skill teaches how to run `bun run validate` before tagging. ## Learned rules -tsforge mines gate failure→fix pairs into [`.tsforge/learned-rules.json`](/uplift/memory/) and promotes recurring patterns to TTSR triggers. These are repo-local "micro-skills" — machine triggers, not markdown runbooks. High-confidence patterns can later be promoted to committed `IRuleDoc` entries or maintainer skills. +tsforge mines gate failure→fix pairs into [`.tsforge/learned-rules.json`](/uplift/memory/) and promotes recurring patterns to TTSR triggers. These are repo-local "micro-skills": machine triggers, not markdown runbooks. High-confidence patterns can later be promoted to committed `IRuleDoc` entries or maintainer skills. → [Recipes](/cli/recipes/) · [Rule packs](/guardrails/rule-packs/) · [When the gate fails](/loop/validation/) · [Contributing](https://github.com/agjs/tsforge/blob/main/CONTRIBUTING.md) diff --git a/apps/docs/src/content/docs/big-picture.mdx b/apps/docs/src/content/docs/big-picture.mdx index 1bed6479..1dbc628f 100644 --- a/apps/docs/src/content/docs/big-picture.mdx +++ b/apps/docs/src/content/docs/big-picture.mdx @@ -1,57 +1,86 @@ --- title: Big picture -description: What tsforge is, how the loop works, and where each doc section fits. +description: What tsforge is, how it's layered, how the loop works, and where each doc section fits. --- -import HarnessBoardDiagram from "../../components/HarnessBoardDiagram.astro"; - tsforge is a **TypeScript coding harness**. It sits between you and a model, runs edits in your repo, and keeps asking the model to fix things until your project passes a real check. It is not an editor plugin. You point it at a folder, give it a task, and it drives file changes plus validation until the gate is green or the loop stops. If you already ran the [Quickstart](/quickstart/), this page explains what you just turned on. **How to read the docs:** start here, then [How the gate is built](/loop/gate-floor/) when "what counts as done?" is unclear. Use the glossary below when a term looks unfamiliar. Eval and spec pages are for benchmark contributors, not required for day-to-day use. -## What the harness is made of +## How it's layered - +tsforge is built as **one general core plus adapters**. The engine (write, check, steer, repeat) is the same for any TypeScript project. Everything stack-specific (how to scaffold a new app, what a given error *means*, how to verify a feature works) lives in an **adapter** that plugs into a fixed set of seams. The core imports nothing from an adapter; a lint rule fails the build if it tries. -The diagram shows **composition** — what subsystems exist. The loop diagram below shows **flow** — what happens on each turn. +```mermaid +flowchart TB + subgraph core["Universal core · stack-agnostic"] + direction LR + A["planning"] --- B["drive-to-green loop"] --- C["gate · ESLint rule-packs"] --- D["tools"] --- E["review panel"] + end + subgraph seam["The seam · adapters plug in here"] + direction LR + S1["IStackAdapter"] --- S2["IConventionProvider"] --- S3["IPlanSchema<TUi>"] --- S4["IGate"] --- S5["external-plugins"] + end + subgraph adapters["Adapters · stack-specific"] + direction LR + BS["BoringStack (built)"] --- PH["Phaser (planned)"] --- DD["your own rules (via external-plugins)"] + end + core --> seam --> adapters +``` -## Why tsforge exists +An **adapter** is exactly: fill those interfaces. `IStackAdapter` detects a repo and scaffolds it; `IConventionProvider` supplies the "here's how to write it right" guidance; `IPlanSchema` describes the plan/UI shape; `IGate` says how to run and verify. [BoringStack](/scaffold/boringstack/) is the first adapter (a full-stack Bun + Elysia + Drizzle + React stack). You can also bring **your own ESLint rule-packs** without writing an adapter. See [external plugins](/guardrails/config/). -tsforge started as a necessity experiment: could a 27B model running on local hardware produce TypeScript you'd actually merge, if the harness enforced `tsc`, stack rules, and stream-level corrections? It could. That proof shaped every subsystem here. +## What runs on a task -The project was developed against Qwen 3.6 27B, but that is where the idea was tested, not what the product is for. The same loop works with larger or hosted models. They usually need fewer retries. They still benefit from an external oracle and stack-aware rules. +The diagram above is **composition**: the subsystems that exist. This is **flow**: what happens when you give it a task: -Guardrails matter most when the model is constrained. They still help when the model is strong. +```mermaid +flowchart TD + you(["You type a task or product description"]) --> repl["CLI · REPL"] + repl --> detect{"Stack adapter
detects the repo?"} + detect -->|"greenfield"| plan["Plan the product
model proposes feature slices"] + detect -->|"existing repo"| loop + plan --> approve{"You approve
the plan?"} + approve -->|"revise"| plan + approve -->|"yes"| loop + + subgraph loop["Drive-to-green loop · repeats per slice"] + direction TB + write["Model writes / edits code
conventions front-loaded"] --> gate["Gate
tsc + ESLint rule-packs + tests"] + gate --> green{"Green?"} + green -->|"errors"| steer["Feedback · steer ladder
exact errors + rule help"] + steer --> write + green -->|"near-green regression"| ckpt["Checkpoint / rollback"] + ckpt --> write + end + + green -->|"green"| acc["Acceptance
reachability · testids · judge"] + acc --> panel["4-model review panel"] + panel -->|"pass"| done(["Done · ready to merge"]) + panel -->|"block"| write +``` -## The problem it targets +The **model agent** is the loop driver: it calls tools, applies edits, runs the [repair ladder](/uplift/repair-ladder/) when a tool call is malformed, and re-prompts on gate failure. The **gate** is the oracle: if it fails, the harness feeds the error list back and the model gets another turn. A single high crash-guard (1000 turns) plus no-progress guards stop runaway churn without cutting off a productive build. See [When the gate fails](/loop/validation/) for details. -Models in a chat window can write TypeScript fast. They also skip types, paste `as any`, forget your stack conventions, and stop before tests pass. Nothing in the chat UI says "this edit failed `tsc`, try again with these exact errors." +## Why tsforge exists -tsforge closes that gap with three ideas: +tsforge started as a necessity experiment: could a small model running on local hardware produce TypeScript you'd actually merge, if the harness enforced `tsc`, stack rules, and stream-level corrections? It could. That proof shaped every subsystem here. -1. **A gate you can trust.** Work is not done until a shell command exits 0. For TypeScript repos that usually means `tsc --noEmit`, ESLint from tsforge's own strict config, and often tests or format checks. The model sees structured errors, not a vague "something broke." +The idea was first tested against a 27B local model; today the default local stack is **DeepSeek‑V4‑Flash served on-box via a custom vLLM setup**. The same loop works with larger or hosted models. They usually need fewer retries and still benefit from an external oracle and stack-aware rules. Guardrails matter most when the model is constrained; they still help when it is strong. -2. **Feedback while the model still types.** Stream rules (TTSR) can cut off bad tool arguments mid-generation. Hashline edits tie replacements to a file hash so line numbers do not drift. The TypeScript language service can flag a bad write before the next turn. +## The problem it targets -3. **Stack-aware rules without hand-wiring.** tsforge reads `package.json`, turns on ESLint packs for React, Drizzle, Elysia, and the rest, and runs them on every validation. You get project-specific guardrails without maintaining a separate lint setup for the harness. +Models in a chat window can write TypeScript fast. They also skip types, paste `as any`, forget your stack conventions, and stop before tests pass. Nothing in the chat UI says "this edit failed `tsc`, try again with these exact errors." -## One task, one loop +tsforge closes that gap with three ideas: -```mermaid -flowchart LR - you[You approve a task] --> agent[Model agent] - agent --> tools[read / edit / create / run] - tools --> gate[Gate: tsc + ESLint + …] - gate -->|fail| repair[Errors back to model] - repair --> agent - gate -->|pass| done[Checkpoint] -``` +1. **A gate you can trust.** Work is not done until a shell command exits 0. For TypeScript repos that usually means `tsc --noEmit`, ESLint from tsforge's own rule-packs, and often tests or format checks. The model sees structured errors, not a vague "something broke." -The **model agent** is the loop driver: it calls tools, applies edits, runs the [repair ladder](/uplift/repair-ladder/) when a tool call is malformed, and re-prompts on gate failure. The **gate** is the oracle: if it fails, the harness feeds the error list back and the model gets another turn. +2. **Feedback while the model still types.** Stream rules (TTSR) can cut off bad tool arguments mid-generation. Hashline edits tie replacements to a file hash so line numbers do not drift. The TypeScript language service can flag a bad write before the next turn. -Caps prevent infinite churn (for example 12 repair cycles, 3 stalled cycles with no progress). Details live in [When the gate fails](/loop/validation/). +3. **Stack-aware rules without hand-wiring.** tsforge reads `package.json`, turns on ESLint packs for React, Drizzle, Elysia, and the rest, and runs them on every validation. You get project-specific guardrails without maintaining a separate lint setup, and you can layer your own packs on top ([external plugins](/guardrails/config/)). ## Why TypeScript specifically @@ -61,13 +90,13 @@ tsforge is opinionated about TypeScript because the language gives you machine-c - **The language server** (same engine as VS Code's TypeScript support) runs in-process for symbol search, rename, and diagnostics on every write. See [TypeScript language server](/lsp/typescript-server/). - **ESLint packs** encode patterns `tsc` cannot catch (component file layout, no `console.log`, Drizzle query shape, etc.). See [Rule packs](/guardrails/rule-packs/). -For greenfield web apps, the harness stands up the full BoringStack stack (Bun + Elysia + Drizzle API and a Vite/React UI) and builds it resource-by-resource against BoringStack's own gate. See [Greenfield scaffolding](/scaffold/boringstack/). +For greenfield web apps, the BoringStack adapter stands up a full stack (Bun + Elysia + Drizzle API and a Vite/React UI) and builds it feature-by-feature against its own gate. See [Greenfield scaffolding](/scaffold/boringstack/). ## Models and deployment tsforge talks to any OpenAI-compatible HTTP API. Configure endpoints in `~/.tsforge/models.json` or override with environment variables. See [Model adapter](/inference/adapter/). -The default `http://localhost:8000/v1` is a convenience for local inference servers. Point `TSFORGE_BASE_URL` at a hosted API when you want. The gate, packs, and stream rules do not change. +The default endpoint (`http://localhost:8000/v1`) is a convenience for a local inference server; point `TSFORGE_BASE_URL` at a hosted API when you want. The gate, packs, and stream rules do not change. ## Doc map (plain language) @@ -77,10 +106,10 @@ The default `http://localhost:8000/v1` is a convenience for local inference serv | [Plan mode](/cli/plan-mode/) | Read-only exploration before you let the model edit | | [How the gate is built](/loop/gate-floor/) | What acceptance check tsforge runs and how it picks tsc + ESLint | | [When the gate fails](/loop/validation/) | Repair loop, stop conditions, error feedback to the model | -| [TypeScript language server](/lsp/typescript-server/) | Navigation tools and write-time diagnostics | -| [Greenfield scaffolding](/scaffold/boringstack/) | Standing up a new BoringStack full-stack app | +| [Greenfield scaffolding](/scaffold/boringstack/) | Standing up a new BoringStack full-stack app (the first adapter) | | [Stack detection](/guardrails/stack-detection/) | Which ESLint packs turn on for your dependencies | -| [Rule packs](/guardrails/rule-packs/) | What each pack enforces | +| [Rule packs](/guardrails/rule-packs/) | What each pack enforces + bringing your own | +| [Config & external plugins](/guardrails/config/) | Profiles, and plugging in your own rule-packs | | [Fix bad tool calls](/uplift/repair-ladder/) | Malformed tool JSON fixed before re-asking the model | | [Safer line edits](/uplift/hashline/) | Line edits anchored to a content hash | | [Stop bad output early](/uplift/ttsr/) | Forbidden patterns cut off mid-stream | @@ -93,6 +122,9 @@ The default `http://localhost:8000/v1` is a convenience for local inference serv | --- | --- | | **Gate** | Shell command that must pass before tsforge treats work as done. Often `tsc`, ESLint, tests. | | **Harness** | The tsforge runtime: CLI, loop, gate, guardrails, and model wiring. | +| **Core** | The stack-agnostic engine: planning, the drive-to-green loop, the gate, the rule-packs, the review panel. Imports no adapter. | +| **Adapter** | The stack-specific half behind the seams: how to scaffold, what an error means, how to verify. BoringStack is the first. | +| **Seam** | An injected interface the adapter fills (`IStackAdapter`, `IConventionProvider`, `IPlanSchema`, `IGate`). | | **LSP / language server** | TypeScript's semantic engine (`LanguageService`): go-to-definition, types, rename, diagnostics. tsforge embeds it; you do not run a separate tsserver process. | | **TTSR** | **T**ool-**t**ext **s**tream **r**ules. Regex watchers on streaming model output. On match, tsforge aborts the stream and retries with short guidance (for example "no `as any`"). | | **Hashline** | Edit format `¶path#HASH` plus line ops. The hash proves the file has not changed since the model read it. | diff --git a/apps/docs/src/content/docs/cli/interactive.mdx b/apps/docs/src/content/docs/cli/interactive.mdx index 5a18971b..9faee122 100644 --- a/apps/docs/src/content/docs/cli/interactive.mdx +++ b/apps/docs/src/content/docs/cli/interactive.mdx @@ -13,10 +13,10 @@ Most users run `tsforge` and stay in the interactive session. | Mode | How | When | | --- | --- | --- | -| **Interactive** | `tsforge` | default — open-ended tasks, steering, exploration | +| **Interactive** | `tsforge` | default: open-ended tasks, steering, exploration | | **One-shot** | `tsforge "task" --accept "gate cmd"` | drive a single task to green and exit | | **New app** | `tsforge scaffold` | stand up a full-stack [BoringStack](/scaffold/boringstack/) project, then build it with the [greenfield loop](/loop/greenfield/) | -| **Plan mode** | default — **Shift+Tab** or `/plan` to switch | read-only exploration before implementing — see [Plan mode](/cli/plan-mode/) | +| **Plan mode** | default: **Shift+Tab** or `/plan` to switch | read-only exploration before implementing. See [Plan mode](/cli/plan-mode/). | ## Flags @@ -27,20 +27,21 @@ Most users run `tsforge` and stay in the interactive session. | `--accept` / `--gate ` | gate command for one-shot mode | | `--no-gate` | skip auto gate detection | | `--browser ` | append headless render check to gate | -| `--plan` | force plan mode on for an interactive session — plan is the default anyway, so this only matters to override a repo that configured an autonomous `policy.mode`; ignored by one-shot/headless | +| `--plan` | force plan mode on for an interactive session. Plan is the default anyway, so this only matters to override a repo that configured an autonomous `policy.mode`; ignored by one-shot/headless | | `--continue` / `-c` | resume latest saved session for this dir | | `--resume ` | resume a specific session | | `--log` | append JSONL event stream to `~/.tsforge/logs/` | -Model endpoint overrides: `TSFORGE_BASE_URL`, `TSFORGE_MODEL` — see [Environment variables](/reference/flags/). +Model endpoint overrides: `TSFORGE_BASE_URL`, `TSFORGE_MODEL`. See [Environment variables](/reference/flags/). ## Slash commands | Command | Action | | --- | --- | | `/help` | list commands | +| `/scaffold` | create a new full-stack project here (BoringStack / Astro) via the wizard | | `/plan` | toggle plan mode (on by default) | -| `/config` | settings hub — model (switch/add), mode, gate, editable scope, and tools (web, TDD); each with a description + live value | +| `/config` | settings hub: model (switch/add), mode, gate, editable scope, and tools (web, TDD); each with a description + live value | | `/gate ` | set gate command (`/gate` alone clears) | | `/files ` | set editable scope | | `/review [base]` | review your current change (logic, regressions, edge cases) | @@ -62,17 +63,17 @@ Anything else is sent to the agent. While it runs, type to **steer** the next tu ## Status bar -On a real terminal, a status bar stays **pinned to the bottom** while output scrolls above it — always in view, never scrolled away: +On a real terminal, a status bar stays **pinned to the bottom** while output scrolls above it, always in view and never scrolled away: ``` - ⎯ qwen3.6-27b · ◆ plan · ctx ~8k/32k 25% · 2 turns · 12s · 48 tok/s · done · src/** + ⎯ deepseek-v4-flash · ◆ plan · ctx ~8k/32k 25% · 2 turns · 12s · 48 tok/s · done · src/** ``` -It shows the active model, the current mode chip (`◆ plan` / `◆ normal`), real context-window usage, the last turn's count + elapsed, the current generation rate, the last outcome, and the editable scope. It **ticks live while the model works** — tokens/sec and the context meter update mid-turn (the status reads `● working`), settling to `✓ done` / `✗ stuck` when the turn ends — and it survives terminal resizes. When stdout isn't a TTY (piped output, `--log`, a tiny window) the bar is skipped and the same line prints inline above each prompt instead, so logs and pipes stay clean. +It shows the active model, the current mode chip (`◆ plan` / `◆ normal`), real context-window usage, the last turn's count + elapsed, the current generation rate, the last outcome, and the editable scope. It **ticks live while the model works**: tokens/sec and the context meter update mid-turn (the status reads `● working`), settling to `✓ done` / `✗ stuck` when the turn ends, and it survives terminal resizes. When stdout isn't a TTY (piped output, `--log`, a tiny window) the bar is skipped and the same line prints inline above each prompt instead, so logs and pipes stay clean. ## Live metrics -The status bar shows the last call's generation rate (`tok/s`) next to the context gauge. `/metrics` prints the running totals for the session — model calls, input/output tokens, and both the last and average output rate: +The status bar shows the last call's generation rate (`tok/s`) next to the context gauge. `/metrics` prints the running totals for the session: model calls, input/output tokens, and both the last and average output rate: ``` 4 call(s) · 8120 in / 1540 out · 47 tok/s last · 44 tok/s avg @@ -86,11 +87,11 @@ Full guide, including log analysis: [Token metrics & tokens/sec](/observability/ On start you see: -- **cwd** — target project -- **scope** — editable file globs -- **gate** — acceptance command (auto-detected or set) -- **model** — active entry from `~/.tsforge/models.json` -- **session** — new or resumed id +- **cwd**: target project +- **scope**: editable file globs +- **gate**: acceptance command (auto-detected or set) +- **model**: active entry from `~/.tsforge/models.json` +- **session**: new or resumed id Gate auto-detection is described in [How the gate is built](/loop/gate-floor/). @@ -100,6 +101,6 @@ Models live in `~/.tsforge/models.json`. Use `/model` to hot-swap without restar ## Run logs -`--log` writes every loop event (tool calls, gate verdicts, repairs) as JSONL under `~/.tsforge/logs/`. Use this for eval analysis — see [A/B testing](/eval/ab-testing/). +`--log` writes every loop event (tool calls, gate verdicts, repairs) as JSONL under `~/.tsforge/logs/`. Use this for eval analysis. See [A/B testing](/eval/ab-testing/). → [Quickstart](/quickstart/) · [Plan mode](/cli/plan-mode/) · [Model adapter](/inference/adapter/) diff --git a/apps/docs/src/content/docs/cli/map.mdx b/apps/docs/src/content/docs/cli/map.mdx index 506e2ab9..2fd33161 100644 --- a/apps/docs/src/content/docs/cli/map.mdx +++ b/apps/docs/src/content/docs/cli/map.mdx @@ -1,6 +1,6 @@ --- title: Map the repo -description: "tsforge map — build a structural map of your TypeScript repo so the agent starts oriented instead of exploring blind." +description: "tsforge map: build a structural map of your TypeScript repo so the agent starts oriented instead of exploring blind." --- When the agent opens an unfamiliar repo, its first instinct is to read files until it finds the relevant code, which burns turns. `tsforge map` gives it a head start: a structural map of the codebase, added to its context so it knows where things live before it starts. diff --git a/apps/docs/src/content/docs/cli/plan-mode.mdx b/apps/docs/src/content/docs/cli/plan-mode.mdx index 6b5658b3..83c5538e 100644 --- a/apps/docs/src/content/docs/cli/plan-mode.mdx +++ b/apps/docs/src/content/docs/cli/plan-mode.mdx @@ -5,21 +5,21 @@ description: Read-only exploration before the model is allowed to edit files. Plan mode is a safety rail for ambiguous work. The model can **read** your repo and **propose** a plan, but it cannot **edit** until you say go. -**It is the default for a fresh interactive session** — tsforge explores, asks the few clarifying questions that matter, and proposes a plan before it writes anything. The status bar shows the current mode as a `◆ plan` / `◆ normal` chip. +**It is the default for a fresh interactive session.** tsforge explores, asks the few clarifying questions that matter, and proposes a plan before it writes anything. The status bar shows the current mode as a `◆ plan` / `◆ normal` chip. ## Switching modes - Press **Shift+Tab** to cycle the mode (plan → normal → …), or type **`/plan`** to toggle it -- When the plan looks right, reply **`approve`**, **`go`**, or **`lgtm`** — the model implements it +- When the plan looks right, reply **`approve`**, **`go`**, or **`lgtm`**. The model implements it -There is no disable *flag*: it's a mode you cycle with Shift+Tab. (`tsforge --plan` forces plan mode on for an interactive session even in a repo that configured an autonomous `policy.mode` — one-shot and headless runs are autonomous regardless.) +There is no disable *flag*: it's a mode you cycle with Shift+Tab. (`tsforge --plan` forces plan mode on for an interactive session even in a repo that configured an autonomous `policy.mode`. One-shot and headless runs are autonomous regardless.) ## What the model can do in plan mode Read tools only: - `read`, `search`, and LSP navigation (`symbol_search`, `find_references`, `type_at`, `diagnostics`) -- `git_context` — structured, read-only repo history/diffs (see [Git context](/reference/flags/#git-context)) +- `git_context`: structured, read-only repo history/diffs (see [Git context](/reference/flags/#git-context)) - `run` for **read-only** shell commands (no installs, no writes) Blocked until approval: `edit`, `create`, `edit_lines`, scaffolders. @@ -35,13 +35,13 @@ After approval, tsforge switches to the normal implement loop with full write to Plan mode is for **exploring and approving a change** in a repo. Building a *new* app from scratch is a different flow: [`tsforge scaffold`](/scaffold/boringstack/) stands up BoringStack, then the [greenfield loop](/loop/greenfield/) drives the app -to green one resource at a time — the harness runs the generators + wiring and the +to green one resource at a time. The harness runs the generators + wiring and the model fills the domain, each feature verified and frozen before the next. That loop plans and checkpoints per feature on its own, so it doesn't need plan mode. ## Leaving plan mode - **Shift+Tab** (or `/plan`) drops to normal mode for hands-on edits in a repo you know -- One-shot runs (`tsforge "task" --accept …`) and headless/eval runs are autonomous already — plan mode is interactive-only, since it needs a human to approve +- One-shot runs (`tsforge "task" --accept …`) and headless/eval runs are autonomous already. Plan mode is interactive-only, since it needs a human to approve → [Greenfield scaffolding](/scaffold/boringstack/) · [Interactive CLI](/cli/interactive/) diff --git a/apps/docs/src/content/docs/cli/recipes.mdx b/apps/docs/src/content/docs/cli/recipes.mdx index d7ce49c8..3c24afbe 100644 --- a/apps/docs/src/content/docs/cli/recipes.mdx +++ b/apps/docs/src/content/docs/cli/recipes.mdx @@ -1,6 +1,6 @@ --- title: Recipes -description: Save a run setup as declarative JSON and invoke it by name with tsforge run — a deterministic, reviewable alternative to retyping flags. +description: Save a run setup as declarative JSON and invoke it by name with tsforge run . A deterministic, reviewable alternative to retyping flags. --- A **recipe** is a named, declarative run setup checked into your repo. Instead of retyping `--files`, `--accept`, the model, and limits every time, you write them once and invoke by name: @@ -10,7 +10,7 @@ tsforge run api-endpoint "add a DELETE /users/:id route" tsforge recipes # list the recipes this repo has ``` -A recipe is **data, not code** — it composes options tsforge already has. It never executes arbitrary logic and never spawns sub-agents, so it stays deterministic and reviewable in a PR. (This is the deliberate divergence from Codebuff's executable `.ts` agent definitions.) +A recipe is **data, not code**. It composes options tsforge already has. It never executes arbitrary logic and never spawns sub-agents, so it stays deterministic and reviewable in a PR. (This is the deliberate divergence from Codebuff's executable `.ts` agent definitions.) ## Where recipes live @@ -30,7 +30,7 @@ A project recipe **overrides** a global one with the same id. `tsforge recipes` "description": "Add an API route under src/api, strict gate", "gate": "bun run validate", "files": ["src/api/**"], - "model": "qwen3-coder", + "model": "deepseek-v4-flash", "maxTurns": 30, "policyMode": "default" } @@ -55,11 +55,11 @@ A project recipe **overrides** a global one with the same id. `tsforge recipes` | `mode` | `--greenfield` | `"greenfield"` selects the [feature-checklist build loop](/loop/greenfield/) | | `plannerModel`, `workModel`, `evaluatorModel` | greenfield roles | per-role model names; each falls back to `model`/active | | `profile` | `tsforge.config.json` `profile` | `recommended`, `strict`, `security`, `frontend`, `backend`, `opinionated` | -| `agents` | `tsforge agents ` | agent spec ids to fan out over the task — resolved from `.tsforge/agents/` (project) and `~/.tsforge/agents/` (global, project wins) | +| `agents` | `tsforge agents ` | agent spec ids to fan out over the task. Resolved from `.tsforge/agents/` (project) and `~/.tsforge/agents/` (global, project wins) | | `web`, `plan`, `log`, `strictFloorOnly` | the matching flags | booleans | A recipe is, in effect, a saved set of CLI options: `tsforge run ` applies them, and you can also combine `--recipe ` with other commands (e.g. `tsforge review --recipe …`). **An explicit CLI flag always wins** over the recipe, so `tsforge run api-endpoint --files lib/**` overrides just the scope. -Fields this version doesn't recognize (a typo, or a not-yet-supported field like `tools`) are **reported, not silently ignored** — so a recipe never quietly does less than it says. +Fields this version doesn't recognize (a typo, or a not-yet-supported field like `tools`) are **reported, not silently ignored**, so a recipe never quietly does less than it says. → [Commands](/reference/commands/) · [tsforge.config.json](/guardrails/config/) · [Models](/inference/models-json/) diff --git a/apps/docs/src/content/docs/cli/review.mdx b/apps/docs/src/content/docs/cli/review.mdx index 50cc40cf..ab2e2a07 100644 --- a/apps/docs/src/content/docs/cli/review.mdx +++ b/apps/docs/src/content/docs/cli/review.mdx @@ -1,6 +1,6 @@ --- title: Review your changes -description: "tsforge review — a functional code review of the change you're on (logic, regressions, edge cases), with every finding re-checked against the code so it doesn't invent problems." +description: "tsforge review: a functional code review of the change you're on (logic, regressions, edge cases), with every finding re-checked against the code so it doesn't invent problems." --- `tsforge review` reviews the change you're working on and prints a focused list of **functional** problems, the things a linter can't catch. @@ -16,7 +16,7 @@ Inside a session, the same review is available as `/review`. ## Gate-aware review -With `--with-gate`, tsforge runs [the gate](/loop/gate-floor/) once before reviewing and tells the reviewer **which rules are already failing**, so it won't spend its attention re-reporting a type or lint error the gate loop will fix anyway. The review focuses on the behaviour of the code the gate already accepts — exactly the part the gate is blind to. The report notes how many gate rules were skipped. Without the flag, review runs as before (no gate run). +With `--with-gate`, tsforge runs [the gate](/loop/gate-floor/) once before reviewing and tells the reviewer **which rules are already failing**, so it won't spend its attention re-reporting a type or lint error the gate loop will fix anyway. The review focuses on the behaviour of the code the gate already accepts (exactly the part the gate is blind to). The report notes how many gate rules were skipped. Without the flag, review runs as before (no gate run). ## What it reviews @@ -45,7 +45,7 @@ By default the per-file find pass and per-finding verify pass run one model call { "agents": { "concurrency": 4 } } ``` -Review then fans out up to that many calls at once — each with a fresh provider — and prints a live `agents: 2 running, 5/16 done (…)` progress line. Results are identical in structure and order to a sequential run; only the wall-clock changes (≈2× measured at cap 8 on a batching endpoint). Keep the cap a couple below your server's `max_num_seqs` so an interactive session is never starved, and leave it at 1 for endpoints that serialize requests anyway. +Review then fans out up to that many calls at once (each with a fresh provider) and prints a live `agents: 2 running, 5/16 done (…)` progress line. Results are identical in structure and order to a sequential run; only the wall-clock changes (≈2× measured at cap 8 on a batching endpoint). Keep the cap a couple below your server's `max_num_seqs` so an interactive session is never starved, and leave it at 1 for endpoints that serialize requests anyway. ## In CI diff --git a/apps/docs/src/content/docs/cli/setup.mdx b/apps/docs/src/content/docs/cli/setup.mdx index b80461e7..11293a10 100644 --- a/apps/docs/src/content/docs/cli/setup.mdx +++ b/apps/docs/src/content/docs/cli/setup.mdx @@ -1,6 +1,6 @@ --- title: Set up a repo -description: "tsforge setup — a wizard that infers your repo's conventions and writes tsforge.config.json so the guardrails match your style, without weakening the safety floor." +description: "tsforge setup: a wizard that infers your repo's conventions and writes tsforge.config.json so the guardrails match your style, without weakening the safety floor." --- By default tsforge brings one opinion: `I`-prefixed interfaces, no enums, tests beside their source, the `src/views/` layout. That's right for a fresh project tsforge scaffolds, but an existing repo usually has its own conventions. `tsforge setup` scans your repo, shows you what it found, and writes those preferences into `tsforge.config.json` so tsforge adapts to your style instead of imposing its own. @@ -10,13 +10,13 @@ tsforge setup # the interactive wizard tsforge setup --yes # write the scan's recommendations, no prompts ``` -Inside a session, the same wizard is available as `/setup`. tsforge never runs it for you — if a repo has no config, you'll just see a one-line hint suggesting it. +Inside a session, the same wizard is available as `/setup`. tsforge never runs it for you; if a repo has no config, you'll just see a one-line hint suggesting it. -## What it can change — and what it can't +## What it can change and what it can't There are two kinds of rules, and setup only touches one of them. -**The safety floor is never negotiable.** No `any`, no `as`/`<>` casts, no non-null `!`, `===`, no `var`, the complexity cap — these catch real bugs and unsafe code. The wizard physically cannot turn them off, and a hand-edited config can't either. +**The safety floor is never negotiable.** No `any`, no `as`/`<>` casts, no non-null `!`, `===`, no `var`, the complexity cap. These catch real bugs and unsafe code. The wizard physically cannot turn them off, and a hand-edited config can't either. **Conventions are taste**, and these are what setup adapts: @@ -27,11 +27,11 @@ There are two kinds of rules, and setup only touches one of them. | `tests` | `co-located` · `mirrored` · `either` | | `componentFolders` | `tsforge-views` · `repo` · `warn` | -Allowing enums removes only the enum ban — the separate `as`-cast ban stays intact. Relaxing a convention never relaxes safety. +Allowing enums removes only the enum ban; the separate `as`-cast ban stays intact. Relaxing a convention never relaxes safety. ## How the wizard works -Each step shows the evidence the scan found (e.g. "300 bare PascalCase, 12 I-prefixed"), the recommended choice, and exactly which config field it sets. Arrow keys to move, Enter to choose, `b` to go back, `q` to cancel. A final overview shows the exact `tsforge.config.json` fragment that will be written — **nothing is written until you Apply**, and Cancel writes nothing. +Each step shows the evidence the scan found (e.g. "300 bare PascalCase, 12 I-prefixed"), the recommended choice, and exactly which config field it sets. Arrow keys to move, Enter to choose, `b` to go back, `q` to cancel. A final overview shows the exact `tsforge.config.json` fragment that will be written. **Nothing is written until you Apply**, and Cancel writes nothing. The scan is read-only: it parses your TypeScript via the compiler's AST and never executes your `eslint.config.*`, `prettier.config.*`, or any project JavaScript. @@ -42,12 +42,12 @@ One choice drives both halves of the harness, so they can never disagree: - **The gate** stops failing your code on a convention you opted out of (choose `bare-pascal-case` and `eslint` no longer demands the `I`). - **The prompts** tell the model to write your style from the start, instead of writing `IFoo` and getting corrected. -Setup writes `tsforge.config.json` (merged — your `mcpServers`, `plugins`, and `policy` are preserved) plus `.tsforge/setup-evidence.json` for audit. Only the choices that differ from the defaults are written, so the file stays minimal. Re-running setup re-decides all four conventions, so the file always matches what the overview showed. +Setup writes `tsforge.config.json` (merged: your `mcpServers`, `plugins`, and `policy` are preserved) plus `.tsforge/setup-evidence.json` for audit. Only the choices that differ from the defaults are written, so the file stays minimal. Re-running setup re-decides all four conventions, so the file always matches what the overview showed. -Conventions govern your everyday sessions (the auto gate, the write-time linter, and the prompts). Scaffolding a brand-new app with [`tsforge scaffold`](/scaffold/boringstack/) is different — it stands up BoringStack and builds against **BoringStack's own** gate and conventions, not your `conventions` setting. +Conventions govern your everyday sessions (the auto gate, the write-time linter, and the prompts). Scaffolding a brand-new app with [`tsforge scaffold`](/scaffold/boringstack/) is different: it stands up BoringStack and builds against **BoringStack's own** gate and conventions, not your `conventions` setting. ## Non-interactive use -Outside a terminal, `tsforge setup` prints the scan and the proposed config and writes nothing — re-run it in a terminal, or pass `--yes` to write the recommendations. If an existing `tsforge.config.json` is invalid JSON, setup refuses to overwrite it and tells you to fix it first. +Outside a terminal, `tsforge setup` prints the scan and the proposed config and writes nothing. Re-run it in a terminal, or pass `--yes` to write the recommendations. If an existing `tsforge.config.json` is invalid JSON, setup refuses to overwrite it and tells you to fix it first. → [tsforge.config.json](/guardrails/config/) · [Commands](/reference/commands/) · [Map the repo](/cli/map/) diff --git a/apps/docs/src/content/docs/edit/engine.mdx b/apps/docs/src/content/docs/edit/engine.mdx index bd79cd51..759844d4 100644 --- a/apps/docs/src/content/docs/edit/engine.mdx +++ b/apps/docs/src/content/docs/edit/engine.mdx @@ -33,10 +33,10 @@ In a full-stack [BoringStack build](/scaffold/boringstack/), generated paths are | Path | Why it's read-only | | --- | --- | -| `**/*.gen.ts` | code generation output (e.g. TanStack's route tree), regenerated by the build — hand-edits are lost | +| `**/*.gen.ts` | code generation output (e.g. TanStack's route tree), regenerated by the build. Hand-edits are lost | Both `edit` and `create` reject writes to it, regardless of the session's editable scope; a type error "in" the generated tree is a wrong **call site** (usually a malformed route file), not a bug in the tree. -Everything else the scaffold lays down is clean, strict, and **editable like any other file**. Earlier versions shipped a "vendored" SDK (`api`/`use-resource`/`result`/mocks) that broke the harness's own strictness (`as` casts, `eslint-disable`) and was hidden behind this guard — that whole layer was removed. The model now writes its own data layer, gated like any code. +Everything else the scaffold lays down is clean, strict, and **editable like any other file**. Earlier versions shipped a "vendored" SDK (`api`/`use-resource`/`result`/mocks) that broke the harness's own strictness (`as` casts, `eslint-disable`) and was hidden behind this guard. That whole layer was removed. The model now writes its own data layer, gated like any code. → [Big picture](/big-picture/) · [Interactive CLI](/cli/interactive/) diff --git a/apps/docs/src/content/docs/eval/ab-testing.mdx b/apps/docs/src/content/docs/eval/ab-testing.mdx index 25cb58ff..12fa5e2e 100644 --- a/apps/docs/src/content/docs/eval/ab-testing.mdx +++ b/apps/docs/src/content/docs/eval/ab-testing.mdx @@ -3,7 +3,7 @@ title: A/B testing description: Run feature sweeps, compare edit mechanisms, and land defaults from measured wins. --- -Compare feature settings across benchmark runs before changing a default. The sweep harness A/Bs any **tool-availability dimension** — whether the model is offered a given tool — by toggling the env var behind it per run. See [Big picture](/big-picture/) for what each feature does. +Compare feature settings across benchmark runs before changing a default. The sweep harness A/Bs any **tool-availability dimension**: whether the model is offered a given tool. Toggle the env var behind it per run. See [Big picture](/big-picture/) for what each feature does. ## Sweepable dimensions @@ -13,34 +13,34 @@ Compare feature settings across benchmark runs before changing a default. The sw | `script` | `script` tool available | `TSFORGE_NO_SCRIPT=1` | | `web` | web research tools available (`TSFORGE_WEB=1`) | off | -Core uplifts ([TTSR](/uplift/ttsr/), [hashline](/uplift/hashline/), [write diagnostics](/uplift/write-diagnostics/)) are always on and no longer sweepable — they landed as defaults from earlier sweeps. Full flag reference: [Environment variables](/reference/flags/). +Core uplifts ([TTSR](/uplift/ttsr/), [hashline](/uplift/hashline/), [write diagnostics](/uplift/write-diagnostics/)) are always on and no longer sweepable. They landed as defaults from earlier sweeps. Full flag reference: [Environment variables](/reference/flags/). :::note -Running a sweep drives a real model, so you need an OpenAI-compatible endpoint (the default is local qwen at `http://localhost:8000/v1`; override with `TSFORGE_BASE_URL`/`TSFORGE_MODEL`/`TSFORGE_API_KEY`). The corpus, analysis, and report tooling below ship with the repo and are exercised by the test suite, but the runs themselves need a model. +Running a sweep drives a real model, so you need an OpenAI-compatible endpoint (the default is a local model at `http://localhost:8000/v1`; override with `TSFORGE_BASE_URL`/`TSFORGE_MODEL`/`TSFORGE_API_KEY`). The corpus, analysis, and report tooling below ship with the repo and are exercised by the test suite, but the runs themselves need a model. ::: ## Seeds A **seed** is the task you run the model against. Committed seeds live in [`evals/corpus/`](https://github.com/agjs/tsforge/tree/main/evals/corpus) and ship with the repo, so the examples work on a fresh clone: -- `math` — single-file pure logic (currency-safe cent arithmetic with half-up rounding). -- `slugify` — single-file string normalization with Unicode edge cases. -- `checkout` — multi-file pricing engine (cart + coupons + tax, integer cents). -- `auth` — multi-file session service (password policy + lockout + token expiry). -- `query` — multi-file mini-SQL engine (lexer → recursive-descent parser → executor). -- `fix-regression` — **brownfield**: an existing repo with git history where a recent commit broke `slugify`; the fix is a `git diff` away. Used to exercise [`git_context`](/reference/flags/#git-context). +- `math`: single-file pure logic (currency-safe cent arithmetic with half-up rounding). +- `slugify`: single-file string normalization with Unicode edge cases. +- `checkout`: multi-file pricing engine (cart + coupons + tax, integer cents). +- `auth`: multi-file session service (password policy + lockout + token expiry). +- `query`: multi-file mini-SQL engine (lexer → recursive-descent parser → executor). +- `fix-regression` (**brownfield**): an existing repo with git history where a recent commit broke `slugify`; the fix is a `git diff` away. Used to exercise [`git_context`](/reference/flags/#git-context). The first two are warm-ups a strong model one-shots; the next three are multi-file and deliberately harder, so a feature has room to move the pass rate (and the [failure breakdown](#why-runs-failed)) off the ceiling. -Pick one with `TSFORGE_SEED=`. A sweep first looks for a local working copy at `evals//`, then falls back to the committed `evals/corpus//`. To add your own, drop a folder under `evals/corpus/` with a `.spec.md` plus its fixture files — see [Spec format](/spec/format/). +Pick one with `TSFORGE_SEED=`. A sweep first looks for a local working copy at `evals//`, then falls back to the committed `evals/corpus//`. To add your own, drop a folder under `evals/corpus/` with a `.spec.md` plus its fixture files. See [Spec format](/spec/format/). ### Brownfield seeds -A greenfield seed is regenerated from scratch (the sweep deletes the task's files and the model rebuilds them). A **brownfield** seed instead sets `mode: existing` in its spec, so the buggy code is kept and the model edits it in place. Such a seed may also ship a `setup.sh`, which the sweep runs in the run directory after laying down the files — it `git init`s, makes the commits, and leaves the red working tree. That is how a `git_context` task has real history (`log`/`blame`/`show`) and a working-tree `diff` to inspect. Greenfield seeds have no `setup.sh` and are unaffected. +A greenfield seed is regenerated from scratch (the sweep deletes the task's files and the model rebuilds them). A **brownfield** seed instead sets `mode: existing` in its spec, so the buggy code is kept and the model edits it in place. Such a seed may also ship a `setup.sh`, which the sweep runs in the run directory after laying down the files. It `git init`s, makes the commits, and leaves the red working tree. That is how a `git_context` task has real history (`log`/`blame`/`show`) and a working-tree `diff` to inspect. Greenfield seeds have no `setup.sh` and are unaffected. ## Running a sweep -`bun run eval:sweep` accepts `TSFORGE_FEATURE_VARIANTS` — a comma-separated list of dimensions to sweep (cartesian product). +`bun run eval:sweep` accepts `TSFORGE_FEATURE_VARIANTS`: a comma-separated list of dimensions to sweep (cartesian product). ### script on/off @@ -82,7 +82,7 @@ Each run directory contains `run.log` (human transcript) and `result.json` (stru ## Statistical report -`bun run eval:report` turns a sweep's JSON into a Markdown table: per-variant pass rate with a **95% Wilson confidence interval**, average cycles/ms/quality, **turns-to-green (T2G)** — the average turns *passed* runs took to go green, the loop-efficiency signal — and, when you name a baseline, a **two-proportion z-test** of each variant's pass-rate difference (a `*` marks p < 0.05). +`bun run eval:report` turns a sweep's JSON into a Markdown table: per-variant pass rate with a **95% Wilson confidence interval**, average cycles/ms/quality, **turns-to-green (T2G)** (the average turns *passed* runs took to go green, the loop-efficiency signal), and when you name a baseline, a **two-proportion z-test** of each variant's pass-rate difference (a `*` marks p < 0.05). ```bash # newest sweep under evals/runs, comparing every variant to the all-off baseline @@ -101,7 +101,7 @@ It prints the table and writes it next to the sweep JSON as `…​.report.md`: | git=on,script=on temp=0 | 10 | 90% | 60%–98% | 4.7 | 33000 | 4.2 | +30% (z=2.13) * | ``` -Wilson intervals (not naive ±) keep the bounds sane at small N, and the z-test tells you whether a pass-rate gap is signal or noise — the bar for "measured wins" before flipping a default. +Wilson intervals (not naive ±) keep the bounds sane at small N, and the z-test tells you whether a pass-rate gap is signal or noise. The bar for "measured wins" before flipping a default is determined by statistical significance. ### Why runs failed @@ -113,11 +113,11 @@ Pass rate tells you *how often* a variant failed; the **failure breakdown** tell - git=on,script=on temp=0: type-error×1 ``` -Each failed run is classified from its event stream into one of: `type-error`, `lint-rule`, `hallucinated-import`, `tool-malformed`, `edit-reject`, `degeneration`, `no-progress`, `build-fail`, `browser-fail`, `route-phantom`, or `timeout`. This turns a sweep from "feature X passes more" into "feature X eliminates the `type-error` failures" — pointing at the next rule, prompt, or fixer to build. The same classifier powers the `failure class` line in [`cli-metrics`](/observability/metrics/) for a single `--log` run. +Each failed run is classified from its event stream into one of: `type-error`, `lint-rule`, `hallucinated-import`, `tool-malformed`, `edit-reject`, `degeneration`, `no-progress`, `build-fail`, `browser-fail`, `route-phantom`, or `timeout`. This turns a sweep from "feature X passes more" into "feature X eliminates the `type-error` failures." It points at the next rule, prompt, or fixer to build. The same classifier powers the `failure class` line in [`cli-metrics`](/observability/metrics/) for a single `--log` run. ## Compare edit mechanisms -`bun run eval:benchmark` reports edit-tool performance across a set of run directories — useful for spotting how `edit` vs `edit_lines` behave, stale-anchor recovery rates, and token cost across models or seeds: +`bun run eval:benchmark` reports edit-tool performance across a set of run directories. It helps spot how `edit` vs `edit_lines` behave, stale-anchor recovery rates, and token cost across models or seeds: ```bash bun run eval:benchmark evals/checkout-* @@ -146,8 +146,8 @@ bun run eval:benchmark \ Each run directory contains: -- **`run.log`** — live event transcript (`renderEvent()` pipeline; source of truth for analysis) -- **`result.json`** — structured metrics, for example: +- **`run.log`**: live event transcript (`renderEvent()` pipeline; source of truth for analysis) +- **`result.json`**: structured metrics, for example: ```json { @@ -180,15 +180,15 @@ Each run directory contains: ## How to read results -**Edit success** — higher `edit_lines` success rate (vs `edit` rejections) means the hashline mechanism is reducing stale-anchor failures. +**Edit success**: Higher `edit_lines` success rate (vs `edit` rejections) means the hashline mechanism is reducing stale-anchor failures. -**Stale recovery** — non-zero recovery counts show the 3-way merge is active; correlate with pass rate. +**Stale recovery**: Non-zero recovery counts show the 3-way merge is active; correlate with pass rate. -**Turns to green** — lower on a variant means less loop churn. +**Turns to green**: Lower on a variant means less loop churn. -**Token efficiency** — smaller `mean args (bytes)` at similar success rate is better. +**Token efficiency**: Smaller `mean args (bytes)` at similar success rate is better. -**Pass rate + quality** — confirm enabling a feature does not regress outcomes beyond noise. +**Pass rate + quality**: Confirm enabling a feature does not regress outcomes beyond noise. ## Landing a new feature @@ -196,14 +196,14 @@ Each run directory contains: 2. Baseline sweep (small: 2 repeats, temp=0): ```bash -TSFORGE_SEED=money TSFORGE_TEMPS=0 TSFORGE_REPEATS=2 \ +TSFORGE_SEED=math TSFORGE_TEMPS=0 TSFORGE_REPEATS=2 \ bun run eval:sweep ``` 3. Disable the feature, same seed/temp/repeats: ```bash -TSFORGE_SEED=money TSFORGE_TEMPS=0 TSFORGE_REPEATS=2 \ +TSFORGE_SEED=math TSFORGE_TEMPS=0 TSFORGE_REPEATS=2 \ TSFORGE_=0 bun run eval:sweep ``` diff --git a/apps/docs/src/content/docs/guardrails/config.mdx b/apps/docs/src/content/docs/guardrails/config.mdx index 04e045ab..793e432b 100644 --- a/apps/docs/src/content/docs/guardrails/config.mdx +++ b/apps/docs/src/content/docs/guardrails/config.mdx @@ -16,11 +16,11 @@ Add **`tsforge.config.json`** at your repo root when you want to override what t | `packs.include` | string[] | Add packs after detection | | `packs.exclude` | string[] | Remove packs after detection | | `rules` | object | Per-rule severity: `"error"`, `"warn"`, or `"off"` (overrides profile defaults) | -| `conventions` | object | Project taste tsforge adapts to: `interfaces`, `enums`, `tests`, `componentFolders` — written by [`tsforge setup`](/cli/setup/) | +| `conventions` | object | Project taste tsforge adapts to: `interfaces`, `enums`, `tests`, `componentFolders` (written by [`tsforge setup`](/cli/setup/)) | | `plugins` | object[] | External modules providing extra rule packs (see below) | | `mcpServers` | object | External [MCP servers](/integrations/mcp/) whose tools the agent can call | -| `policy` | object | Permission `mode` + deny/allow/ask `rules` for tool actions — see [Permissions & policy](/guardrails/policy/) | -| `agents.concurrency` | number | Multiagent cap, integer 1–16 (default `1` = sequential). Bounds how many subagents run at once — both [delegated `spawn_agent` calls](/agent/delegation/) and `tsforge review`'s parallel find/verify passes (fresh provider per unit, live `agents:` progress). Leave at 1 for a local endpoint that serializes requests anyway | +| `policy` | object | Permission `mode` + deny/allow/ask `rules` for tool actions. See [Permissions & policy](/guardrails/policy/) | +| `agents.concurrency` | number | Multiagent cap, integer 1–16 (default `1` = sequential). Bounds how many subagents run at once: both [delegated `spawn_agent` calls](/agent/delegation/) and `tsforge review`'s parallel find/verify passes (fresh provider per unit, live `agents:` progress). Leave at 1 for a local endpoint that serializes requests anyway | Resolution order: detect from the repo → apply **`profile`** (packs + default severities) → apply `stack` → apply `include` → apply `exclude` → add external plugin packs → dedupe → merge `rules` overrides (user wins). @@ -28,7 +28,7 @@ If the file is missing, tsforge uses detection with the **`recommended`** profil ## Profiles -Profiles control which **extra packs** load and which **default severities** apply. They do not replace stack detection — they layer on top. +Profiles control which **extra packs** load and which **default severities** apply. They do not replace stack detection; they layer on top. | Profile | What it adds | | --- | --- | @@ -36,10 +36,10 @@ Profiles control which **extra packs** load and which **default severities** app | `strict` | `recommended` + `typescript-core` pack, CI/supply-chain meta-rules at **error**, optional type-aware async ESLint when `tsconfig.json` exists | | `security` | `recommended` + experimental **`authorization`** heuristics (mutating routes/actions without authz calls) | | `frontend` | `recommended` with React/Next effect and image rules at **warn** | -| `backend` | `recommended` — stack detection adds Fastify/Elysia/Drizzle/BullMQ as usual | +| `backend` | `recommended`. Stack detection adds Fastify/Elysia/Drizzle/BullMQ as usual | | `opinionated` | Full house-style architecture rules (component folder structure, hooks layout, early returns at **error**) | -Every rule in the [Rule catalog](/reference/rules-catalog/) is tagged by **tier** (safety, framework, architecture, experimental). Profiles are the ergonomic switch — you rarely need to tune individual rules. +Every rule in the [Rule catalog](/reference/rules-catalog/) is tagged by **tier** (safety, framework, architecture, experimental). Profiles are the ergonomic switch. You rarely need to tune individual rules. ```json { @@ -94,6 +94,6 @@ Ship your own rule packs without recompiling tsforge. Each `plugins` entry names } ``` -A module that fails to import is reported on stderr and skipped — a broken plugin never blocks a run. +A module that fails to import is reported on stderr and skipped. A broken plugin never blocks a run. → [Stack detection](/guardrails/stack-detection/) · [Rule packs](/guardrails/rule-packs/) · [MCP servers](/integrations/mcp/) · [Environment variables](/reference/flags/) diff --git a/apps/docs/src/content/docs/guardrails/meta-rules.mdx b/apps/docs/src/content/docs/guardrails/meta-rules.mdx index dc4aab73..b8c20f44 100644 --- a/apps/docs/src/content/docs/guardrails/meta-rules.mdx +++ b/apps/docs/src/content/docs/guardrails/meta-rules.mdx @@ -49,7 +49,7 @@ Tune meta-rule severity in [tsforge.config.json](/guardrails/config/) the same w | `no-eslint-disable-comments` | no inline disables | | `no-ts-suppressions` | no `@ts-ignore` / `@ts-expect-error` / `@ts-nocheck` | -Both scan every hand-written `.ts`/`.tsx` under `src/`, `tests/`, and `scripts/`. Generated `*.gen.ts` files are **skipped** — codegen output (e.g. TanStack's route tree) legitimately ships with `/* eslint-disable */` + `@ts-nocheck`, and the model can't write `*.gen.ts` anyway (it's [vendored/read-only](/edit/engine/#vendored--read-only-scope)), so the ban stays airtight everywhere the model actually writes. +Both scan every hand-written `.ts`/`.tsx` under `src/`, `tests/`, and `scripts/`. Generated `*.gen.ts` files are **skipped**. Codegen output (e.g. TanStack's route tree) legitimately ships with `/* eslint-disable */` + `@ts-nocheck`, and the model can't write `*.gen.ts` anyway (it's [vendored/read-only](/edit/engine/#vendored--read-only-scope)), so the ban stays airtight everywhere the model actually writes. ### Config @@ -66,7 +66,7 @@ Both scan every hand-written `.ts`/`.tsx` under `src/`, `tests/`, and `scripts/` | ID | Checks | | --- | --- | -| `test-sibling-required` | a logic file (one exporting a function/class) the agent changes has a test — co-located `*.test.ts` or mirrored under `tests/`. **Error** when [TDD mode](/reference/flags/) is on (the default), warn when off. Scoped to changed files, so it never blocks on pre-existing untested code. | +| `test-sibling-required` | a logic file (one exporting a function/class) the agent changes has a test: co-located `*.test.ts` or mirrored under `tests/`. **Error** when [TDD mode](/reference/flags/) is on (the default), warn when off. Scoped to changed files, so it never blocks on pre-existing untested code. | ### CI @@ -86,17 +86,17 @@ Per-file ESLint can't see the module graph; this builds it from the project's ow | ID | Checks | | --- | --- | -| `no-circular-imports` | no import cycle (A → B → A) — they cause partial-initialization bugs and defeat tree-shaking | +| `no-circular-imports` | no import cycle (A → B → A). They cause partial-initialization bugs and defeat tree-shaking | ### Container -Active whenever a `Dockerfile` (`Dockerfile`, `Dockerfile.*`, or `*.Dockerfile`) exists at the project root or one level down — no-op otherwise. +Active whenever a `Dockerfile` (`Dockerfile`, `Dockerfile.*`, or `*.Dockerfile`) exists at the project root or one level down. No-op otherwise. | ID | Checks | | --- | --- | | `dockerfile-base-image-pinned` | every `FROM` pins an explicit non-`latest` tag (or `@sha256:` digest); build-stage refs and `scratch` are exempt | | `dockerfile-non-root-user` | a non-root `USER` is declared so the container does not run as root | -| `dockerfile-no-secrets-in-env-arg` | no secret-looking `ENV`/`ARG` literal (`*_KEY`, `*_TOKEN`, `*_SECRET`, `*_PASSWORD`) — inject at runtime | +| `dockerfile-no-secrets-in-env-arg` | no secret-looking `ENV`/`ARG` literal (`*_KEY`, `*_TOKEN`, `*_SECRET`, `*_PASSWORD`): inject at runtime | ## Severity diff --git a/apps/docs/src/content/docs/guardrails/policy.mdx b/apps/docs/src/content/docs/guardrails/policy.mdx index 59f5699c..3e761683 100644 --- a/apps/docs/src/content/docs/guardrails/policy.mdx +++ b/apps/docs/src/content/docs/guardrails/policy.mdx @@ -1,9 +1,9 @@ --- title: Permissions & policy -description: "tsforge evaluates every tool action against a deny-first policy — a mode sets the baseline (plan, default, CI…), config rules refine it, and a few critical denies always win." +description: "tsforge evaluates every tool action against a deny-first policy: a mode sets the baseline (plan, default, CI…), config rules refine it, and a few critical denies always win." --- -Every action the agent takes — read a file, write one, run a shell command, hit the network, call an MCP tool — is checked against a policy before it runs. The policy is **deny-first**: the order is critical denies → your config rules → the active mode's default. Nothing ambiguous is ever silently allowed. +Every action the agent takes (read a file, write one, run a shell command, hit the network, call an MCP tool) is checked against a policy before it runs. The policy is **deny-first**: the order is critical denies → your config rules → the active mode's default. Nothing ambiguous is ever silently allowed. ## Modes @@ -14,9 +14,9 @@ A mode sets the baseline posture. Pick one with `--policy-mode `, or `poli | `default` | allow | allow | allow | deny | Interactive day-to-day (the default). | | `plan` | **deny** | read-only | allow | deny | Read-only exploration before you approve a plan. | | `acceptEdits` | allow | **ask** | deny | deny | Auto-accept edits, but still confirm shell. | -| `ci` | allow | **deny** | **deny** | deny | Non-interactive pipelines — anything that would prompt is denied. | +| `ci` | allow | **deny** | **deny** | deny | Non-interactive pipelines; anything that would prompt is denied. | | `dontAsk` | allow | **deny** | **deny** | deny | A local "never prompt me" run (identical to `ci`). | -| `bypassPermissions` | allow | allow | allow | allow | The escape hatch — allow everything (critical denies still apply). | +| `bypassPermissions` | allow | allow | allow | allow | The escape hatch: allow everything (critical denies still apply). | In `plan` mode the `run` tool is allowed through but its own read-only guard restricts it to read-only commands, so exploration can't mutate. `delete` is denied in every mode except `bypassPermissions`. Anything that would otherwise **ask** collapses to **deny** when there's no interactive approval path (non-interactive runs). @@ -37,18 +37,18 @@ Refine a mode with deny / allow / ask lists under `policy.rules`. They're evalua } ``` -A rule matches when **every field it specifies** matches the action (an empty rule matches everything — a deliberate catch-all). Available fields: `kind`, `toolName`, `pathPattern` (glob), `commandPrefix`, `commandPattern` (regex), `mcpServer`. Action kinds are `read_file`, `write_file`, `edit_file`, `delete_file`, `shell`, `network`, `mcp_tool`, `plugin_tool`, `unknown`. +A rule matches when **every field it specifies** matches the action (an empty rule matches everything as a deliberate catch-all). Available fields: `kind`, `toolName`, `pathPattern` (glob), `commandPrefix`, `commandPattern` (regex), `mcpServer`. Action kinds are `read_file`, `write_file`, `edit_file`, `delete_file`, `shell`, `network`, `mcp_tool`, `plugin_tool`, `unknown`. ## Critical denies (always win) -These fire in **every** mode, including `bypassPermissions` — they have no safe override: +These fire in **every** mode, including `bypassPermissions`. They have no safe override: -- **Destructive shell** — commands like `rm -rf /` are blocked. -- **Private-key reads** — reading an SSH/PEM private-key path is blocked. -- **Unregistered MCP servers** — an `mcp__*` call to a server not in your config is blocked. +- **Destructive shell**: commands like `rm -rf /` are blocked. +- **Private-key reads**: reading an SSH/PEM private-key path is blocked. +- **Unregistered MCP servers**: an `mcp__*` call to a server not in your config is blocked. ## The run ledger -With `--log`, tsforge records every tool call, its policy decision (allow / ask / deny, with the matched rule and a risk level), the model's reasoning, and each gate verdict as JSONL under `~/.tsforge/logs/`. [`tsforge trace`](/observability/trace/) turns that ledger into a one-screen summary — deterministically, with no model call — so you can audit exactly what ran and why. +With `--log`, tsforge records every tool call, its policy decision (allow / ask / deny, with the matched rule and a risk level), the model's reasoning, and each gate verdict as JSONL under `~/.tsforge/logs/`. [`tsforge trace`](/observability/trace/) turns that ledger into a one-screen summary. Deterministically, with no model call, you can audit exactly what ran and why. → [tsforge.config.json](/guardrails/config/) · [Trace a run](/observability/trace/) · [Plan mode](/cli/plan-mode/) diff --git a/apps/docs/src/content/docs/guardrails/rule-packs.mdx b/apps/docs/src/content/docs/guardrails/rule-packs.mdx index 9a432d30..28ec1efc 100644 --- a/apps/docs/src/content/docs/guardrails/rule-packs.mdx +++ b/apps/docs/src/content/docs/guardrails/rule-packs.mdx @@ -21,7 +21,6 @@ These load without waiting for a dependency match: | ID | What it covers | | --- | --- | -| `generic-ts` | Core TypeScript safety rules for every project (the bundled ESLint safety config) | | `env-access` | Validated env access, no `process.exit` in libraries | | `module-boundaries` | Layering, no React in services | | `code-flow` | Deterministic time/random, early returns | @@ -29,7 +28,7 @@ These load without waiting for a dependency match: | `security` | Command injection, ReDoS, DOM XSS, silent catch blocks, no tokens in storage | | `runtime-boundaries` | Open redirects, SSRF fetches, prototype pollution, webhook verify, upload limits | -`generic-ts` runs on every project alongside `tsc`; stack detection layers framework-specific packs (`react`, `elysia`, `nextjs`, …) on top. +These run on every project alongside `tsc`; stack detection layers framework packs (`react-component-architecture`, `elysia`, `nextjs`, …) on top. (`generic-ts` and `react` in the list below are **detection labels only**. They name a stack for reporting and carry no rules of their own.) ## Pack list @@ -55,6 +54,18 @@ These load without waiting for a dependency match: Every implemented rule with severity and fix text: [Rule catalog](/reference/rules-catalog/) (grouped by adoption tier). +## Bring your own rules + +You are not limited to the built-in packs. Point `tsforge.config.json` at your own ESLint rule-pack module and tsforge loads it into the gate alongside everything above. No fork required: + +```json +{ + "plugins": [{ "path": "./tsforge-rules/index.ts" }] +} +``` + +Your pack's rules run on every validation, exactly like the built-ins. They can't shadow a built-in rule, and a name collision is reported at startup. This is how you enforce a company's or repo's own conventions through tsforge. See [Config & external plugins](/guardrails/config/) for the full shape. + ## Web stacks and accessibility When stack packs load on a web project, the bundled web ESLint config also enables **jsx-a11y** rules on `.tsx` files (alt text, button types, label associations, and similar). This applies to all web stacks, not just Next.js. @@ -68,8 +79,8 @@ When the `react-component-architecture` pack is enabled (React in `package.json` | `no-jsx-computation` | `.map()` / `.filter()` / arithmetic / chained logic inside JSX `{…}` | | `no-state-in-component-body` | `useState`, `useEffect`, etc. directly in component `.tsx` files | | `no-inline-jsx-functions` | Inline arrow/function handlers in JSX attributes | -| `no-loading-text-use-skeleton` | `"Loading…"` text / spinners in loading branches — render a `` instead | -| `component-file-purity` | A component `.tsx` holds **only** imports + the component — no inline types, constants, or helper functions | +| `no-loading-text-use-skeleton` | `"Loading…"` text / spinners in loading branches. Render a `` instead | +| `component-file-purity` | A component `.tsx` holds **only** imports + the component. No inline types, constants, or helper functions | | `component-folder-structure` | A component `.tsx` must live in `src/views//components/`, be the view root `src/views//index.tsx`, or a shared primitive in `src/components/ui/` | | `no-cross-feature-imports` | Cross-feature runtime imports | diff --git a/apps/docs/src/content/docs/guardrails/stack-detection.mdx b/apps/docs/src/content/docs/guardrails/stack-detection.mdx index 814ec962..39c81ad3 100644 --- a/apps/docs/src/content/docs/guardrails/stack-detection.mdx +++ b/apps/docs/src/content/docs/guardrails/stack-detection.mdx @@ -29,6 +29,7 @@ Optional file at your repo root: [tsforge.config.json](/guardrails/config/). | `packs.include` | Turn on extra pack IDs after detection | | `packs.exclude` | Turn off specific pack IDs | | `rules` | Set individual rules to `error`, `warn`, or `off` | +| `plugins` | Load **your own** ESLint rule-packs on top of the detected ones. See [Config & external plugins](/guardrails/config/). | ## Full pack list diff --git a/apps/docs/src/content/docs/index.mdx b/apps/docs/src/content/docs/index.mdx index ef26ec05..831a1dd9 100644 --- a/apps/docs/src/content/docs/index.mdx +++ b/apps/docs/src/content/docs/index.mdx @@ -8,9 +8,9 @@ head: - tag: meta attrs: property: og:title - content: "tsforge — Ship TypeScript that can't cheat" + content: "tsforge: Ship TypeScript that can't cheat" - tag: meta attrs: name: twitter:title - content: "tsforge — Ship TypeScript that can't cheat" + content: "tsforge: Ship TypeScript that can't cheat" --- diff --git a/apps/docs/src/content/docs/inference/adapter.mdx b/apps/docs/src/content/docs/inference/adapter.mdx index 2a926ed3..c9ceb1d5 100644 --- a/apps/docs/src/content/docs/inference/adapter.mdx +++ b/apps/docs/src/content/docs/inference/adapter.mdx @@ -12,7 +12,7 @@ Configure models in `~/.tsforge/models.json`. Override per run with environment | Setting | Default | | --- | --- | | Base URL | `http://localhost:8000/v1` | -| Model | `qwen3.6-27b` | +| Model | `deepseek-v4-flash` | | Max tokens | 16384 | These defaults match tsforge's dev setup. Replace `baseUrl` and `model` with yours. The gate and guardrails stay the same regardless of provider. diff --git a/apps/docs/src/content/docs/inference/models-json.mdx b/apps/docs/src/content/docs/inference/models-json.mdx index 7c8b824d..d17909d8 100644 --- a/apps/docs/src/content/docs/inference/models-json.mdx +++ b/apps/docs/src/content/docs/inference/models-json.mdx @@ -1,6 +1,6 @@ --- title: models.json (any provider) -description: Configure any OpenAI-compatible model — local qwen, DeepSeek, OpenAI o-series, or a custom gateway — from one registry file. No env vars needed. +description: Configure any OpenAI-compatible model (local vLLM, DeepSeek, OpenAI o-series, or a custom gateway) from one registry file. No env vars needed. --- tsforge reads its model registry from **`~/.tsforge/models.json`** (or `$TSFORGE_HOME/.tsforge/models.json`). It's the one place you configure providers; you never need environment variables. `active` names the entry to use; `/model ` switches live. @@ -21,7 +21,7 @@ tsforge reads its model registry from **`~/.tsforge/models.json`** (or `$TSFORGE ``` :::caution -The entry **key** (`"deepseek"`) is just a local label. The field that's sent to the API is **`model`**. Pointing `baseUrl` at a cloud provider while leaving `model` as the qwen default is the #1 mistake — tsforge warns at startup when it detects this. +The entry **key** (`"deepseek"`) is just a local label. The field that's sent to the API is **`model`**. Pointing `baseUrl` at a cloud provider while leaving `model` as the local default is the #1 mistake. tsforge warns at startup when it detects this. ::: ## Fields @@ -30,14 +30,14 @@ The entry **key** (`"deepseek"`) is just a local label. The field that's sent to | --- | --- | --- | | `baseUrl` | yes | OpenAI-compatible API root, e.g. `https://api.deepseek.com/v1`. `/chat/completions` is appended (and not double-appended). | | `model` | yes | The model id sent to the API. | -| `apiKey` / `apiKeyEnv` | no | Inline key, or the name of an env var holding it (preferred — keeps secrets out of the file). | -| `reasoning` | no | Provider thinking dialect: `qwen` (default), `deepseek`, `openai`, `none` — see below. **Auto-detected as `deepseek`** when `baseUrl` or `model` contains "deepseek", so a DeepSeek model usually needs no explicit value. | +| `apiKey` / `apiKeyEnv` | no | Inline key, or the name of an env var holding it (preferred: keeps secrets out of the file). | +| `reasoning` | no | Provider thinking dialect: `qwen` (default), `deepseek`, `openai`, `none`. See below. **Auto-detected as `deepseek`** when `baseUrl` or `model` contains "deepseek", so a DeepSeek model usually needs no explicit value. | | `reasoningEffort` | no | `low` / `medium` / `high` for `deepseek`/`openai`. | | `contextWindow` | no | Tokens, for the status gauge + auto-compaction. Auto-detected from `/models` when omitted. | | `maxTokens` | no | Output-token cap (default 16384). | -| `extraBody` | no | Arbitrary JSON merged into the request body (overrides built-ins) — the escape hatch for any provider param. | +| `extraBody` | no | Arbitrary JSON merged into the request body (overrides built-ins). The escape hatch for any provider param. | | `extraHeaders` | no | Arbitrary request headers; `${VAR}` values are interpolated from the environment. | -| `imageApi` | no | For an `imageGen` capability entry only: wire shape — `chat-modalities` (default; `/chat/completions` with `modalities:["image","text"]`, e.g. OpenRouter/Gemini) or `images-generations` (OpenAI `/images/generations`). | +| `imageApi` | no | For an `imageGen` capability entry only: wire shape. `chat-modalities` (default; `/chat/completions` with `modalities:["image","text"]`, e.g. OpenRouter/Gemini) or `images-generations` (OpenAI `/images/generations`). | ## Reasoning dialects @@ -46,18 +46,18 @@ Providers express "thinking" differently. `reasoning` picks the wire shape: | `reasoning` | Sends | Notes | | --- | --- | --- | | `qwen` (default) | `chat_template_kwargs.enable_thinking`, `thinking_token_budget` | vLLM/Qwen. | -| `deepseek` | `thinking: { type: "enabled"\|"disabled" }`, `reasoning_effort` | Omits `tool_choice` entirely (DeepSeek's thinking mode rejects an explicit one) — the model still gets the tools and decides. | +| `deepseek` | `thinking: { type: "enabled"\|"disabled" }`, `reasoning_effort` | Omits `tool_choice` entirely (DeepSeek's thinking mode rejects an explicit one). The model still gets the tools and decides. | | `openai` | `reasoning_effort` | Uses `max_completion_tokens` and omits `temperature` (o-series rejects both). | | `none` | — | No reasoning params. | -tsforge **auto-detects** the `deepseek` dialect when `baseUrl`/`model` contains "deepseek", so DeepSeek thinking models work out of the box — the `reasoning_content` from each turn is replayed on the next request (DeepSeek 400s otherwise). Set `reasoning` explicitly only when a DeepSeek-compatible gateway hides that string in its URL/model name. +tsforge **auto-detects** the `deepseek` dialect when `baseUrl`/`model` contains "deepseek", so DeepSeek thinking models work out of the box. The `reasoning_content` from each turn is replayed on the next request (DeepSeek 400s otherwise). Set `reasoning` explicitly only when a DeepSeek-compatible gateway hides that string in its URL/model name. ## Examples -### Local qwen (vLLM) — the default +### Local model (vLLM): the default ```jsonc -{ "baseUrl": "http://localhost:8000/v1", "model": "qwen3.6-27b", "thinking": true } +{ "baseUrl": "http://localhost:8000/v1", "model": "deepseek-v4-flash", "thinking": true } ``` ### DeepSeek @@ -86,7 +86,7 @@ tsforge **auto-detects** the `deepseek` dialect when `baseUrl`/`model` contains ## Extra capabilities (vision, image generation) -The primary chat model is often text-only (e.g. a local DeepSeek). When it can't read or generate images, route **just those capabilities** to a separate backend with a top-level `capabilities` block — a sibling of `active`/`models`, **not** inside `models`. Each value names a `models` entry, so the vision/image backend reuses the same endpoint config (key resolution, headers) as any chat model. Absent → the capability (and its `read_image` / `generate_image` tool + the drag/paste/`@` attachment UX) stays off. +The primary chat model is often text-only (e.g. a local DeepSeek). When it can't read or generate images, route **just those capabilities** to a separate backend with a top-level `capabilities` block (a sibling of `active`/`models`, **not** inside `models`). Each value names a `models` entry, so the vision/image backend reuses the same endpoint config (key resolution, headers) as any chat model. When absent, the capability (and its `read_image` / `generate_image` tool, the drag/paste/`@` attachment UX) stays off. ```jsonc { @@ -103,7 +103,7 @@ The primary chat model is often text-only (e.g. a local DeepSeek). When it can't } ``` -How it works: an attached image is sent to the `vision` backend and only its **text** description enters the conversation (the chat model stays text-only). `generate_image` calls the `imageGen` backend, saves the result under `.tsforge/images/`, and previews it inline in iTerm2. When local hardware arrives, point `capabilities.vision`/`imageGen` at a local entry — no code change. +How it works: an attached image is sent to the `vision` backend and only its **text** description enters the conversation (the chat model stays text-only). `generate_image` calls the `imageGen` backend, saves the result under `.tsforge/images/`, and previews it inline in iTerm2. When local hardware arrives, point `capabilities.vision`/`imageGen` at a local entry. No code change needed. Env overrides (an ad-hoc backend without editing the file): `TSFORGE_VISION_BASE_URL` / `TSFORGE_VISION_MODEL` / `TSFORGE_VISION_API_KEY` and the `TSFORGE_IMAGE_*` equivalents (+ `TSFORGE_IMAGE_API`). See the [attachment UX](/cli/interactive/) and the [flags reference](/reference/flags/). @@ -111,7 +111,7 @@ Env overrides (an ad-hoc backend without editing the file): `TSFORGE_VISION_BASE `apiKeyEnv` is the **name of an environment variable**, not the key. Putting a literal `sk-...` as its value makes tsforge look up an env var named `sk-...` → no auth. Use `apiKey` for an inline key. ::: -### Any other provider — `extraBody` / `extraHeaders` +### Any other provider: `extraBody` / `extraHeaders` When a provider needs a param or auth scheme tsforge doesn't model, express it directly. `extraBody` is merged **last** (so it overrides anything), and `extraHeaders` can set a non-Bearer scheme: diff --git a/apps/docs/src/content/docs/integrations/mcp.mdx b/apps/docs/src/content/docs/integrations/mcp.mdx index 7e8e15c5..0c623fec 100644 --- a/apps/docs/src/content/docs/integrations/mcp.mdx +++ b/apps/docs/src/content/docs/integrations/mcp.mdx @@ -1,6 +1,6 @@ --- title: MCP servers -description: Give the agent extra tools from external Model Context Protocol servers — live library docs, search, and more. +description: Give the agent extra tools from external Model Context Protocol servers. Live library docs, search, and more. --- tsforge can connect to external **MCP (Model Context Protocol) servers** and offer their tools to the agent during an interactive session. The classic use is **[Context7](https://github.com/upstash/context7)** for up-to-date library documentation, but any stdio MCP server works. @@ -39,13 +39,13 @@ On startup tsforge connects each server, lists its tools, and advertises them to MCP fits tsforge's deterministic, local-first design as an **opt-in context source**: -- MCP tools **never touch your editable scope** and **cannot satisfy or bypass the acceptance gate** — they add context, they don't certify "done". +- MCP tools **never touch your editable scope** and **cannot satisfy or bypass the acceptance gate**. They add context, they don't certify "done". - Because they don't write the workspace, they remain available in **plan mode** (e.g. look up docs while planning). -- A server that fails to connect, crashes, or times out is **reported and skipped** — it can never block a session from starting or wedge the loop. A failed tool call comes back as an error string the model can react to. +- A server that fails to connect, crashes, or times out is **reported and skipped**. It can never block a session from starting or wedge the loop. A failed tool call comes back as an error string the model can react to. ## Limits today -- **stdio transport only** — `http` entries are reported and skipped (HTTP/SSE is planned). +- **stdio transport only**: `http` entries are reported and skipped (HTTP/SSE is planned). - MCP tools are offered in the **interactive CLI**; the headless eval path does not load them. - Connected servers are child processes; they exit with the tsforge process. diff --git a/apps/docs/src/content/docs/integrations/web-tools.mdx b/apps/docs/src/content/docs/integrations/web-tools.mdx index b029015b..fc8a7159 100644 --- a/apps/docs/src/content/docs/integrations/web-tools.mdx +++ b/apps/docs/src/content/docs/integrations/web-tools.mdx @@ -1,21 +1,21 @@ --- title: Web research (no API keys) -description: "Opt-in web_fetch, web_search, package_info, package_docs, and web_browse tools — no paid search/browser API, no required service key." +description: "Opt-in web_fetch, web_search, package_info, package_docs, and web_browse tools. No paid search/browser API, no required service key." --- -Interactive sessions get read-only internet research tools **on by default** (an assistant that can't look things up is silly); toggle them under **Web tools** in [`/config`](/cli/interactive/). They're built for **no required API keys and no paid vendor coupling**: npm metadata comes from the configured registry, search defaults to DuckDuckGo's keyless HTML endpoint, pages are extracted locally, and browser rendering uses local Playwright/Chromium when available. One-shot and eval runs stay **off** unless you set `TSFORGE_WEB=1`, so headless sweeps have no network reach beyond your model endpoint. +Interactive sessions get read-only internet research tools **on by default** (an assistant that can't look things up is silly); toggle them under **Web tools** in [`/config`](/cli/interactive/). They're built for no required API keys and no paid vendor coupling: npm metadata comes from the configured registry, search defaults to DuckDuckGo's keyless HTML endpoint, pages are extracted locally, and browser rendering uses local Playwright/Chromium when available. One-shot and eval runs stay **off** unless you set `TSFORGE_WEB=1`, so headless sweeps have no network reach beyond your model endpoint. ```bash -TSFORGE_WEB=1 tsforge "update the deprecated API call — check the library's current docs" +TSFORGE_WEB=1 tsforge "update the deprecated API call (check the library's current docs)" ``` ## The tools -- **`package_info`** — read current npm package metadata from the configured registry: latest dist-tag, versions, deprecation status, peer deps, homepage, repository, and dependency names. -- **`package_docs`** — read package docs without a docs API: local `node_modules` README/package metadata/types first, then the npm registry README when needed. -- **`web_fetch`** — retrieve a URL and return readable markdown. The page is fetched and the content extracted **on-machine** (no third-party reader API), so you get clean text without shipping the URL to a service. -- **`web_browse`** — open a public URL in local headless Chromium via Playwright and return rendered visible text, final URL, title, and links. Use it for JavaScript-rendered docs that `web_fetch` cannot see. No hosted browser service. -- **`web_search`** — discover sources for a query. It returns public result URLs only, supports `recency` (`day`, `month`, `year`), `domains` for official-site scoping, and `maxResults` up to 20. Defaults to DuckDuckGo's keyless HTML endpoint. +- **`package_info`**: read current npm package metadata from the configured registry. Latest dist-tag, versions, deprecation status, peer deps, homepage, repository, and dependency names. +- **`package_docs`**: read package docs without a docs API. Local `node_modules` README/package metadata/types first, then the npm registry README when needed. +- **`web_fetch`**: retrieve a URL and return readable markdown. The page is fetched and the content extracted **on-machine** (no third-party reader API), so you get clean text without shipping the URL to a service. +- **`web_browse`**: open a public URL in local headless Chromium via Playwright and return rendered visible text, final URL, title, and links. Use it for JavaScript-rendered docs that `web_fetch` cannot see. No hosted browser service. +- **`web_search`**: discover sources for a query. It returns public result URLs only, supports `recency` (`day`, `month`, `year`), `domains` for official-site scoping, and `maxResults` up to 20. Defaults to DuckDuckGo's keyless HTML endpoint. SearXNG is **not bundled or started by tsforge**. If you run your own SearXNG service (Docker, Compose, or any deployment), point `TSFORGE_SEARXNG_URL` at it for full privacy/control: @@ -31,7 +31,7 @@ Check the current TanStack Query docs before changing this hook. Use domain-scop ## When they're active -The tools are read-only and offline-safe. Interactive sessions enable them by default, but one-shot and eval runs stay offline unless you opt in — so headless sweeps are deterministic. Under a policy mode that denies `network` (e.g. `ci`), the tools are unavailable even with the flag set. See [Permissions & policy](/guardrails/policy/). +The tools are read-only and offline-safe. Interactive sessions enable them by default, but one-shot and eval runs stay offline unless you opt in, so headless sweeps are deterministic. Under a policy mode that denies `network` (e.g. `ci`), the tools are unavailable even with the flag set. See [Permissions & policy](/guardrails/policy/). | Env var | Default | Effect | | --- | --- | --- | diff --git a/apps/docs/src/content/docs/loop/gate-floor.mdx b/apps/docs/src/content/docs/loop/gate-floor.mdx index c32772a4..e93ee5e3 100644 --- a/apps/docs/src/content/docs/loop/gate-floor.mdx +++ b/apps/docs/src/content/docs/loop/gate-floor.mdx @@ -25,7 +25,7 @@ You can override anytime: | --- | --- | | `/gate bun test` | use your command for the rest of the session | | `tsforge "task" --accept "bun test"` | one-shot run with a custom gate | -| `--strict-floor-only` | strict tsc + ESLint only — do **not** append the project's tests | +| `--strict-floor-only` | strict tsc + ESLint only. Do **not** append the project's tests | | `--no-gate` | skip auto-detection (advanced) | See [When the gate fails](/loop/validation/) for what happens when the gate fails and how the repair loop stops. @@ -54,7 +54,7 @@ Three common cases: ### Where the overlay lives -The strict overlay is a **cache artifact, not a project file**: it lives under `.tsforge/` (tsforge's own folder, alongside `.tsforge/rules.json`), never as a `tsforge.tsconfig.json` in your repo root. tsforge also drops a scoped `.tsforge/.gitignore` so the overlay stays out of git — created only if you do not already have one, so a `.tsforge/.gitignore` you wrote (e.g. to track `rules.json`) is never overwritten. +The strict overlay is a **cache artifact, not a project file**: it lives under `.tsforge/` (tsforge's own folder, alongside `.tsforge/rules.json`), never as a `tsforge.tsconfig.json` in your repo root. tsforge also drops a scoped `.tsforge/.gitignore` so the overlay stays out of git. It's created only if you do not already have one, so a `.tsforge/.gitignore` you wrote (e.g. to track `rules.json`) is never overwritten. ## TypeScript check (`tsc`) @@ -68,20 +68,20 @@ After typecheck, tsforge runs **its own** ESLint setup. It does **not** run your Why separate? tsforge ships rules tuned for AI-written code and your stack (React, Drizzle, Elysia, etc.). Those rules come from [Rule packs](/guardrails/rule-packs/) enabled by [Stack detection](/guardrails/stack-detection/). Tune them with a [profile](/guardrails/config/#profiles) or per-rule overrides in [tsforge.config.json](/guardrails/config/). -The core ESLint config is **syntactic-only** (no type-aware rules) so it runs on any `.ts` file without the target's full type graph. It also caps **cognitive complexity at 20** and **nesting depth at 4** — a sprawling, deeply-nested function fails the gate until it is split into named helpers. +The core ESLint config is **syntactic-only** (no type-aware rules) so it runs on any `.ts` file without the target's full type graph. It also caps **cognitive complexity at 20** and **nesting depth at 4**. A sprawling, deeply-nested function fails the gate until it is split into named helpers. -On the **strict** [profile](/guardrails/config/#profiles) — any project with a `tsconfig.json` — tsforge adds a **type-aware pass** that catches what `tsc --strict` and the syntactic rules can't: +On the **strict** [profile](/guardrails/config/#profiles) (any project with a `tsconfig.json`), tsforge adds a **type-aware pass** that catches what `tsc --strict` and the syntactic rules can't: -- **Async correctness** — `no-floating-promises`, `no-misused-promises`: a dropped `await` is a gate error. -- **Implicit-`any` containment** — the `no-unsafe-*` family. `no-explicit-any` bans the literal `any` token, but it cannot see `any` that leaks in from an untyped boundary (`JSON.parse`, `await res.json()`, an untyped dependency), which `tsc` propagates silently. These rules force you to validate the boundary, then the data is typed. +- **Async correctness**: `no-floating-promises`, `no-misused-promises`. A dropped `await` is a gate error. +- **Implicit-`any` containment**: the `no-unsafe-*` family. `no-explicit-any` bans the literal `any` token, but it cannot see `any` that leaks in from an untyped boundary (`JSON.parse`, `await res.json()`, an untyped dependency), which `tsc` propagates silently. These rules force you to validate the boundary, then the data is typed. -The default `recommended` profile keeps the gate syntactic-only — faster, and these type-aware rules can be noisy on existing code. Set `"profile": "strict"` in [tsforge.config.json](/guardrails/config/) to turn the pass on. Web-app gates run it regardless of profile. +The default `recommended` profile keeps the gate syntactic-only for speed, since type-aware rules can be noisy on existing code. Set `"profile": "strict"` in [tsforge.config.json](/guardrails/config/) to turn the pass on. Web-app gates run it regardless of profile. -Baseline stylistic rules (all stacks) include blank lines before `return` and around block-like statements via `@stylistic/padding-line-between-statements`. That rule is **auto-fixed by `eslint --fix`**, not Prettier — Prettier does not enforce semantic blank lines. Non-web sessions run the same eslint + prettier janitor (`buildCoreFix`) before the gate, so the blank line is inserted without model turns. +Baseline stylistic rules (all stacks) include blank lines before `return` and around block-like statements via `@stylistic/padding-line-between-statements`. That rule is **auto-fixed by `eslint --fix`**, not Prettier. Prettier does not enforce semantic blank lines. Non-web sessions run the same eslint + prettier janitor (`buildCoreFix`) before the gate, so the blank line is inserted without model turns. ## Your tests run too -Type-check and lint prove the code is well-formed; your **tests** prove it's correct. So the auto-gate runs them as well — "green" means the strict floor **and** your tests pass. +Type-check and lint prove the code is well-formed; your **tests** prove it's correct. So the auto-gate runs them as well. "Green" means the strict floor **and** your tests pass. tsforge finds the test command automatically: @@ -89,7 +89,7 @@ tsforge finds the test command automatically: - otherwise, if the project has `*.test.ts` / `*.spec.ts` files → `bun test` - neither (a greenfield app with no tests yet) → nothing is appended; the gate stays at the strict floor -The npm-init placeholder (`"echo \"Error: no test specified\" && exit 1"`) is ignored — it never counts as "has tests." Tests run **last**, after the cheap static checks, so a type or lint error fails fast without waiting on a test run. +The npm-init placeholder (`"echo \"Error: no test specified\" && exit 1"`) is ignored. It never counts as "has tests." Tests run **last**, after the cheap static checks, so a type or lint error fails fast without waiting on a test run. Opt out with **`--strict-floor-only`** when you want the type/lint floor without running tests every cycle (e.g. a slow suite). An explicit `--accept` gate always wins. @@ -99,16 +99,16 @@ Two extra gate steps prove the work is real, not just well-formed. Both default | Env var | Adds | | --- | --- | -| `TSFORGE_COVERAGE=` | runs the suite with coverage and fails if the **weaker of line/function coverage** is below `` — a green suite that never calls the new code no longer passes | -| `TSFORGE_BOOT=""` | actually **boots the server** (`TSFORGE_BOOT_URL`, default `http://localhost:3000/`) and fails unless it answers without a 5xx — catches boot-time crashes that type-check and lint clean | -| `TSFORGE_PROPTEST=1` | **property-fuzzes** every exported function: derives `fast-check` inputs from its TypeScript parameter types and fails if it throws on any valid input — catches edge-case crashes (empty array, NaN, out-of-range) that example tests miss. Best for pure functions. | +| `TSFORGE_COVERAGE=` | runs the suite with coverage and fails if the **weaker of line/function coverage** is below ``. A green suite that never calls the new code no longer passes | +| `TSFORGE_BOOT=""` | actually **boots the server** (`TSFORGE_BOOT_URL`, default `http://localhost:3000/`) and fails unless it answers without a 5xx. Catches boot-time crashes that type-check and lint clean | +| `TSFORGE_PROPTEST=1` | **property-fuzzes** every exported function: derives `fast-check` inputs from its TypeScript parameter types and fails if it throws on any valid input. Catches edge-case crashes (empty array, NaN, out-of-range) that example tests miss. Best for pure functions. | Coverage checks **function** coverage as well as lines, so a one-line export that's never exercised can't hide behind a "line executed at declaration" count. ## Full-stack (BoringStack) builds use the project's own gate A web app built via [`tsforge scaffold`](/scaffold/boringstack/) doesn't get a -tsforge-invented gate — "done" is **BoringStack's own** composed gate, run exactly as +tsforge-invented gate. Instead, "done" is **BoringStack's own** composed gate, run exactly as a developer runs it: ``` @@ -120,14 +120,14 @@ Two things make this honest rather than brittle in the [greenfield loop](/loop/g - **Auto-fix before the gate.** After the model writes, the harness runs the project's own `bun run format` (prettier) then `bun run lint:fix` (eslint `--fix`) over both apps. Those classes are deterministic and 100% auto-fixable, so they never cost the - model a gate attempt — a developer gets them on save; the model should too. Only + model a gate attempt. A developer gets them on save; the model should too. Only genuine, non-auto-fixable violations reach the gate as feedback. - **Baseline-aware differential gate.** The gate is captured once on the *pristine* scaffold before any model work. A feature then passes when it introduces **no new - failures** — a pre-existing base-suite/scaffold defect the model is frozen out of + failures**. A pre-existing base-suite/scaffold defect the model is frozen out of can never wedge the build, and the model's feedback shows only the failures *it* caused. A red baseline is surfaced loudly (the scaffold itself should be fixed); it - is not silently tolerated. This is not a gate relaxation — the full gate still runs + is not silently tolerated. This is not a gate relaxation. The full gate still runs on everything the model touches or newly breaks. ## Two kinds of checking diff --git a/apps/docs/src/content/docs/loop/greenfield.mdx b/apps/docs/src/content/docs/loop/greenfield.mdx index 2bbc56b2..eafca40f 100644 --- a/apps/docs/src/content/docs/loop/greenfield.mdx +++ b/apps/docs/src/content/docs/loop/greenfield.mdx @@ -1,6 +1,6 @@ --- title: Greenfield builds -description: Build a whole app feature-by-feature with a filesystem-tracked checklist, a planner/generator/evaluator split, and the same gate + browser + judge verification used everywhere else. +description: Build a whole app feature-by-feature with a filesystem-tracked checklist, a planner/generator/evaluator split, and the same gate + reachability + judge verification used everywhere else. --- Most of tsforge drives **one change to green**. Greenfield mode drives a **whole build to green**, one feature at a time, keeping its state on disk so a long run can be interrupted and resumed without losing the plot. @@ -11,27 +11,27 @@ tsforge --greenfield "build a kanban board" --accept "bun run build" ## How it works -1. **Plan** — a planner model turns your one-line goal into a high-level spec and a flat feature checklist (sprints, not file-level steps). Written to `.tsforge/greenfield/`. -2. **Loop** — for the first unfinished feature: the work model implements it, then the **layered evaluator** verifies it. On success the feature is ticked; the loop moves on. -3. **Stop** — when every feature is verified (`done`), or a single feature exhausts its attempts (`stuck`) so a non-converging feature can't wedge the whole build. +1. **Plan**: a planner model turns your one-line goal into a high-level spec and a flat feature checklist (sprints, not file-level steps). Written to `.tsforge/greenfield/`. +2. **Loop**: for the first unfinished feature, the work model implements it, then the **layered evaluator** verifies it. On success the feature is ticked; the loop moves on. +3. **Stop**: when every feature is verified (`done`), or a single feature exhausts its attempts (`stuck`) so a non-converging feature can't wedge the whole build. The state lives in the repo, not the model's context: | File | Purpose | | --- | --- | | `.tsforge/greenfield/spec.md` | The planner's high-level spec | -| `.tsforge/greenfield/features.json` | The checklist (`{id, desc, passes, attempts}`) — the source of truth | +| `.tsforge/greenfield/features.json` | The checklist (`{id, desc, passes, attempts}`). The source of truth. | | `.tsforge/greenfield/progress.md` | Human-readable status | Because `features.json` is the source of truth, re-running the same command **resumes** from the last verified feature. ## The layered evaluator -Each feature passes only if all three layers agree, checked cheapest-first and short-circuiting on the first failure: +Each feature passes only if every stage agrees, checked cheapest-first and short-circuiting on the first failure: -1. **Gate** — the deterministic build gate (`--accept`: tsc / eslint / tests). The authority for "done". -2. **Browser** — optional per-feature interaction steps via the Playwright oracle (skipped gracefully when Playwright isn't installed, so it never blocks). -3. **Judge** — a harsh, reject-by-default quality review that sees **only the built code**, never how it was produced. +1. **Gate**: the deterministic build gate (`--accept`: tsc / eslint / tests / build). The authority for "done". +2. **Reachability**: for a web build (BoringStack), the feature's route must actually be served (an HTTP fetch), and its required test ids must be present in the code (a static contract check). No headless browser is run. +3. **Judge**: a harsh, reject-by-default quality review that sees **only the built code**, never how it was produced. That last point is a deliberate rule: the evaluator never sees the generator's reasoning or tool trace, so it judges the result, not the persuasion behind it. @@ -40,14 +40,14 @@ That last point is a deliberate rule: the evaluator never sees the generator's r A web app scaffolded with [`tsforge scaffold`](/scaffold/boringstack/) rides this same loop, with one resource (entity) per feature and a clear division of labor: -- **The harness runs the generators + wiring** — `bun run new:resource` / `new:feature`, +- **The harness runs the generators + wiring**: `bun run new:resource` / `new:feature`, the deterministic edits that wire the resource into the route tree, app, swagger, and the test-helper's schema re-export, then `db:push` to sync the schema. -- **The model fills the domain** — real fields, service logic, and the required test +- **The model fills the domain**: real fields, service logic, and the required test siblings, scoped to just that resource's files and frozen once it passes. The gate is BoringStack's own `validate`, run **baseline-aware and differential** with a -prettier + `eslint --fix` auto-fix pass first — see +prettier + `eslint --fix` auto-fix pass first. See [How tsforge builds the gate](/loop/gate-floor/#full-stack-boringstack-builds-use-the-projects-own-gate). ## Role-based model routing @@ -71,7 +71,7 @@ tsforge run kanban "build a kanban board" ## Unattended runs & scheduling -Greenfield runs are long and headless-friendly. There's no built-in scheduler — wire one with your OS: +Greenfield runs are long and headless-friendly. There's no built-in scheduler; wire one with your OS: ```bash # nightly build attempt, pinging you when it finishes or gets stuck diff --git a/apps/docs/src/content/docs/loop/scout.mdx b/apps/docs/src/content/docs/loop/scout.mdx index e88b776a..a248f464 100644 --- a/apps/docs/src/content/docs/loop/scout.mdx +++ b/apps/docs/src/content/docs/loop/scout.mdx @@ -1,9 +1,9 @@ --- title: Pre-edit scout -description: Before editing existing code, tsforge can seed the run with a deterministic caller blast-radius — who depends on the files about to change — so a small model edits with awareness instead of blind. +description: Before editing existing code, tsforge can seed the run with a deterministic caller blast-radius (who depends on the files about to change) so a small model edits with awareness instead of blind. --- -A small local model editing brownfield code can't see what depends on it, so it breaks callers it never read — and then burns repair turns fixing the regressions. The **scout** removes that blind spot: before the first edit, it hands the model the **blast radius** of the files it's about to change. +A small local model editing brownfield code can't see what depends on it, so it breaks callers it never read and then burns repair turns fixing the regressions. The **scout** removes that blind spot: before the first edit, it hands the model the **blast radius** of the files it's about to change. ```bash tsforge "rename area() to rect()" --files "src/geometry.ts" --scout @@ -11,10 +11,10 @@ tsforge "rename area() to rect()" --files "src/geometry.ts" --scout ## What it adds -For each editable file that already exists, scout computes — **type-exactly, from the TypeScript LanguageService** — who calls that file's exports, and seeds it into the opening context: +For each editable file that already exists, scout computes (**type-exactly, from the TypeScript LanguageService**) who calls that file's exports, and seeds it into the opening context: ``` -Blast radius — who calls the files you're about to change (type-exact; check these for regressions before editing): +Blast radius: who calls the files you're about to change (type-exact; check these for regressions before editing): src/geometry.ts: - area → src/floorplan.ts:12; src/report.ts:48 ``` @@ -23,14 +23,14 @@ That's the same caller signal [`tsforge review`](/cli/review/) uses, applied **b ## Deterministic by design -Scout makes **no model call** — it's pure compiler facts (the divergence from Codebuff's LLM file-picker). It's: +Scout makes **no model call**: it's pure compiler facts (the divergence from Codebuff's LLM file-picker). It's: -- **Opt-in** — off unless you pass `--scout` (or a [recipe](/cli/recipes/) sets `"scout": true`). Runs stay fast by default. -- **Brownfield only** — it does nothing on a from-scratch build (there are no callers yet) or when there's no `tsconfig.json`. A non-TS editable file (`.json`, `.md`) is skipped, never a crash. -- **Bounded** — capped in files probed and total size; if it hits the size cap it trims to the last whole line and appends `… [scout truncated]`, so the model never sees a half-written call site. +- **Opt-in**: off unless you pass `--scout` (or a [recipe](/cli/recipes/) sets `"scout": true`). Runs stay fast by default. +- **Brownfield only**: it does nothing on a from-scratch build (there are no callers yet) or when there's no `tsconfig.json`. A non-TS editable file (`.json`, `.md`) is skipped, never a crash. +- **Bounded**: capped in files probed and total size; if it hits the size cap it trims to the last whole line and appends `… [scout truncated]`, so the model never sees a half-written call site. ## Scope -Scout seeds the opening prompt of a **one-shot** drive-to-green run (`tsforge "task" --files … --scout`, or a recipe run with `"scout": true`). Interactive sessions gather context conversationally, so the flag doesn't apply there — passing `--scout` to an interactive session prints a note saying so rather than silently ignoring it. +Scout seeds the opening prompt of a **one-shot** drive-to-green run (`tsforge "task" --files … --scout`, or a recipe run with `"scout": true`). Interactive sessions gather context conversationally, so the flag doesn't apply there. Passing `--scout` to an interactive session prints a note saying so rather than silently ignoring it. → [Review your changes](/cli/review/) · [Recipes](/cli/recipes/) · [Map the repo](/cli/map/) diff --git a/apps/docs/src/content/docs/loop/spec-runner.mdx b/apps/docs/src/content/docs/loop/spec-runner.mdx index b1dc569f..5218723a 100644 --- a/apps/docs/src/content/docs/loop/spec-runner.mdx +++ b/apps/docs/src/content/docs/loop/spec-runner.mdx @@ -29,7 +29,7 @@ Outputs include per-task status (`done`, `stuck`, interrupted) and a final pass/ ```bash bun run eval:spec -TSFORGE_SEED=money TSFORGE_FEATURE_VARIANTS=script \ +TSFORGE_SEED=math TSFORGE_FEATURE_VARIANTS=script \ bun run eval:sweep ``` diff --git a/apps/docs/src/content/docs/loop/validation.mdx b/apps/docs/src/content/docs/loop/validation.mdx index 76d26300..86b5d436 100644 --- a/apps/docs/src/content/docs/loop/validation.mdx +++ b/apps/docs/src/content/docs/loop/validation.mdx @@ -32,14 +32,14 @@ The loop stops when one of these happens: | --- | --- | | **Done** | Gate passed. Task complete. | | **Stalled (no progress)** | The model keeps failing at the **same thing**. tsforge stops and hands back a blocker diagnosis. | -| **Cap (runaway backstop)** | A turn ceiling hit. Rarely reached — the no-progress guard stops first. | +| **Cap (runaway backstop)** | A turn ceiling hit. Rarely reached; the no-progress guard stops first. | tsforge's primary stop is **lack of progress, not a raw turn count**. Two guards detect it: -- **Same-error persistence** — if one specific error (the same `file` + `rule`) survives **5 consecutive** fix cycles, tsforge stops, even if _other_ errors are changing around it. The stop names the blocker: `stuck on no-explicit-any in src/views/Foo/index.tsx after 5 attempts (last: …)`. Interactively, you get that diagnosis and the prompt back — the session stays alive, so you can re-steer. -- **Whole-set stall** — a coarser net: the entire error set unchanged for 6 cycles. +- **Same-error persistence**: if one specific error (the same `file` + `rule`) survives **5 consecutive** fix cycles, tsforge stops, even if _other_ errors are changing around it. The stop names the blocker: `stuck on no-explicit-any in src/views/Foo/index.tsx after 5 attempts (last: …)`. Interactively, you get that diagnosis and the prompt back. The session stays alive, so you can re-steer. +- **Whole-set stall**: a coarser net. The entire error set unchanged for 6 cycles. -The **turn cap** is only a runaway backstop now. Interactive sessions ride a high ceiling (≈250 turns) so long, productive back-and-forth is never cut off; headless/eval runs keep a real cap (40, or 400 for web builds) since no human is present to intervene. +The **turn cap** is only a runaway backstop now. A single high ceiling (1000 turns) shared by every mode, set far above any converging build. It's a crash-guard, not a task limit: what actually stops a run is the no-progress guard above, not the turn count. When the gate fails, tsforge sends structured errors (file, line, rule name, message) back to the model, not a generic failure blob. That is what makes repair workable. diff --git a/apps/docs/src/content/docs/lsp/typescript-server.mdx b/apps/docs/src/content/docs/lsp/typescript-server.mdx index fcecfb0d..01bd3545 100644 --- a/apps/docs/src/content/docs/lsp/typescript-server.mdx +++ b/apps/docs/src/content/docs/lsp/typescript-server.mdx @@ -29,7 +29,7 @@ Offered when tsforge detects real code to explore (existing repo, resumed sessio Navigation and write feedback (instant per-file type diagnostics after each edit) are always on for real work; navigation can be withheld for eval/headless runs with `TSFORGE_NO_LSP_TOOLS=1`. -On existing repos the model is also offered `git_context` — read-only, structured access to history and diffs (scope a fix to what changed). It is git-backed, not part of the language server, so `TSFORGE_NO_LSP_TOOLS` does not affect it; withhold it for eval/headless runs with `TSFORGE_NO_GIT_TOOL=1`. See [Git context](/reference/flags/#git-context). +On existing repos the model is also offered `git_context`: read-only, structured access to history and diffs (scope a fix to what changed). It is git-backed, not part of the language server, so `TSFORGE_NO_LSP_TOOLS` does not affect it; withhold it for eval/headless runs with `TSFORGE_NO_GIT_TOOL=1`. See [Git context](/reference/flags/#git-context). ## Safe auto-fixes diff --git a/apps/docs/src/content/docs/observability/metrics.mdx b/apps/docs/src/content/docs/observability/metrics.mdx index 0a6cfed2..481edaeb 100644 --- a/apps/docs/src/content/docs/observability/metrics.mdx +++ b/apps/docs/src/content/docs/observability/metrics.mdx @@ -1,21 +1,21 @@ --- title: Token metrics & tokens/sec -description: See how fast the model is generating — live in the status line, on demand with /metrics, or in the logs for deeper analysis. +description: See how fast the model is generating. Live in the status line, on demand with /metrics, or in the logs for deeper analysis. --- Want to know how fast your model is actually going? tsforge surfaces real token throughput in three places, from "glance at it" to "analyse a whole run." -## 1. The status bar — always on +## 1. The status bar: always on On a real terminal a status bar stays pinned to the bottom (see [Interactive CLI](/cli/interactive/#status-bar)), showing the **last call's generation rate** next to the context gauge: ``` - ⎯ qwen3.6-27b · ctx ~8k/32k 25% · 2 turns · 12s · 47 tok/s · done · src/** + ⎯ deepseek-v4-flash · ctx ~8k/32k 25% · 2 turns · 12s · 47 tok/s · done · src/** ``` -`47 tok/s` is how many output tokens per second the model produced on that call. Nothing to enable — it appears as soon as the first model call returns. (Piped/`--log` runs show the same line inline instead.) +`47 tok/s` is how many output tokens per second the model produced on that call. Nothing to enable; it appears as soon as the first model call returns. (Piped/`--log` runs show the same line inline instead.) -## 2. `/metrics` — the running total +## 2. `/metrics`: the running total Type `/metrics` at the prompt for the whole session so far: @@ -31,17 +31,17 @@ Type `/metrics` at the prompt for the whole session so far: | `in` / `out` | total prompt (input) and completion (output) tokens | | `tok/s last` | output rate of the most recent call | | `tok/s avg` | output rate averaged across all calls | -| `turns to green` | how many turns the last successful run took to pass the gate — the loop-efficiency signal (lower is better) | +| `turns to green` | how many turns the last successful run took to pass the gate. The loop-efficiency signal is lower values = better. | ## What "tokens/sec" measures -It's **output decode speed** — measured from the *first* streamed token to the end of the call. That deliberately excludes prompt-processing / time-to-first-token, so a big prompt doesn't make the model look slow. It reflects how fast tokens come out once they start. +It's **output decode speed**: measured from the *first* streamed token to the end of the call. That deliberately excludes prompt-processing and time-to-first-token, so a big prompt doesn't make the model look slow. It reflects how fast tokens come out once they start. -A low number usually points at the *serving* setup (quantization, batch size, GPU), not the model — "measure, don't assume." +A low number usually points at the *serving* setup (quantization, batch size, GPU), not the model. Measure, don't assume. -## 3. `--log` — analyse a whole run +## 3. `--log`: analyse a whole run -Start the CLI (or an eval) with `--log` and the whole run is written to `~/.tsforge/logs/.jsonl` as a typed ledger — one event per line, each wrapped in a `payload` (model calls, tool calls, policy decisions, gate verdicts): +Start the CLI (or an eval) with `--log` and the whole run is written to `~/.tsforge/logs/.jsonl` as a typed ledger. One event per line, each wrapped in a `payload` (model calls, tool calls, policy decisions, gate verdicts): ```json { "type": "model_call_finished", "runId": "…", "timestamp": "…", @@ -49,7 +49,7 @@ Start the CLI (or an eval) with `--log` and the whole run is written to `~/.tsfo "completionTokens": 240, "tokensPerSecond": 46, "ms": 5200 } } ``` -Turn a log into a one-screen summary — including **policy denials by risk** — with [`tsforge trace`](/observability/trace/): +Turn a log into a one-screen summary (including **policy denials by risk**) with [`tsforge trace`](/observability/trace/): ```bash tsforge trace # newest log @@ -63,6 +63,6 @@ bun run packages/core/scripts/cli-metrics.ts # newest log bun run packages/core/scripts/cli-metrics.ts run.jsonl # a specific one ``` -It reports tokens-to-solution, peak context, edits, gate runs, and — for a run that didn't reach green — a **`failure class`** (e.g. `type-error (TS18048)`, `tool-malformed`, `hallucinated-import`): one structured reason for *why* it failed, so a failure points at the rule, prompt, or fixer to build next. The same distillation is available programmatically as `analyzeEvents(events)` from `@agjs/tsforge`'s eval module, and feeds the [A/B sweep report](/eval/ab-testing/#why-runs-failed). +It reports tokens-to-solution, peak context, edits, gate runs, and for runs that didn't reach green, a **`failure class`** (e.g. `type-error (TS18048)`, `tool-malformed`, `hallucinated-import`). This is one structured reason for *why* it failed, so a failure points at the rule, prompt, or fixer to build next. The same distillation is available programmatically as `analyzeEvents(events)` from `@agjs/tsforge`'s eval module, and feeds the [A/B sweep report](/eval/ab-testing/#why-runs-failed). → [Interactive CLI](/cli/interactive/) · [A/B testing](/eval/ab-testing/) · [Model adapter](/inference/adapter/) diff --git a/apps/docs/src/content/docs/observability/trace.mdx b/apps/docs/src/content/docs/observability/trace.mdx index 80e98f99..0d17eb42 100644 --- a/apps/docs/src/content/docs/observability/trace.mdx +++ b/apps/docs/src/content/docs/observability/trace.mdx @@ -1,9 +1,9 @@ --- title: Trace a run -description: Turn a --log run into a human-readable summary — model/tool calls, policy decisions, gate verdicts, and turns-to-green — with tsforge trace. +description: Turn a --log run into a human-readable summary (model/tool calls, policy decisions, gate verdicts, and turns-to-green) with tsforge trace. --- -When you run with `--log`, tsforge writes the whole run to `~/.tsforge/logs/.jsonl` as a typed [ledger](#the-log-ledger). `tsforge trace` reads that ledger back and prints a one-screen summary of what actually happened — **deterministically, with no model call**. +When you run with `--log`, tsforge writes the whole run to `~/.tsforge/logs/.jsonl` as a typed [ledger](#the-log-ledger). `tsforge trace` reads that ledger back and prints a one-screen summary of what actually happened (deterministically, with no model call). ## `tsforge trace` @@ -18,7 +18,7 @@ Inside an interactive session, **`/trace`** does the same and prefers the curren › /trace trace of ~/.tsforge/logs/2026-06-19-…​.jsonl -model qwen3.6-27b +model deepseek-v4-flash final status done failure class none turns 1 @@ -37,11 +37,11 @@ wall clock 0s | Field | Meaning | | --- | --- | -| `final status` / `failure class` | how the run ended, and — if it didn't reach green — one structured reason why (e.g. `type-error (TS18048)`) | +| `final status` / `failure class` | how the run ended (if it didn't reach green, one structured reason why, e.g. `type-error (TS18048)`) | | `turns` / `turns to green` | repair iterations, and how many it took to first pass the gate (lower is better) | | `model calls` / `tokens out` / `peak context` | call count, completion tokens, and the context high-water mark | | `edits/creates` / `gate runs` | file mutations and how many times the gate ran | -| `accept rate` / `cost/accepted` | share of edits that stuck (vs were reverted), and completion tokens per durable change — tokens mean little if most edits get reverted | +| `accept rate` / `cost/accepted` | share of edits that stuck (vs were reverted), and completion tokens per durable change (tokens mean little if most edits get reverted) | | `policy denials` / `policy asks` | [action-policy](/guardrails/config/) verdicts that blocked or paused an action, denials bucketed by risk | ## The `--log` ledger @@ -54,7 +54,7 @@ Each line is one typed event (valid JSON), so a run is fully reconstructable. Ev "message": "blocked rm -rf /", "rules": ["no-destructive-shell"] } } ``` -Policy decisions — allow / ask / deny, with their risk and the rules that matched — are recorded alongside model calls, tool calls, and gate verdicts. `tsforge trace` surfaces the denials and asks so you can see where the harness stepped in. +Policy decisions (allow / ask / deny, with their risk and the rules that matched) are recorded alongside model calls, tool calls, and gate verdicts. `tsforge trace` surfaces the denials and asks so you can see where the harness stepped in. ## Programmatic access diff --git a/apps/docs/src/content/docs/quality/tests.mdx b/apps/docs/src/content/docs/quality/tests.mdx index 5dfd0409..a2d95c22 100644 --- a/apps/docs/src/content/docs/quality/tests.mdx +++ b/apps/docs/src/content/docs/quality/tests.mdx @@ -1,6 +1,6 @@ --- title: Tests by default -description: "tsforge enforces tests out of the box — a logic file the agent writes without a test fails the gate, scoped to what it changes so it never blocks on your repo's legacy code." +description: "tsforge enforces tests out of the box. A logic file the agent writes without a test fails the gate, scoped to what it changes so it never blocks on your repo's legacy code." --- Most AI tools will happily write a function and move on. tsforge won't. Out of the box, a logic file the agent changes without a test **fails the gate**. Quality and strictness are the default, not a flag you have to remember. diff --git a/apps/docs/src/content/docs/quickstart.mdx b/apps/docs/src/content/docs/quickstart.mdx index b7058f19..e19b2d16 100644 --- a/apps/docs/src/content/docs/quickstart.mdx +++ b/apps/docs/src/content/docs/quickstart.mdx @@ -3,8 +3,6 @@ title: Quickstart description: Install tsforge, configure your model, and run the CLI against a TypeScript project. --- -import HarnessBoardDiagram from "../../components/HarnessBoardDiagram.astro"; - From install to a running loop in a few minutes. For the why behind each step, read [Big picture](/big-picture/) first. ## Before you start @@ -50,11 +48,11 @@ export PATH="$(bun pm bin -g):${PATH}" mkdir -p ~/.tsforge cat > ~/.tsforge/models.json <<'EOF' { - "active": "qwen-local", + "active": "local", "models": { - "qwen-local": { + "local": { "baseUrl": "http://localhost:8000/v1", - "model": "qwen3.6-27b", + "model": "deepseek-v4-flash", "thinking": true } } @@ -74,9 +72,16 @@ Point tsforge at your target project when prompted, or pass `--dir`. See [Intera ## Harness at a glance - +```mermaid +flowchart LR + you(["task"]) --> agent["model agent"] + agent --> tools["read · edit · run"] + tools --> gate["gate: tsc + ESLint + tests"] + gate -->|"fail"| agent + gate -->|"pass"| done(["checkpoint"]) +``` -For the full board with captions, see [Big picture](/big-picture/). +You give it a task; the model edits; the gate decides done. For the full architecture (the general core, the adapters at the seams, and the request flow), see [Big picture](/big-picture/). ## The loop (short version) diff --git a/apps/docs/src/content/docs/reference/commands.mdx b/apps/docs/src/content/docs/reference/commands.mdx index 232e0ddf..e55ee796 100644 --- a/apps/docs/src/content/docs/reference/commands.mdx +++ b/apps/docs/src/content/docs/reference/commands.mdx @@ -29,7 +29,7 @@ tsforge scaffold --dest ./site --archetype astro # Astro static site tsforge scaffold --dest ./my-app --set WITH_OBSERVABILITY=0 --no-boot ``` -`tsforge scaffold` stands up a new project from [BoringStack](https://boringstack.xyz) — +`tsforge scaffold` stands up a new project from [BoringStack](https://boringstack.xyz): clone → configure (via BoringStack's own scripts) → boot → hands off the build gate. Interactively it previews the container topology + required secrets before applying. Greenfield only; editing an existing repo never scaffolds. See @@ -42,7 +42,7 @@ tsforge setup # the interactive conventions wizard tsforge setup --yes # write the scan's recommendations, no prompts ``` -`tsforge setup` (or `/setup` in a session) scans the repo and writes a `conventions` block to `tsforge.config.json` so the guardrails match your repo's taste — interface naming, enums, test layout, component folders. It never weakens the safety floor (no `any`/`as`/`!`, complexity cap, `===`), and writes nothing until you Apply. Outside a terminal it prints the proposal and writes nothing unless you pass `--yes`. See [Set up a repo](/cli/setup/). +`tsforge setup` (or `/setup` in a session) scans the repo and writes a `conventions` block to `tsforge.config.json` so the guardrails match your repo's taste: interface naming, enums, test layout, component folders. It never weakens the safety floor (no `any`/`as`/`!`, complexity cap, `===`), and writes nothing until you Apply. Outside a terminal it prints the proposal and writes nothing unless you pass `--yes`. See [Set up a repo](/cli/setup/). ## Review the change you're on @@ -53,9 +53,9 @@ tsforge review --base develop # diff against an explicit base ref tsforge review --with-gate # run the gate first; skip what it already covers ``` -`tsforge review` reviews the change you're working on — the working tree (committed **and** uncommitted edits) vs the auto-detected base (merge-base with `main`/`master`). No commit or push required. +`tsforge review` reviews the change you're working on: the working tree (committed **and** uncommitted edits) vs the auto-detected base (merge-base with `main`/`master`). No commit or push required. -It is **functional** review — logic, regressions, edge cases, and business rules — guided by a built-in senior-review rubric (lenses), with each finding adversarially re-checked against the real code so false positives are dropped. Types, structure, and style are out of scope (the gate covers those). When a `tsconfig.json` is present it also feeds the reviewer a **caller blast-radius** signal (who calls each changed export, computed type-exactly) so the regression lens has concrete call sites to check. Exits non-zero if any **error**-severity finding survives, so it's CI-usable. +It is **functional** review: logic, regressions, edge cases, and business rules, guided by a built-in senior-review rubric (lenses), with each finding adversarially re-checked against the real code so false positives are dropped. Types, structure, and style are out of scope (the gate covers those). When a `tsconfig.json` is present it also feeds the reviewer a **caller blast-radius** signal (who calls each changed export, computed type-exactly) so the regression lens has concrete call sites to check. Exits non-zero if any **error**-severity finding survives, so it's CI-usable. The same review is available inside an interactive session as **`/review`** (optionally `/review ` to diff against a specific ref). @@ -67,7 +67,7 @@ tsforge run api-endpoint "add a route" # run a saved recipe by name tsforge run api-endpoint --files lib/** # ...with a CLI override (CLI wins) ``` -A recipe is declarative JSON in `.tsforge/recipes/.json` (project) or `~/.tsforge/recipes/.json` (global) that composes a run's options — model, gate, scope, limits, policy mode. See [Recipes](/cli/recipes/). +A recipe is declarative JSON in `.tsforge/recipes/.json` (project) or `~/.tsforge/recipes/.json` (global) that composes a run's options: model, gate, scope, limits, policy mode. See [Recipes](/cli/recipes/). ## Build a whole app (greenfield) @@ -94,7 +94,7 @@ tsforge map status # when it was built, files/hubs, how much drifted tsforge map forget # delete it ``` -`tsforge map` (or `/map` in a session) builds a deterministic structural map of a TypeScript repo — the directory shape, what each module exports, and the **hub modules** everything imports — and persists it under `.tsforge/`. On the next session it's injected into the agent's context so it starts oriented instead of exploring from scratch. All compiler facts (no AI guessing, no extra dependencies); it primes future sessions (run `/clear` to apply mid-session). Best for existing repos; greenfield builds don't need it. +`tsforge map` (or `/map` in a session) builds a deterministic structural map of a TypeScript repo: the directory shape, what each module exports, and the **hub modules** everything imports. It persists under `.tsforge/`, and on the next session it's injected into the agent's context so it starts oriented instead of exploring from scratch. All compiler facts (no AI guessing, no extra dependencies); it primes future sessions (run `/clear` to apply mid-session). Best for existing repos; greenfield builds don't need it. ## Trace a run @@ -103,7 +103,7 @@ tsforge trace # summarize the newest --log run tsforge trace run.jsonl # a specific log file ``` -`tsforge trace` (or `/trace` in a session) reads a `--log` ledger and prints a one-screen summary — model/tool calls, **policy decisions** (allow/ask/deny by risk), gate verdicts, turns-to-green, and the **accept rate / cost-per-accepted-change** (how many edits stuck vs were reverted) — deterministically, with no model call. See [Trace a run](/observability/trace/). +`tsforge trace` (or `/trace` in a session) reads a `--log` ledger and prints a one-screen summary: model/tool calls, **policy decisions** (allow/ask/deny by risk), gate verdicts, turns-to-green, and the **accept rate / cost-per-accepted-change** (how many edits stuck vs were reverted). All deterministically, with no model call. See [Trace a run](/observability/trace/). ## Merge gate (repo root) diff --git a/apps/docs/src/content/docs/reference/flags.mdx b/apps/docs/src/content/docs/reference/flags.mdx index f7cbdc50..6804a72d 100644 --- a/apps/docs/src/content/docs/reference/flags.mdx +++ b/apps/docs/src/content/docs/reference/flags.mdx @@ -3,7 +3,7 @@ title: Environment variables description: Canonical list of every TSFORGE_* environment variable. --- -## Behavior & tools — configure in `/config` +## Behavior & tools: configure in `/config` Feature toggles are configured **inside the harness**, not through env vars. Run [`/config`](/cli/interactive/) in an interactive session: every setting shows a @@ -12,7 +12,7 @@ there: | Setting | Default | What it does | | --- | --- | --- | -| Web tools | on (interactive) | keyless `web_fetch` + `web_search` (DuckDuckGo); off in one-shot/eval for offline determinism — see [Web access](/integrations/web-tools/) | +| Web tools | on (interactive) | keyless `web_fetch` + `web_search` (DuckDuckGo); off in one-shot/eval for offline determinism. See [Web access](/integrations/web-tools/) | | TDD enforcement | on | test-first guidance + `test-sibling-required` as an error on changed logic files | `/config` also sets the model, interactive mode, gate command, and editable scope. @@ -22,29 +22,29 @@ explicit value always wins over the interactive default) and `TSFORGE_TDD` (`0` to opt out). Only genuine human choices live in `/config`. The rest run unconditionally: the -**update check** always happens in an interactive, non-CI session — set the +**update check** always happens in an interactive, non-CI session. Set the cross-tool `NO_UPDATE_NOTIFIER=1` (note: not a `TSFORGE_*` variable) to suppress -it; [programmatic tool calling](/agent/model-agent/), LSP navigation, +it. [Programmatic tool calling](/agent/model-agent/), LSP navigation, `git_context`, hashline, TTSR, and write diagnostics are always on. The variables listed below the fold are **endpoint, tuning, and operational** knobs -(model endpoint, timeouts, eval/test harness) — not user-facing feature switches. +(model endpoint, timeouts, eval/test harness), not user-facing feature switches. ### Eval / CI knobs (not in `/config`) The always-on tools can be withheld via env only for eval sweeps or non-git / -headless environments — never something you'd change interactively: +headless environments. Never change these interactively: | Variable | Default | Effect | | --- | --- | --- | | `TSFORGE_NO_LSP_TOOLS` | off | withhold the LSP navigation tools (`=1`) | | `TSFORGE_NO_GIT_TOOL` | off | withhold the `git_context` tool (`=1`) | | `TSFORGE_NO_SCRIPT` | off | withhold the `script` (programmatic tool calling) tool (`=1`) | -| `TSFORGE_NO_DELEGATION` | off | withhold `spawn_agent` — run single-stream with no subagents (`=1`); see [delegation](/agent/delegation/) | +| `TSFORGE_NO_DELEGATION` | off | withhold `spawn_agent`: run single-stream with no subagents (`=1`). See [delegation](/agent/delegation/) | ## Git context -On existing-code runs tsforge offers `git_context` — a read-only tool giving the model structured, token-bounded access to repo state, so it can scope a review or a fix to **what actually changed** instead of shelling out to raw `git`. Ops: `diff`, `changed_files`, `log` (incl. a line range's history), `blame`, and `show`. It wraps the `git` binary via an explicit argv (no shell), validates the `sha`, and rejects shell metacharacters / option injection in `ref`/`path`; output is char-capped (`maxChars` to raise it). +On existing-code runs tsforge offers `git_context`, a read-only tool giving the model structured, token-bounded access to repo state, so it can scope a review or a fix to **what actually changed** instead of shelling out to raw `git`. Ops: `diff`, `changed_files`, `log` (incl. a line range's history), `blame`, and `show`. It wraps the `git` binary via an explicit argv (no shell), validates the `sha`, and rejects shell metacharacters / option injection in `ref`/`path`; output is char-capped (`maxChars` to raise it). Offered only when there is existing code to inspect (greenfield scratch builds have no history), and it is independent of `TSFORGE_NO_LSP_TOOLS` since it is not an LSP tool. Being read-only, it is available in [plan mode](/cli/plan-mode/) too. For eval/headless runs it can be withheld with `TSFORGE_NO_GIT_TOOL=1`. @@ -60,7 +60,7 @@ Opt-in, free, and no required service keys. Turn on **Web tools** in [`/config`] ## Image capabilities (vision, generation) -When the chat model is text-only, route **reading** and **generating** images to a separate backend — configured in [`models.json` `capabilities`](/inference/models-json/#extra-capabilities-vision-image-generation) or the env vars below. Each capability is off unless configured; when on, it enables the `read_image` / `generate_image` tools and the drag / paste / `@` attachment UX ([interactive](/cli/interactive/)). The env trio is an ad-hoc backend without editing the file; a `_MODEL` alone (no `_BASE_URL`) names an existing registry entry. +When the chat model is text-only, route **reading** and **generating** images to a separate backend. Configure it in [`models.json` `capabilities`](/inference/models-json/#extra-capabilities-vision-image-generation) or the env vars below. Each capability is off unless configured; when on, it enables the `read_image` / `generate_image` tools and the drag / paste / `@` attachment UX ([interactive](/cli/interactive/)). The env trio is an ad-hoc backend without editing the file; a `_MODEL` alone (no `_BASE_URL`) names an existing registry entry. | Variable | Default | Effect | | --- | --- | --- | @@ -69,7 +69,7 @@ When the chat model is text-only, route **reading** and **generating** images to | `TSFORGE_IMAGE_API` | `chat-modalities` | image-gen wire shape: `chat-modalities` or `images-generations` | | `TSFORGE_IMAGE_PROTOCOL` | auto | inline preview of generated images: `iterm2` to force, `off`/`none` to disable (auto-detects iTerm2, disabled under tmux) | -Generated images are saved under `.tsforge/images/`. Pasting an image is **Ctrl+V** (a terminal app can't receive Cmd+V); identical images are described once (no duplicate cost). Model IDs + costs: OpenRouter runs everything through `/chat/completions` (no `/images` endpoint) — e.g. `google/gemini-2.5-flash-lite` (read) and `google/gemini-3.1-flash-lite-image` (generate). +Generated images are saved under `.tsforge/images/`. Pasting an image is **Ctrl+V** (a terminal app can't receive Cmd+V); identical images are described once (no duplicate cost). Model IDs + costs: OpenRouter runs everything through `/chat/completions` (no `/images` endpoint). For example, `google/gemini-2.5-flash-lite` (read) and `google/gemini-3.1-flash-lite-image` (generate). ## Opt-in gate oracles @@ -86,7 +86,7 @@ Extra gate steps (default off; each skips cleanly when nothing applies). See [Ho | Variable | Default | Toggles | | --- | --- | --- | | `TSFORGE_BASE_URL` | `http://localhost:8000/v1` | API endpoint | -| `TSFORGE_MODEL` | `qwen3.6-27b` | model name | +| `TSFORGE_MODEL` | `deepseek-v4-flash` | model name | | `TSFORGE_API_KEY` | unset | API key | | `TSFORGE_MAX_TOKENS` | `16384` | max output tokens | | `TSFORGE_REPETITION_PENALTY` | off | vLLM repetition penalty | @@ -123,7 +123,7 @@ Extra gate steps (default off; each skips cleanly when nothing applies). See [Ho | --- | --- | | `TSFORGE_PACKS` | comma-separated pack IDs | | `TSFORGE_RULE_OVERRIDES` | JSON severity map | -| `TSFORGE_CONVENTIONS` | JSON conventions block (interface naming / enums) — rebuilds the bundled rule options; written by [`tsforge setup`](/cli/setup/) | +| `TSFORGE_CONVENTIONS` | JSON conventions block (interface naming / enums): rebuilds the bundled rule options. Written by [`tsforge setup`](/cli/setup/) | ## Eval scripts @@ -148,7 +148,7 @@ On by default; withhold with `TSFORGE_NO_SCRIPT=1` (above). Tuning knobs: | Variable | Set by | Meaning | | --- | --- | --- | -| `TSFORGE_STATUS` | tsforge (for your `--notify` command) | the run outcome (e.g. `greenfield done 7/7`) — read it in the notifier script | +| `TSFORGE_STATUS` | tsforge (for your `--notify` command) | the run outcome (e.g. `greenfield done 7/7`). Read it in the notifier script | ## Debug & tracing diff --git a/apps/docs/src/content/docs/reference/input-editor.mdx b/apps/docs/src/content/docs/reference/input-editor.mdx index b813967f..c7fd1a78 100644 --- a/apps/docs/src/content/docs/reference/input-editor.mdx +++ b/apps/docs/src/content/docs/reference/input-editor.mdx @@ -54,7 +54,7 @@ When your cursor is at the **top** of the buffer (line 0, any column), press ** ## Multi-line paste -Paste a block of text with your terminal's paste (⌘V on macOS / Ctrl+Shift+V on Linux) — the terminal delivers it as a bracketed paste. The pasted content lands in the buffer as-is and may span multiple lines; edit it like any buffer text. When you press Enter, the entire buffer (including pasted lines) is submitted as a single message. +Paste a block of text with your terminal's paste (⌘V on macOS / Ctrl+Shift+V on Linux). The terminal delivers it as a bracketed paste. The pasted content lands in the buffer as-is and may span multiple lines; edit it like any buffer text. When you press Enter, the entire buffer (including pasted lines) is submitted as a single message. Very large pastes are displayed as `[paste #N +M lines]` in the UI and expand to their full content when sent. @@ -65,10 +65,10 @@ Requires a [`vision` capability](/inference/models-json/#extra-capabilities-visi | Method | Action | | --- | --- | | **Ctrl+V** | Paste an image from the clipboard. A `📋 reading clipboard…` hint shows during the read, then an `[image #N]` chip lands in the buffer. (Falls back to pasting clipboard text if there's no image.) | -| **Drag & drop** | Drag an image file onto the terminal — it drops the path, which is attached on send. | -| **`@` mention** | `@path/to/image.png` — the picker lists image files too. | +| **Drag & drop** | Drag an image file onto the terminal. It drops the path, which is attached on send. | +| **`@` mention** | `@path/to/image.png`: the picker lists image files too. | -**Why Ctrl+V, not ⌘V:** the terminal emulator intercepts ⌘V and, for an image, hands the app nothing — a terminal program can only receive Ctrl+V (a real control byte). Identical images attached more than once are described **once** (no duplicate cost). See also [generating images](/reference/flags/#image-capabilities-vision-generation). +**Why Ctrl+V, not ⌘V:** the terminal emulator intercepts ⌘V and, for an image, hands the app nothing. A terminal program can only receive Ctrl+V (a real control byte). Identical images attached more than once are described **once** (no duplicate cost). See also [generating images](/reference/flags/#image-capabilities-vision-generation). ## Command palette and file picker diff --git a/apps/docs/src/content/docs/reference/roadmap.mdx b/apps/docs/src/content/docs/reference/roadmap.mdx index 95f1a9ea..ac41a7fe 100644 --- a/apps/docs/src/content/docs/reference/roadmap.mdx +++ b/apps/docs/src/content/docs/reference/roadmap.mdx @@ -3,24 +3,24 @@ title: Roadmap description: What's shipped through 0.27, the path to 1.0, and candidate work. --- -TypeScript coding harness for web projects — `packages/core` for the loop and gate, [tsforge.dev](https://tsforge.dev) for docs. +TypeScript coding harness for web projects: `packages/core` for the loop and gate, [tsforge.dev](https://tsforge.dev) for docs. ## Shipped through 0.27 **Strictness & the gate** - Stack detection → **21 ESLint rule packs (122 rules)** + a **31-rule meta-rule engine** (config, CI, supply chain, container, testing, structure). See [Rule packs](/guardrails/rule-packs/) · [Meta-rules](/guardrails/meta-rules/). - **6 profiles** (recommended → strict / security / frontend / backend / opinionated); a type-aware ESLint overlay; progress-based loop termination (no blunt turn cap). See [When the gate fails](/loop/validation/). -- **Tests by default** — `test-sibling-required` is an error on changed logic files. **Opt-in gate oracles** (coverage, boot smoke, property tests). See [Tests by default](/quality/tests/) · [the gate](/loop/gate-floor/). +- **Tests by default**: `test-sibling-required` is an error on changed logic files. **Opt-in gate oracles** (coverage, boot smoke, property tests). See [Tests by default](/quality/tests/) · [the gate](/loop/gate-floor/). **Adapt to your repo** -- **`tsforge setup`** — a conventions wizard that scans the repo and writes `tsforge.config.json` (interface naming, enums, test layout, component folders) without weakening the safety floor. See [Set up a repo](/cli/setup/). +- **`tsforge setup`**: a conventions wizard that scans the repo and writes `tsforge.config.json` (interface naming, enums, test layout, component folders) without weakening the safety floor. See [Set up a repo](/cli/setup/). - `tsforge.config.json` overrides + **external plugin packs**. See [config](/guardrails/config/). **The agent surface** - **Functional review** (`tsforge review`, gate-aware) · **workspace map** (`tsforge map`) · **pre-edit scout** · **declarative recipes** (`tsforge run`) · **plan mode** · **cross-session memory**. -- **Permissions & policy** — 6 modes, deny-first config rules, and a run **ledger** summarized by `tsforge trace`. See [Permissions & policy](/guardrails/policy/) · [Trace a run](/observability/trace/). -- **Keyless web research** — opt-in package metadata/docs, search, fetch, and local browser reads with no required API key. See [Web access](/integrations/web-tools/). -- **Greenfield scaffolding** — stands up the full BoringStack stack and builds it resource-by-resource against BoringStack's own gate. See [Greenfield scaffolding](/scaffold/boringstack/). +- **Permissions & policy**: 6 modes, deny-first config rules, and a run **ledger** summarized by `tsforge trace`. See [Permissions & policy](/guardrails/policy/) · [Trace a run](/observability/trace/). +- **Keyless web research**: opt-in package metadata/docs, search, fetch, and local browser reads with no required API key. See [Web access](/integrations/web-tools/). +- **Greenfield scaffolding**: stands up the full BoringStack stack and builds it resource-by-resource against BoringStack's own gate. See [Greenfield scaffolding](/scaffold/boringstack/). **Models & observability** - **Provider-agnostic** `~/.tsforge/models.json` (any OpenAI-compatible API; `qwen`/`deepseek`/`openai` reasoning dialects, DeepSeek auto-detected). See [models.json](/inference/models-json/). diff --git a/apps/docs/src/content/docs/scaffold/boringstack.mdx b/apps/docs/src/content/docs/scaffold/boringstack.mdx index c53fb0e3..74636195 100644 --- a/apps/docs/src/content/docs/scaffold/boringstack.mdx +++ b/apps/docs/src/content/docs/scaffold/boringstack.mdx @@ -14,11 +14,11 @@ committed in the BoringStack repo, which tsforge reads after cloning. | Archetype | What you get | Gate | | --- | --- | --- | | `boringstack` | The full stack: Bun + Elysia + Drizzle API (`apps/api`) + Vite/React UI (`apps/ui`), Postgres + Valkey, and whatever observability/billing/auth services your toggles enable | `cd apps/api && bun run validate` · `cd apps/ui && bun run validate` · `bun run check` | -| `astro` | The Astro Starlight static site (`apps/docs`) — no API, no Docker, no `.env` | `bun run build` | +| `astro` | The Astro Starlight static site (`apps/docs`); no API, no Docker, no `.env` | `bun run build` | ## When it runs -Greenfield only. Editing an existing repo never scaffolds — tsforge auto-detects the +Greenfield only. Editing an existing repo never scaffolds; tsforge auto-detects the gate and patches in place. The wizard is its own subcommand so its flags don't collide with the harness flags: @@ -35,8 +35,8 @@ blocking cross-rule violation. Off a TTY, it uses flags directly. 1. **Clone** BoringStack at the manifest's `defaultRef`, resolving the exact commit SHA into `/.tsforge/scaffold.json` for replay. -2. **Configure** by driving BoringStack's own scripts — `scripts/rename-project.sh`, - then `setup.sh` (which bootstraps `compose/.env` and generates a GlitchTip secret) — +2. **Configure** by driving BoringStack's own scripts: `scripts/rename-project.sh`, + then `setup.sh` (which bootstraps `compose/.env` and generates a GlitchTip secret), then writing your toggle/provider answers into the right `.env` file (infra toggles → `infra/compose/compose/.env`; app features → `infra/compose/compose/api..env`). Prod-only secrets (`JWT_SECRET`, `MFA_ENCRYPTION_KEY`, `VALKEY_PASSWORD`) are @@ -44,7 +44,7 @@ blocking cross-rule violation. Off a TTY, it uses flags directly. 3. **Boot** the stack with `setup.sh --up` and health-poll the API + UI (skip with `--no-boot` / `STACK=smoke`). Boot is a one-time scaffold-step, not part of the per-edit gate. -4. **Hand off** — prints the exact command to start building features against the +4. **Hand off**: prints the exact command to start building features against the scaffolded project (its own `bun run validate` becomes the gate). ## Flags @@ -54,8 +54,8 @@ blocking cross-rule violation. Off a TTY, it uses flags directly. | `--dest ` | Where to create the project (required) | | `--archetype ` | Default `boringstack` | | `--stack ` | Default `dev` | -| `--set KEY=VALUE` | Set a toggle/provider answer (repeatable) — e.g. `--set WITH_OBSERVABILITY=0` | -| `--multi KEY=a,b` | Set a multi-select answer — e.g. `--multi OAUTH_PROVIDERS=google,github` | +| `--set KEY=VALUE` | Set a toggle/provider answer (repeatable). For example: `--set WITH_OBSERVABILITY=0` | +| `--multi KEY=a,b` | Set a multi-select answer. For example: `--multi OAUTH_PROVIDERS=google,github` | | `--ref ` | Override the manifest's clone ref | | `--no-boot` | Clone + configure, but don't start Docker | @@ -72,13 +72,13 @@ tsforge models none of BoringStack's surface itself. `.tsforge/scaffold-manifest services each toggle spawns, cross-rules (OAuth ⇒ Valkey, `EMAIL_PROVIDER=smtp` ⇒ `WITH_MAILPIT`, OTel-vs-Sentry exclusion), and which `.env` file each value targets. tsforge reads it post-clone and a **completeness alarm** fails the build if a watched -`WITH_*`/`*_ENABLED` toggle in BoringStack's `.env.example` isn't modelled — so the +`WITH_*`/`*_ENABLED` toggle in BoringStack's `.env.example` isn't modelled. The wizard can never silently drop a capability. Evolving BoringStack means editing that manifest, not tsforge. ## How to test it -**Unit (fast, no clone/Docker)** — the planner, manifest parser, wizard flow, env +**Unit (fast, no clone/Docker)**: the planner, manifest parser, wizard flow, env apply, and completeness alarm are pure and fully covered: ```bash @@ -89,7 +89,7 @@ This includes a snapshot of BoringStack's real `.env.example` files asserting th bundled manifest has zero coverage gaps, plus an exhaustive on/off matrix for every toggle and provider. -**Real clone + configure (opt-in, needs a local BoringStack checkout)** — runs git + +**Real clone + configure (opt-in, needs a local BoringStack checkout)**: runs git + BoringStack's actual `setup.sh`, no Docker boot: ```bash @@ -97,7 +97,7 @@ TSFORGE_SCAFFOLD_E2E=1 BORINGSTACK_REPO=/path/to/boringstack \ bun test packages/core/tests/scaffold-clone-configure.e2e.test.ts ``` -**End-to-end by hand** — drive the whole flow against a local checkout (the +**End-to-end by hand**: drive the whole flow against a local checkout (the `BORINGSTACK_REPO` override avoids hitting GitHub): ```bash diff --git a/apps/docs/src/content/docs/spec/format.mdx b/apps/docs/src/content/docs/spec/format.mdx index eadd2da6..eb9b06a0 100644 --- a/apps/docs/src/content/docs/spec/format.mdx +++ b/apps/docs/src/content/docs/spec/format.mdx @@ -50,7 +50,7 @@ A1. sum adds two numbers correctly. ### Acceptance criteria -Prose under `## Acceptance criteria` is attached to every task as **intent** — the model works from stated goals instead of reverse-engineering from test calls alone. +Prose under `## Acceptance criteria` is attached to every task as **intent**. The model works from stated goals instead of reverse-engineering from test calls alone. ### Tasks @@ -67,17 +67,17 @@ Task numbers become stable ids (`1`, `2`, …). ## Running specs -Specs are consumed by the loop's `runSpec()` — used in eval scripts and benchmarks: +Specs are consumed by the loop's `runSpec()`: used in eval scripts and benchmarks. ```bash # Example: eval script against a seed spec bun run eval:spec # Multi-seed sweep (benchmark harness) -TSFORGE_SEED=money bun run eval:sweep +TSFORGE_SEED=math bun run eval:sweep ``` -Benchmark seeds ship as `.spec.md` under [`seeds/`](https://github.com/agjs/tsforge/tree/main/seeds) (e.g. `seeds/money/`, `seeds/orders/`); the harness copies a seed into a fresh run directory under `evals/` for each run. See [Spec runner](/loop/spec-runner/) and [A/B testing](/eval/ab-testing/). +Benchmark seeds ship as `.spec.md` under [`evals/corpus/`](https://github.com/agjs/tsforge/tree/main/evals/corpus) (e.g. `evals/corpus/math/`, `evals/corpus/auth/`); the harness copies a seed into a fresh run directory under `evals/` for each run. See [Spec runner](/loop/spec-runner/) and [A/B testing](/eval/ab-testing/). ## Related tools diff --git a/apps/docs/src/content/docs/uplift/memory.mdx b/apps/docs/src/content/docs/uplift/memory.mdx index 7322b2e1..409435af 100644 --- a/apps/docs/src/content/docs/uplift/memory.mdx +++ b/apps/docs/src/content/docs/uplift/memory.mdx @@ -5,7 +5,7 @@ description: tsforge mines its own runs for mistakes it fixed and recalls them a tsforge **learns from its own runs**. After each run it mines the event stream for gate failures it then **fixed**, accumulates those failure→fix lessons in a per-repo ledger, and recalls the recurring ones as [TTSR](/uplift/ttsr/) rules that fire only when the **same mistake** recurs. -The design principle: **aggregate aggressively and automatically; inject conservatively.** The ledger fills on its own with no curation from you, but almost nothing reaches a new session's prompt — a learned rule is a dormant trigger that costs zero context until the exact pattern shows up again. +The design principle: **aggregate aggressively and automatically; inject conservatively.** The ledger fills on its own with no curation from you, but almost nothing reaches a new session's prompt. A learned rule is a dormant trigger that costs zero context until the exact pattern shows up again. Always on (part of the [TTSR](/uplift/ttsr/) stream-rules system). @@ -18,22 +18,22 @@ run ends → mine events for (rule that failed, the edit that fixed it) next run → loaded as TTSR rules → fire only if the model re-commits the pattern ``` -- **Mine** — when a gate rule (e.g. `no-explicit-any`, `TS18048`) is failing and then **disappears after an edit**, that edit's before→after is a candidate lesson. Deterministic; no model call. -- **Consolidate** — candidates accumulate in `.tsforge/memory.json` with a `hits` count per distinct run and a last-seen timestamp. -- **Activate** — a lesson is promoted to an active rule (written to `.tsforge/learned-rules.json`) only once it has recurred in **≥ 2 runs**. A single fluke fix never starts steering future runs. -- **Decay** — a learned rule unseen for ~45 days drops out of the active set (it stays in the ledger). +- **Mine**: when a gate rule (e.g. `no-explicit-any`, `TS18048`) is failing and then **disappears after an edit**, that edit's before→after is a candidate lesson. Deterministic; no model call. +- **Consolidate**: candidates accumulate in `.tsforge/memory.json` with a `hits` count per distinct run and a last-seen timestamp. +- **Activate**: a lesson is promoted to an active rule (written to `.tsforge/learned-rules.json`) only once it has recurred in **≥ 2 runs**. A single fluke fix never starts steering future runs. +- **Decay**: a learned rule unseen for ~45 days drops out of the active set (it stays in the ledger). Both the [headless loop](/loop/validation/) and the [interactive session](/cli/interactive/) write lessons at the end of a run and load them at the start of the next one. ## Why this beats plain recall -A learned lesson becomes a **deterministic trigger**, not text the model might or might not read. When the pattern recurs mid-stream, tsforge interrupts and injects the known fix — the same mechanism as a built-in [TTSR](/uplift/ttsr/) rule. So memory rides the gate's enforcement rather than hoping for recall. +A learned lesson becomes a **deterministic trigger**, not text the model might or might not read. When the pattern recurs mid-stream, tsforge interrupts and injects the known fix. This is the same mechanism as a built-in [TTSR](/uplift/ttsr/) rule. So memory rides the gate's enforcement rather than hoping for recall. ## Safety - **Non-blocking only.** Auto-writes only ever produce TTSR guidance rules (they nudge and self-disable after 3 fires per run). They never change gate severity or block a build, so a wrong lesson can nudge but can never wedge. - **Local + inspectable.** Everything lives under `.tsforge/` in the repo. Nothing is sent anywhere. -- **Accumulation ≠ injection.** A large ledger costs a new session nothing — learned rules fire only on the actual mistake. +- **Accumulation ≠ injection.** A large ledger costs a new session nothing. Learned rules fire only on the actual mistake. ## Inspecting and clearing @@ -48,7 +48,7 @@ The files are plain JSON you can read, edit, commit, or delete: | File | What it holds | | --- | --- | -| `.tsforge/memory.json` | the full ledger — every candidate lesson with `hits`, `source`, `lastSeen` | +| `.tsforge/memory.json` | the full ledger (every candidate lesson with `hits`, `source`, `lastSeen`) | | `.tsforge/learned-rules.json` | the active subset, loaded as TTSR rules each run | `.tsforge/` is git-ignored by default; commit `learned-rules.json` if you want the team to share a repo's lessons. diff --git a/apps/docs/src/content/docs/uplift/ttsr.mdx b/apps/docs/src/content/docs/uplift/ttsr.mdx index 50740ced..76c8917e 100644 --- a/apps/docs/src/content/docs/uplift/ttsr.mdx +++ b/apps/docs/src/content/docs/uplift/ttsr.mdx @@ -52,7 +52,7 @@ Invalid JSON or invalid regex is skipped silently. On load, tsforge reports how ## Learned rules -tsforge also writes its **own** TTSR rules: failures it made and then fixed in a repo are mined into `.tsforge/learned-rules.json` and loaded alongside the built-ins and your custom rules. So a mistake the model keeps making gets caught mid-stream next time — automatically. See [Learning from past runs](/uplift/memory/). +tsforge also writes its **own** TTSR rules: failures it made and then fixed in a repo are mined into `.tsforge/learned-rules.json` and loaded alongside the built-ins and your custom rules. So a mistake the model keeps making gets caught mid-stream next time. Automatically. See [Learning from past runs](/uplift/memory/). ## Limits diff --git a/apps/docs/src/content/docs/uplift/write-diagnostics.mdx b/apps/docs/src/content/docs/uplift/write-diagnostics.mdx index df38fb23..cf1bee81 100644 --- a/apps/docs/src/content/docs/uplift/write-diagnostics.mdx +++ b/apps/docs/src/content/docs/uplift/write-diagnostics.mdx @@ -14,12 +14,11 @@ Always on. ``` ✓ edited src/foo.ts -Type diagnostics (src/foo.ts): - L12:5 TS2322: Type 'string' is not assignable to type 'number'. - …and 3 more +⚠ write-check: 1 type + 0 lint issue(s) in foo.ts: + L12: Type 'string' is not assignable to type 'number'. (TS2322) ``` -At most **5** errors are shown; extra ones are summarized as `…and N more`. +Up to **200** issues are surfaced per write (a runaway backstop); beyond that they are summarized as `…and N more`. Write diagnostics do not replace the gate. They shorten the feedback loop. diff --git a/packages/core/src/cli/repl.ts b/packages/core/src/cli/repl.ts index 655b2efc..05d86df5 100644 --- a/packages/core/src/cli/repl.ts +++ b/packages/core/src/cli/repl.ts @@ -469,7 +469,7 @@ async function initReplSession(args: ICliArgs): Promise<{ ...(isPolicyMode(args.policyMode) ? { policyMode: args.policyMode } : {}), ...(profile === undefined ? {} : { profile }), // Thinking OFF for interactive replies so they STREAM immediately instead of - // stalling on a long hidden chain-of-thought (qwen-local defaults thinking on). + // stalling on a long hidden chain-of-thought (the local default has thinking on). // The session still flips thinking ON automatically while repairing gate errors. enableThinking: false, }; diff --git a/packages/core/src/inference/inference.constants.ts b/packages/core/src/inference/inference.constants.ts index 4e6a16ad..08bf0537 100644 --- a/packages/core/src/inference/inference.constants.ts +++ b/packages/core/src/inference/inference.constants.ts @@ -39,5 +39,5 @@ export const PROVIDER_LIMITS = { */ export const PROVIDER_DEFAULTS = { baseUrl: "http://localhost:8000/v1", - model: "qwen3.6-27b", + model: "deepseek-v4-flash", } as const; diff --git a/packages/core/src/models-config.ts b/packages/core/src/models-config.ts index 44e84927..6600a9c7 100644 --- a/packages/core/src/models-config.ts +++ b/packages/core/src/models-config.ts @@ -9,7 +9,7 @@ import type { ReasoningStyle } from "./inference/inference.types"; * The model registry — `~/.tsforge/models.json`, the central place a user * configures N model endpoints and switches between them with `/model`. Mirrors * the sessions/logs layout (under `$TSFORGE_HOME` if set, else the home dir). - * Loading is read-only and falls back to the built-in local-qwen default when no + * Loading is read-only and falls back to the built-in local default when no * file exists, so behaviour is identical to the env-driven path until the user * writes a registry. */ @@ -103,9 +103,10 @@ export interface IReviewPanel { reviewers: IReviewer[]; } -/** The built-in local-qwen entry — matches PROVIDER_DEFAULTS so an absent - * registry behaves exactly like the current env-default path. */ -const QWEN_LOCAL: IModelEntry = { +/** The built-in local default entry — matches PROVIDER_DEFAULTS so an absent + * registry behaves exactly like the current env-default path. The reasoning + * dialect is auto-detected from the model name (see request.ts). */ +const LOCAL_DEFAULT: IModelEntry = { baseUrl: PROVIDER_DEFAULTS.baseUrl, model: PROVIDER_DEFAULTS.model, thinking: true, @@ -113,7 +114,7 @@ const QWEN_LOCAL: IModelEntry = { /** The default registry used when no models.json exists yet. */ export function defaultModelsConfig(): IModelsConfig { - return { active: "qwen-local", models: { "qwen-local": QWEN_LOCAL } }; + return { active: "local", models: { local: LOCAL_DEFAULT } }; } /** The registry path: `$TSFORGE_HOME`/.tsforge/models.json, else under home. */ @@ -454,7 +455,7 @@ export async function resolveActiveModel(): Promise<{ const cfg = await loadModelsConfig(); - return { name: cfg.active, entry: cfg.models[cfg.active] ?? QWEN_LOCAL }; + return { name: cfg.active, entry: cfg.models[cfg.active] ?? LOCAL_DEFAULT }; } /** Resolve a model by its registry name, falling back to the active model when @@ -479,7 +480,7 @@ export async function resolveModelByName( const entry = cfg.models[name]; return entry === undefined - ? { name: cfg.active, entry: cfg.models[cfg.active] ?? QWEN_LOCAL } + ? { name: cfg.active, entry: cfg.models[cfg.active] ?? LOCAL_DEFAULT } : { name, entry }; } diff --git a/packages/core/tests/models-config.test.ts b/packages/core/tests/models-config.test.ts index a2a052c0..aaaaf35e 100644 --- a/packages/core/tests/models-config.test.ts +++ b/packages/core/tests/models-config.test.ts @@ -90,12 +90,12 @@ test("resolveModelByName: known name, unknown name, and unset all resolve", asyn expect((await resolveModelByName("")).entry.model).toBe("work-model"); }); -test("missing registry → the built-in local-qwen default (no file written)", async () => { +test("missing registry → the built-in local default (no file written)", async () => { const cfg = await loadModelsConfig(); - expect(cfg.active).toBe("qwen-local"); - expect(cfg.models["qwen-local"]?.model).toBe( - defaultModelsConfig().models["qwen-local"]?.model + expect(cfg.active).toBe("local"); + expect(cfg.models.local?.model).toBe( + defaultModelsConfig().models.local?.model ); // load is read-only: it must NOT create the file as a side effect. expect(await Bun.file(modelsConfigPath()).exists()).toBe(false);