Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 24 additions & 10 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Database schema **diff & migration** tool. Compare a source schema against a tar
generate dialect-native migration SQL, and deploy it. Fourteen SQL dialects:
Postgres (and CockroachDB / YugabyteDB / Redshift), MySQL (and MariaDB / TiDB),
SQL Server / Azure SQL, Oracle, Db2, SQLite, DuckDB, ClickHouse. MongoDB and Redis
carry connection settings only. Distributions: **CLI** (`foxschema` via npm/Homebrew)
and **Docker** (single amd64 image with Db2).
run a fail-closed SQL subset in the editor (no schema compare). Distributions:
**CLI** (`foxschema` via npm/Homebrew) and **Docker** (single amd64 image with Db2).

## Commands

Expand Down Expand Up @@ -277,13 +277,21 @@ per-dialect gotchas live in `packages/sql/src/providers/DIALECTS.md` (tracked).
Safe column change is a sibling registry, not a `SqlDialect` hook:
`planColumnChange` (`modules/utilities/column-change.ts`) finds impact and
blockers from the schema Fox already loaded; each family writes steps in
`providers/<d>/*.column-change.ts` (`column-change.registry.ts`). Engines not
listed answer "not yet" rather than SQL that would fail there. The Utils panel
computes the plan in the browser (`@foxschema/ui-shared`); Fox runs none of it.
A rename alone is in-place; a type change adds, copies in batches, switches
over, and drops last. PostgreSQL / YugabyteDB / CockroachDB, MySQL / MariaDB /
TiDB, and SQL Server / Azure SQL are in v1. SQLite needs a table rebuild;
Redshift's `ALTER TABLE` cannot express the steps.
`providers/<d>/*.column-change.ts` (`column-change.registry.ts`). All 14 SQL
engines are registered. Redis and MongoDB have no column-change entry; omit an
engine to answer "not yet" rather than SQL that would fail there. The Utils
panel (`ColumnChangePanel`) computes the plan in the browser
(`@foxschema/ui-shared`). **Copy** / **Open in editor** send the script;
**Run** and **Execute all** go through `POST /api/sql/execute` (Safe mode
confirms). A rename alone is in-place (`RENAME COLUMN` / `sp_rename`). A type
change is **one ALTER in place** by default (`alterTypeSteps`); tick
**Large table mode** for the add / batched-copy / switch-over path
(`rebuildSteps` + `progressSql`) on PostgreSQL, YugabyteDB, CockroachDB,
MySQL, MariaDB, TiDB, SQL Server and Azure SQL. The other six engines change
the column in place only, and refuse a change they cannot make (SQLite type
change, DuckDB while the table has an index, Redshift except lengthening a
VARCHAR, ClickHouse sorting-key columns). `columnChangeLargeTable()` hides
the checkbox where that path does not exist.

Version-aware DDL: `SchemaProvider.detectVersion?()` → stored in Zustand as
`targetServerVersion` → flows into `SchemaMapping` → dialect drop hooks use it. Oracle pre-23c
Expand Down Expand Up @@ -396,6 +404,12 @@ Constraints:
fence in text that does reach the database is two `--` lines around SQL, and
the SQL is priced like any other (`-- @node\nDROP TABLE t\n-- @end` needs
`editor.ddl`).
- **Routine DDL stays one statement.** `CREATE`/`ALTER` `FUNCTION`|
`PROCEDURE`|`TRIGGER` (and an anonymous `BEGIN … END`) keep inner `;` in
one editor cell. Qualification is read from the token stream: a `--` review
note that ends with `.` must not make the next `CREATE` look like
`s.begin` and split the trigger body. A split body is then `terminated` so
Run does not drag the next statement into it.
- Adding a dialect: say how it lexes in `sql-readings.ts` (see
`packages/sql/src/providers/DIALECTS.md` step 5). Missing there is safe but
slow (every reading of every dialect).
Expand All @@ -406,7 +420,7 @@ Constraints:
2. Register in `provider-settings.ts`, `adapter-registry.ts`, `provider-registry.ts`, `modules/dialect/registry.ts`
(and `modules/access/user-sql.registry.ts` / `modules/access/access-sql.registry.ts` when the engine has account or GRANT SQL).
Backup/restore commands: `<d>.backup.ts` in `modules/utilities/backup.registry.ts`. A `history` query, where the engine records its backups, lets the panel list them; a server-side SQL backup can be run from the panel, a restore never is.
Safe column change: `<d>.column-change.ts` in `modules/utilities/column-change.registry.ts`. Omit the engine to answer "not yet"; do not guess SQL. SQLite is a table rebuild (later); Redshift is not in v1.
Safe column change: `<d>.column-change.ts` in `modules/utilities/column-change.registry.ts`. Omit the engine to answer "not yet"; do not guess SQL. Implement `alterTypeSteps` (in-place) and, if the engine can, `rebuildSteps` + `progressSql` (large-table mode). A change the engine cannot make belongs on `refuse`, not as SQL that fails there. SQLite still has no type-change path (table rebuild).
3. Add the dialect name to the `Dialect` union **and** the `DIALECTS` array in
`packages/sql/src/providers/provider-settings.ts` (`dialect-registry.test.ts`
fails until they match `PROVIDER_SETTINGS`). Nothing in `apps/web` needs
Expand Down
8 changes: 8 additions & 0 deletions docs/BACKEND_ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,14 @@ Every query emits one structured line: `component`, `operation`, `engine`,
are the user's data. `target` is `host:port/database` — never the connection
string, which carries the password.

**Driver-load errors shown in the UI do not include install paths.**
`packages/db/src/cores/driver-loader.ts` (`driverLoadReason`,
`redactInstallPaths`) strips Node's Require stack and absolute
`node_modules` paths from a missing-driver or ENOENT / dlopen message. The
browser sees the package-relative file (`@clickhouse/client/dist/index.js`)
or the filename, never `/Users/…/foxSchema/node_modules/…`. Action responses
for install / update / remove also omit `cwd` and raw npm output.

**Failures are logged at `debug`, not `error`** — the Fastify error boundary
logs the failure once. Logging at each layer turns one failure into four lines.

Expand Down
13 changes: 8 additions & 5 deletions docs/CODE_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,13 @@ modules/ One folder per domain, named to match the frontend feature
| Folder | What it covers |
|---|---|
| `dialect` | The `SqlDialect` contract, the registry, type mapping, capability flags |
| `sql-text` | Shared tokenizer (`sql-tokens.ts`), per-dialect readings (`sql-readings.ts`), statement splitting, and SQL templating. Classifiers (`sqlStatementCategories`, …) take the destination dialect and ask every way it can lex the text. Code fences that reach the database are `--` comments. |
| `sql-text` | Shared tokenizer (`sql-tokens.ts`), per-dialect readings (`sql-readings.ts`), statement splitting, and SQL templating. Classifiers (`sqlStatementCategories`, …) take the destination dialect and ask every way it can lex the text. Code fences that reach the database are `--` comments. CREATE / ALTER FUNCTION, PROCEDURE and TRIGGER bodies stay one statement (inner `;` do not split). |
| `schema-diff` | Comparing two schemas, and browsing one. `normalizeDefinitionText` is shared with Lokee |
| `migrations` | Generating DDL, ordering drops, validating a plan |
| `lokee-weave` | Content-addressed schema versioning and revert |
| `sql-editor` | FoxScript parsing, code cells, SELECT aliasing, the fail-closed SQL subset Redis and MongoDB run (`sql-subset.ts`) |
| `access` | Permission intent, effective access, GRANT/REVOKE and account DDL (facades; emitters live in `providers/<dialect>`) |
| `utilities` | DBA queries: server insights, index fragmentation (`isSafeIndexFragmentationCustomSql`: one read-only SELECT; Db2 defrag is `providers/db2/db2.index-fragmentation.ts`), backup/restore command builders, SQL Tracer, safe column change (`planColumnChange` in `column-change.ts`; per-family SQL in `providers/<d>/*.column-change.ts`, registered in `column-change.registry.ts` — engines not listed answer "not yet") |
| `utilities` | DBA queries: server insights, index fragmentation (`isSafeIndexFragmentationCustomSql`: one read-only SELECT; Db2 defrag is `providers/db2/db2.index-fragmentation.ts`), backup/restore command builders, SQL Tracer, safe column change (`planColumnChange` in `column-change.ts`; per-engine SQL in `providers/<d>/*.column-change.ts`, registered in `column-change.registry.ts` — all 14 SQL engines; omit one to answer "not yet"). A type change is `alterTypeSteps` in place by default; large-table mode is `rebuildSteps` + `progressSql` (`columnChangeLargeTable`). |

`dialect` and `sql-text` are the foundations: the other folders build on
them, never the reverse.
Expand Down Expand Up @@ -207,8 +207,11 @@ app/ The application shell and settings screens. Home's Services card
features/ The view registry: `featureRegistry.ts` (each view's rail
button, who may see it, where
someone goes who may not, what it adds to the top toolbar,
and its lazy entry). App.tsx, the toolbar, the activity rail
and prefetch all read it.
and its lazy entry). `NAV_FLAGS.workflow` is `false` in 0.3.1:
Workflow is off the rail and the command palette, and a stored
`activeView` for it is sent Home (`redirectFor`). The view and
`/api/workflow` stay; set the flag `true` to put it back.
App.tsx, the toolbar, the activity rail and prefetch all read it.
features/ One folder per business domain. The files at its root are its
public API; its folders (components/, state/, lib/, api/) are
internal.
Expand Down Expand Up @@ -291,7 +294,7 @@ the page-epoch guard, bookmarks and recents, SQL variables.
| Calling an API endpoint | use `api` from `@/shared/api/client` — never `fetch` directly |
| SQL for permissions or accounts | `packages/sql/src/modules/access/` (facade) + `packages/sql/src/providers/<dialect>/*.user-sql.ts` / `*.access-sql.ts`. Db2 OS-user docker steps: `buildDb2OsUserInstructions`. |
| Backup / restore commands for an engine | `packages/sql/src/providers/<dialect>/*.backup.ts` registered in `modules/utilities/backup.registry.ts`. Saved folder/format defaults: `packages/server/src/features/backup/`. |
| Safe column-change SQL for an engine | `packages/sql/src/providers/<dialect>/*.column-change.ts` registered in `modules/utilities/column-change.registry.ts`. The facade (`planColumnChange`) finds impact, blockers, and lossy type-conversion warnings; omit the engine to answer "not yet" rather than guessed SQL. The panel is `apps/web/.../ColumnChangePanel.tsx`; Run / Execute all go through `POST /api/sql/execute`. |
| Safe column-change SQL for an engine | `packages/sql/src/providers/<dialect>/*.column-change.ts` registered in `modules/utilities/column-change.registry.ts`. The facade (`planColumnChange`) finds impact, blockers, and lossy type-conversion warnings; omit the engine to answer "not yet" rather than guessed SQL. Implement `alterTypeSteps` (in-place default) and optionally `rebuildSteps` + `progressSql` (large-table mode). A change the engine cannot make is `refuse`. The panel is `apps/web/.../ColumnChangePanel.tsx`; Run / Execute all go through `POST /api/sql/execute`. |
| Home Services / engine reachability | `packages/server/src/features/system/service-status.ts` (`GET /api/system/services`). Types in `packages/shared/src/service-status.ts`. The card is `apps/web/.../app/shell/ServiceStatusCard.tsx`. |
| Home Drivers inventory | `packages/server/src/features/system/drivers.service.ts` (`GET /api/system/drivers`, install/update/remove). Types in `packages/shared/src/driver-inventory.ts`. The card is `apps/web/.../app/shell/DriverInventoryCard.tsx`. |
| A dialect capability the app must branch on | `packages/sql/src/modules/capabilities/` (`dialect-features.ts`) and `packages/sql/src/modules/dialect/` |
Expand Down
4 changes: 3 additions & 1 deletion docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,9 @@ docker compose -f docker-compose.app.yml --profile workflow up -d --build
`WORKFLOW_ENGINE_ENDPOINT` is the unsaved default only. An endpoint saved in
Workflow settings still wins. Leave the profile off and the app runs as Fox
Schema alone. Home → **Services** (`GET /api/system/services`) shows whether
this API can reach the engine; it does not start it.
this API can reach the engine; it does not start it. In **0.3.1** the
Workflow rail item is held back (`NAV_FLAGS.workflow`); the engine and this
proxy still run.

| Variable | Set on | Default | Purpose |
|----------|--------|---------|---------|
Expand Down
Loading
Loading