Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion skills/debugging-gas-issues/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ gc gastownhall/gascity · gastownhall/gastown
├─ dolt dolthub/dolt (mysql-protocol, sql-server, port-based)
│ └─ gms dolthub/go-mysql-server (the SQL engine — planning, indexes)
│ └─ vitess dolthub/vitess (MySQL protocol/parser)
├─ doltlite dolthub/doltlite (SQLite-backed — different diagnostics, no sql-server)
├─ doltlite dolthub/doltlite (SQLite-backed; linked OR backend-plugin = bd-backend-doltlite + gc-doltlite-fastpath — no sql-server/port)
└─ driver dolthub/driver
```

Expand Down
19 changes: 15 additions & 4 deletions skills/debugging-gas-issues/references/gas-stack-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The symptom surfaces high in the stack (a `gc` command feels slow, an agent stal
| go-mysql-server (gms) | `dolthub/go-mysql-server` | The SQL engine embedded in dolt: query planning, index selection, joins, expression eval | a dolt dependency (check dolt's `go.mod`) |
| vitess | `dolthub/vitess` | MySQL wire protocol + SQL parser used by gms | a gms dependency |
| driver | `dolthub/driver` | The Go `database/sql` driver gc/bd use to talk to a dolt server | a gc/bd dependency |
| doltlite | `dolthub/doltlite` (`doltlite-python`) | SQLite-backed version-controlled store — an alternative to dolt; **no mysql sql-server, no port** | the consumer's build cache; an on-disk SQLite file |
| doltlite | `dolthub/doltlite` + the beads **backend plugin** (`bd-backend-doltlite`) & gc **fastpath** (`gc-doltlite-fastpath`) | SQLite-backed version-controlled store — an alternative to dolt; **no mysql sql-server, no port**. In the *plugin* deployment bd/gc are plain (unlinked) and talk to a `…serve` subprocess over stdio | `.beads/doltlite/*.db`; the co-located plugin binaries (e.g. `~/.local/lib/beads-plugin/`); `.beads/metadata.json` names them |

## Ground-truth sources (consult before tracing or changing behavior)

Expand Down Expand Up @@ -43,9 +43,20 @@ git -C <source> log -1 --format='%H %ci %s' # source HEAD
# mismatch -> your trace may not reflect the binary; rebuild or check out the right commit
```

## dolt vs doltlite (don't assume mysql)
## Which data plane? — detect the backend FIRST (don't assume mysql)

- **dolt**: a MySQL-protocol `sql-server` on a TCP port. Diagnose with `SHOW PROCESSLIST`, `information_schema.processlist`, global status, `ss`/`lsof` on the port. Port resolution: `--port` flag > city `dolt.port` config > `<rig>/.beads/dolt-server.port` file > legacy default.
- **doltlite**: a version-controlled **SQLite** file. **No server, no port, no PROCESSLIST.** Diagnose with SQLite tooling against the file (`.dolt`/`.doltlite` dir), file size on disk, and the consuming process's own profiling. The CPU-vs-load, binary-grep, bead-store-layout, and dogfood techniques still apply; the *server* techniques do not. Check which backend the city/store actually uses before reaching for `SHOW PROCESSLIST`.
A beads store can sit on any of several data-plane backends, and their diagnostics differ completely — so identify the store's backend and **trace to that layer's repo** (the stack table above). `.beads/metadata.json` is the ground truth — read it *before* reaching for `SHOW PROCESSLIST`. The common ones are below; the list isn't closed (others may exist), so key off metadata, not a fixed set:

```bash
jq -r '.backend, (.backend_plugin_command // "—")' .beads/metadata.json # backend + plugin cmd (or —)
ls -d .beads/dolt .beads/doltlite 2>/dev/null # which store dir exists
pgrep -af 'dolt sql-server|bd-backend-doltlite|gc-doltlite-fastpath' # which serve procs are live
```

- **dolt (sql-server)** — `backend: dolt`; a MySQL-protocol server on a TCP port, store under `.beads/dolt`. Diagnose with `SHOW PROCESSLIST`, `information_schema.processlist`, global status, `ss`/`lsof` on the port. Port resolution: `--port` > city `dolt.port` > `<rig>/.beads/dolt-server.port` > legacy default.
- **doltlite, linked** — `backend: doltlite`, **no** `backend_plugin_command`; DoltLite compiled *into* bd/gc. Version-controlled, SQLite-backed; **no server, no port, no PROCESSLIST.**
- **doltlite, backend-plugin** — `backend: doltlite` **with** `backend_plugin_command` set. bd/gc are plain (unlinked) and launch `bd-backend-doltlite serve` (bd storage, over stdio, per-invocation) + `gc-doltlite-fastpath serve` (gc's always-on read fastpath). Store is `.beads/doltlite/*.db`; the plugin binaries are co-located (metadata names them). **No TCP server** — you cannot `SHOW PROCESSLIST` or `ss` a port. Inspect via `bd sql` (routed through the plugin), the `…serve` processes, the store dir on disk, and the DoltLite SQL maintenance functions. Deep commands + gotchas (`dolt_gc`, flatten, maintenance, locks, native read fastpath) live in the DoltLite contract itself (`dolthub/doltlite` + its README) and the plugin repos (`duncan4123/beads-backend-doltlite`, `duncan4123/gascity`). A city that imports the `beads-doltlite` pack (`gastownhall/gascity` `//examples/beads-doltlite`) also gets its `doltlite` skill as a convenience layer over those.

Detect which backend a given store uses and **trace to that owning layer (and its repo)** — don't assume. On a shared box the backends can coexist **per city** (one city on one backend, a throwaway on another), so detect per store, never assume per host.

See `gc-diagnostic-toolkit.md` for the concrete commands per backend.
19 changes: 13 additions & 6 deletions skills/debugging-gas-issues/references/gc-diagnostic-toolkit.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ pidstat 3 1 | awk '/^Average:/ && $3 ~ /^[0-9]+$/ {cpu[$NF]+=$8} END{for(c in cp
```
Aggregates sustained CPU by command. (`ps` %CPU is lifetime-average — misleading for "now".)

## dolt server introspection
## dolt server introspection (backend: dolt only — confirm first)

Applies to the **dolt sql-server** backend. On a doltlite backend there is no server/port/PROCESSLIST — skip to the doltlite section. Confirm with `jq -r .backend .beads/metadata.json`.

```bash
PORT=$(cat <rig>/.beads/dolt-server.port 2>/dev/null || echo 3307) # resolution: --port > city dolt.port > port-file > legacy
Expand Down Expand Up @@ -68,11 +70,16 @@ gc dolt-cleanup --probe --json # orphan dbs / stale procs (NEVER --force wit
```
Disk reclaim after deleting rows is a **dolt** GC: `CALL DOLT_GC('--full')` (online-safe; quiesce writers first). `gc dolt compact` gates on commit count and skips low-commit/high-churn dbs.

## doltlite (SQLite backend)
## doltlite (embedded backend — linked or plugin)

No server/port/PROCESSLIST. Inspect the file:
No server/port/PROCESSLIST. First confirm which doltlite deployment (see `gas-stack-map.md` → "Which data plane?"):
```bash
jq -r '.backend, (.backend_plugin_command // "—")' .beads/metadata.json # doltlite + (plugin cmd | —)
pgrep -af 'bd-backend-doltlite|gc-doltlite-fastpath' # plugin serve procs (plugin model)
du -sh .beads/doltlite 2>/dev/null # store size on disk
```
Query **through bd** (routed to the backend — works for linked and plugin), not a raw `sqlite3` open of the `.db` (that bypasses the working-set/commit model and can read a stale or locked view):
```bash
du -sh <store>/.doltlite 2>/dev/null
sqlite3 <store-file> 'SELECT issue_type, COUNT(*) FROM issues GROUP BY issue_type;' # adapt to the actual schema/path
bd sql -q "SELECT issue_type, COUNT(*) n, SUM(status='closed') closed FROM issues GROUP BY issue_type ORDER BY n DESC"
```
The CPU-vs-load, binary-grep, store-layout, and dogfood techniques still apply; the server techniques do not. **Check the backend before reaching for `SHOW PROCESSLIST`.**
Disk reclaim / maintenance is a **DoltLite** operation, not `CALL DOLT_GC` over a wire — use the DoltLite SQL functions (`SELECT dolt_gc();`, flatten, maintenance) and mind the store locks. The authoritative references for those, the native read fastpath, and DoltLite lock/maintenance gotchas are the DoltLite contract (`dolthub/doltlite` + README) and the plugin repos (`duncan4123/beads-backend-doltlite`, `duncan4123/gascity`); the `beads-doltlite` pack's `doltlite` skill (if your city imports it) collects them. The CPU-vs-load, binary-grep, store-layout, and dogfood techniques still apply; the server techniques (`PROCESSLIST`, `ss` on a port, `DOLT_GC('--full')` over the wire) do not. **Check the backend before reaching for `SHOW PROCESSLIST`.**