diff --git a/.changeset/18dce807.md b/.changeset/18dce807.md new file mode 100644 index 000000000..9eb3a1b89 --- /dev/null +++ b/.changeset/18dce807.md @@ -0,0 +1,5 @@ +--- +"aicodeman": minor +--- + +Config backups: timestamped snapshots of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases and, in multi-user mode, accounts), taken at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder and the retention (keep N backups, drop older than N days) are configurable in Settings → System, with Back up now and Restore. Retention only deletes folders Codeman made, never the newest backup. A comparison view shows what changed between two backups, or between a backup and the running config, without ever showing a credential's value. diff --git a/CLAUDE.md b/CLAUDE.md index bb78756b9..357c1db42 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -156,6 +156,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Auto-resume on usage limit** (opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit, `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time and `SessionAutoOps` arms a timer for reset+2min, then sends Esc + `continue`. ⚠️ Respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected`), which is what prevents `/clear` from wiping the paused conversation. Claude-mode only. → [architecture-invariants#auto-resume-on-usage-limit](docs/architecture-invariants.md#auto-resume-on-usage-limit) +**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 2-500 default 20, `configBackupKeepDays` 0-3650 default 30 (an ABSENT key means what the newest backup in the folder recorded, `withRecordedRetention`); `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases (incl. `remote-cases.json`), webviews, push-keys, users, `.env`) into `/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. ⚠️ A custom folder is probed with `probePathKind(dir, { pastCap: true })` before every pass and in each route (#516: it may be a network mount; `unknown` skips the pass / answers 503 `OPERATION_FAILED`), and a missing custom folder is only created when its parent is present. ⚠️ Each manifest records an `origin`, a random id kept in `/config-backup-id` (NOT the hostname: a Docker container gets a new one per recreate, which orphaned every snapshot); change check, prune, list, restore and diff are scoped to it so instances sharing one folder do not snapshot or prune each other. Restoring `intents.json` calls `intentStore.reload()`; `settings.json` applies its watcher toggles after a restart; Settings re-reads `/api/settings` into the form after a restore of it, since Save would otherwise PUT the old values back. ⚠️ Restoring `clis.json` writes it directly (the one writer besides `mutateRegistryFile()`) and then calls `reloadCliRegistry()`. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md) + **Plan-usage chip** (`showPlanUsageLimits`, per-device: desktop default **ON**, handhelds OFF): resolve DISPLAY only through `planUsageChipEnabled()` (settings-ui.js). The same setting is the server-side COLLECTION switch, read fresh by `readPlanUsageTelemetryEnabled()` (hooks-config.ts) at every claude create/respawn. ⚠️ An ABSENT key reads as ON in the reader; `GET /api/settings` must never write. ⚠️ A save sends `showPlanUsageLimits` ONLY when it flips the chip on that device (`planUsageCollectionFlip()`), or a phone switches collection off for every desktop. Claude data comes from the statusLine exporter, injected as an EPHEMERAL `claude --settings` flag (`resolveStatusLineCliCommand`), never written to disk, WRAPPING a user's own statusLine, posting to `POST /api/status-telemetry`. Codex comes from a read-only `account/rateLimits/read` poll (main bucket only). The statusline's `prompt_cache` object (Claude Code v2.1.251+) is kept by `StatusTelemetrySchema` (every documented scalar field, null-tolerant; `parsePromptCache()` in usage-telemetry.ts normalizes only the stored subset) and shown in the footer as `cache:until HH:MM` / `cache:cold` (only when the user has no statusline of their own) and in Session Options → Respawn (`#promptCacheStatus`, fed by `Session.promptCache` in `toState()` plus the change-only `session:promptCache` SSE event, worded by the pure `CodemanPromptCache.format`): warm-until/cold, the re-cache stake, the hit rate, the last miss's causes, and a `Context compacted at HH:MM` note after a compaction or a tool-result clearing (stamped on the first warm report whose `recache_tokens_if_cold` is null after one that carried it, never from `expected_rebuilds`, which Claude Code bumps only at the next request; NEVER an alert). ⚠️ `Session.setPromptCache()` stamps `compactedAt` onto a NEW object (never mutating the parsed report) and the route broadcasts the STORED object; the last report is restored on restart (constructor `promptCache`, like `displayModel`) and the readout is time-aware (a passed `expiresAt` reads cold, not stale-warm). It is the source of truth for the cache TTL, never the plan tier. → [architecture-invariants#plan-usage-chip-statusline-telemetry](docs/architecture-invariants.md#plan-usage-chip-statusline-telemetry), `docs/usage-limits-display-plan.md` **Orchestrator**: State machine that turns a user goal into a phased plan and drives it to completion: `idle → planning → approval → executing → verifying → (replanning) → completed/failed`. `OrchestratorLoop` (engine) delegates plan generation to `orchestrator-planner` and per-phase verification gates to `orchestrator-verifier`, executing phases via team agents/`task-queue`. State persists under the `orchestrator` key in `state.json`. Distinct from Ralph (single-session autonomous loop) — orchestrator coordinates multi-phase, multi-agent execution. See `docs/orchestrator-loop-architecture.md`. @@ -170,7 +172,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Docker Compose deployment** (`docker/`): Codeman runs in a container and spawns Docker cases as **SIBLING** containers via the host socket, never nested. `resolveDockerDaemonMountSource()` maps HOME bind sources into the daemon's namespace (`CODEMAN_DOCKER_HOST_HOME`); `CODEMAN_CASES_PATH` makes workspaces resolve to the same absolute path on both sides. ⚠️ `CODEMAN_CASES_PATH` must move every consumer: resolve it only via `config/cases-dir.ts`. ⚠️ `.dockerignore` matches whole paths: keep `**/.env` or `docker/.env` secrets ship in the image. ⚠️ Long-form binds create missing sources ROOT-OWNED: `Start-Codeman.sh` pre-creates them, and `docker/entrypoint.sh` (root, `cap_add: [CHOWN, DAC_OVERRIDE, KILL, SETGID, SETUID]` against `cap_drop: ALL`, `KILL` for tini; pinned by the test) fixes ownership then drops to `PUID:PGID` via `setpriv`, never re-owning foreign dirs. ⚠️ Append `/opt/codeman-cli` and `~/.local/bin` to `PATH`, never prepend. ⚠️ `server.Dockerfile`, the compose file and `.env.example` feed the self-updater's environment gate (`docs/docker-self-update.md`). → [architecture-invariants#docker-compose-deployment](docs/architecture-invariants.md#docker-compose-deployment), `docs/docker-compose.md` -**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts`, which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md` +**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts` (the one exception: a config-backup restore of `clis.json`, which then calls `reloadCliRegistry()`), which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md` **External CLI modes (OpenCode, Codex, Gemini, Antigravity, Pi, Grok, DeepSeek, OMP, Copilot)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token parsing, ❯ readiness; readiness is output stabilization); work detection is per-CLI `capabilities.workDetect` data, not this gate. All nine **require tmux, no direct PTY fallback** (secrets go via socket-scoped `tmux setenv`, never the command line). ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope. ⚠️ **Codex uses predictive write-through echo, never the buffer overlay**: `_predictHookOnData` must never `return` (wire path stays byte-identical), and flushed text and a bracketed paste must go out as separate delayed writes. ⚠️ **Pi**: no bypass flag, never invent one; `approveProjectTrust` executes repo code, so it is in the clamp's **materialize** branch; never wire `--api-key`. ⚠️ **Grok**: `alwaysApprove` is stripped for non-granted owners (only-if-sent). ⚠️ **DeepSeek**: the agent is a PROFILE (Run gates on `isDeepSeekRunnable()`); the permission switch is the `DSH_PERMISSION_MODE` env var, so `clampEnvOverridesForOwner()` must DROP `DSH_PERMISSION_MODE`, `DSH_HOME` and `DEEPSEEK_BASE_URL` for non-granted owners; `hooksAvailableForMode()` is per-SESSION for it (pass `sessionHookOptions(session)`) and is never a stand-in for `mode === 'claude'`; answers come from `deepseek-transcript.ts`, paired by header `cwd` + boot window, never newest-mtime. ⚠️ **OMP**: `OMP_AUTH_BROKER_URL`/`_TOKEN` are clamped the same way. ⚠️ **Copilot**: `allowAll` (`--yolo`) is a `privilegedParam` and `COPILOT_HOME`/`COPILOT_ALLOW_ALL`/`COPILOT_PROVIDER_*`/`COPILOT_MODEL` are privileged env keys, and so is every other `COPILOT_*` variable the CLI reads that switches approvals, runs code with no approval row or sends its sign-in token elsewhere (the 1.0.98 list is in `stock.ts`; the prefix admits whatever a later release adds, so re-read the bundle whenever the measured version moves); the resume id must start with a letter or digit (`path-segment`), since `--resume`'s value is optional and `--resume --yolo` would otherwise make `--yolo` its own flag; `--name` is dropped next to `--resume`/`--continue` (Copilot refuses the pair); its mouse DECSETs are stripped (`strip-mux-and-mouse`) so a drag selects and the wheel is forwarded by `_shouldForwardWheelToApp` while `cliMouseTracking` is on; `composerReadyMark` is the footer hint `/ commands · ? help`, NOT `❯` (the folder-trust dialog draws `❯ 1. Yes`); `last-response` reads the pane's visible screen (`transcriptNeedsScreenCapture`) through a per-CLI `PANE_DIALECTS` row, because its stream is cursor moves, not lines (a prompt row scrolled off by a tall answer reads as the answer's tail unless a banner/trust dialog is on screen, and typed composer text is not a prompt); Docker seeds no Copilot credential (`config.json` holds a plain-text token without a keyring). → [architecture-invariants#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp) diff --git a/README.md b/README.md index d3ad103a9..a9a4e3c22 100644 --- a/README.md +++ b/README.md @@ -479,6 +479,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt - **OSC 52 clipboard**: a program in a session that sets the clipboard with OSC 52 (Claude Code's `/copy`, a tmux yank, vim) puts the text on your browser's clipboard, in the main terminal and in every tile. Clipboard read requests are refused, and replaying a buffer never copies - **MCP server sync** _(opt-in)_: copies the user-level MCP servers each installed CLI has (Claude, Codex, Gemini, OpenCode, Antigravity, and GitHub Copilot CLI) into the others' own config files. Additive only, a server you switched off is never copied, **Preview** shows the changes first, and the previous file is kept as `.codeman-bak`. App Settings → Agents & CLIs; admin only in multi-user mode - **CJK / IME input** — full composition support for Chinese / Japanese / Korean, with Ctrl- and Alt-modified navigation keys passed through to the CLI +- **Config backups** — timestamped copies of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases, remote and Docker hosts, saved dashboards and, in multi-user mode, accounts), made at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder, how many to keep (2–500) and how long (days, 0 = no limit) are settings, with **Back up now** and a verified **Restore** in Settings → System. Retention only ever deletes folders Codeman made and never the newest. See the [wiki page](docs/wiki/Config-Backups.md) - **Plan usage in the header** — live Claude subscription usage (the 5-hour and weekly windows) from a statusline exporter Codeman hands to `claude` at spawn and never writes into your settings files, plus Codex limits from its own app-server; per device, on for desktops and off for phones - **Prompt-cache readout**: each Claude session shows whether its prompt cache is warm (`cache:until HH:MM`) or cold, in the footer (when you have no statusline of your own) and in Session Options → Respawn, with the cost of a re-cache and a note after a compaction. Display only; it survives a restart - **Session list, your way** — the header strip, a left sidebar with a filter box, or a vertical rail whose detailed rows carry created and state stamps and sort by activity; the phone home screen and the desktop home rail use the same order. On the rail, **Search sessions** narrows the list to the tabs that match, across every group, and a tab with an alert stays visible even when its name does not. Named **tab groups** collapse, reorder and take drags; a group's menu has **New session**, and a worker an agent spawns lands right after its parent (**Spawned tabs follow their parent**, off by default, keeps it moving with the parent). **Add to Focus** pins a shortcut to any tab in a section at the top of the rail, and **CLI Logos on Tabs** can be switched off per device diff --git a/docs/api-reference.md b/docs/api-reference.md index c0350a66b..ae5e881c9 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1152,6 +1152,21 @@ Result (`data`): The result carries server **names** only, never `env` values, `headers` or file content. Each changed file keeps its previous content as `.codeman-bak` (overwritten by each sync); a file that receives servers carrying `env` or `headers` is left mode `0600`. +## Config backups + +Timestamped snapshots of the files that hold what a user has customized (`src/config-backup.ts`). On by default; a snapshot is taken at startup and whenever the tracked files' content changes (checked every 5 minutes), so there is one entry per real change. All four routes are admin only in multi-user mode (`403`), because the files include credentials. Responses carry file names and sizes; only the diff returns values, and never a credential's. + +Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not. + +Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (2–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). A key absent from `settings.json` means the retention the newest backup in the folder recorded, and the default only when none did. Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, the change-check `hash`, an `origin` id, the `keepCount` and `keepDays` in force, and a SHA-256 and size per file), all `0600`. A snapshot is never dated earlier than the instance's newest one, so a clock that read ahead once does not make every later pass write a copy. All four routes answer `503 OPERATION_FAILED` when a custom folder does not respond, and diff and restore answer `409` for a symlinked or group/world-accessible backup as well as a checksum mismatch. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`. + +| Method | Path | Body | Notes | +| ------ | ------------------------------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `GET` | `/api/config-backups` | none | `settings` in effect, `dir`, `defaultDir`, `tracked[]` and `backups[]` (`id`, `createdAt`, `reason`, `appVersion`, `files[]`, `bytes`), newest first. | +| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet); `removed[]` lists the ids retention deleted. `503` when a custom folder does not answer. | +| `GET` | `/api/config-backups/diff?from=&to=` | none | What differs between two states: `from`/`to` are a backup id or `current` (the running config; the default for `to`). `files[]` (`name`, `status` `added`/`removed`/`changed`/`unchanged`, `changes[]` of `path`, `kind`, `before`/`after`, `truncated`, `detail`) and `changed`. Credential values are never returned (`redacted: true`). `400` without `from`; `404` unknown id; `409` if a file fails its checksum. | +| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones. `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). Returns `restored[]`, `untouched[]` (tracked files present now that the backup did not hold; left as they are), `safetySnapshot` (the `pre-restore` id) and `restartRequired[]` (files the running server only reads at start: `.env`, `push-keys.json`, `settings.json`). | + ## Webhook notifications Posts the Web Push events to ntfy, Slack, Discord or a generic JSON URL (Settings → Notifications). Off by default. The webhook URL is a bearer secret (anyone holding a Slack/Discord URL can post as it), so it lives in `~/.codeman/webhook.json` (0600), is **never returned**, and is kept out of `settings.json`. All three routes answer `403` for a non-admin in multi-user mode. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index a50539c86..4c9bdbde2 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -113,7 +113,7 @@ Further detail: ⚠️ An ADOPTED container may back SEVERAL cases at different Spawn commands are pinned as literal strings in `test/cli-registry-spawn-golden.test.ts`, remote/docker pane commands in `test/location-overlay-commands.test.ts`. ⚠️ That second golden no longer covers **remote claude or remote omp**: both now have their own arm in `buildRemoteLaunchCommand` (a `--session-id || --resume` pair, and `--continue`, so a respawn continues the same conversation) and never reach `defaultRemoteCommandForMode`, which is what that test asserts. Their real pins are `toContain` substrings in `test/tmux-manager.test.ts` and `test/remote-shared-sessions.test.ts`; changing either arm will NOT fail the golden. -⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()`: mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`. +⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()` (the one exception is a config-backup restore of `clis.json`, which writes it atomically and calls `reloadCliRegistry()`): mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`. ## Session data and lifecycle diff --git a/docs/wiki/Config-Backups.md b/docs/wiki/Config-Backups.md new file mode 100644 index 000000000..5dabfd243 --- /dev/null +++ b/docs/wiki/Config-Backups.md @@ -0,0 +1,248 @@ +# Config Backups + +Codeman keeps timestamped copies of the files that hold what you have customized, so a bad +write, a hand edit or a botched restart can be undone. It is on by default, needs no setup, +and everything about it (where the copies go, how many are kept, how long) is a setting. + +## What is backed up + +A fixed list of files from the data directory (`~/.codeman/`, or the directory of the +[instance](Core-Concepts#instances) you run): + +| File | What it holds | +| ---- | ------------- | +| `settings.json` | Your preferences that sync across devices, including model and CLI settings. | +| `clis.json` | CLI enable/disable and custom CLI entries. | +| `custom-model-hosts.json` | Saved custom model endpoints. **Holds API keys.** | +| `webhook.json` | Webhook notification target. | +| `intents.json` | Read My Mind per-case profiles. | +| `linked-cases.json` | Registered cases. | +| `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json` | Remote SSH and Docker overlay configuration. | +| `webviews.json` | Saved dashboard URLs. | +| `push-keys.json` | Web-push keys (regenerating them drops every device's subscription). | +| `users.json` | Multi-user accounts. **Holds password hashes.** | +| `.env` | Credentials the TUI client reads. **Holds a password.** | + +Only files that exist are included. This is an allowlist: nothing else in the data +directory is ever copied. + +**Not backed up:** `state.json` (sessions, and also cron jobs and respawn presets: it is rewritten +with every session event, so it would bury the real changes), logs, `mux-sessions.json`, the hook +secret, Docker seeds and caches. Back `state.json` up with your normal file backups if you rely on +cron jobs. + +## When a backup is made + +- **At startup**, so the state the server found is always on record. +- **Whenever the content changes.** The files are read and hashed every five minutes; a new + backup is written only if the set differs from the latest backup. An idle server writes + nothing, and the history is one entry per real change rather than one per timer tick. + A field a file rewrites on its own is not a change, and the comparison looks through it (the + snapshot still stores the file exactly as written, and a restore puts those fields back as they + were): `custom-model-hosts.json` `lastDiscoveredAt` (the model-discovery poll, every five + minutes), `users.json` `lastLoginAt`, `settings.json` `lastUsedCase`, `intents.json` + `recentPrompts` and `updatedAt` (every captured prompt) and `docker-cases.json` + `lastClaudeSessionId` (a Docker case's conversation id after a `/clear`) and `webviews.json` + `lastOpenedAt` (opening a web tab). Without that, every + tick looked like an edit and 20 copies covered only about 100 minutes. So Compare shows + `intents.json` as unchanged when only prompts differ, yet a restore drops the prompts captured + since. +- **On demand**, with **Back up now** (always writes one, even if nothing changed). +- **Before a restore**, as a `pre-restore` safety copy of the files about to be replaced. + +## Where they go and what they are called + +The default folder is `/backups/config` (`~/.codeman/backups/config`). You can point +it anywhere with an absolute path (a leading `~` is expanded). A path that is not absolute, or +that is the data directory itself, falls back to the default. + +Each backup is one folder named for the moment it was made, in the server's local time: + +``` +~/.codeman/backups/config/ + cfg-20261011-120305/ + manifest.json id, createdAt (epoch ms), reason, appVersion, hash, origin, + keepCount and keepDays in force, SHA-256 + size per file + settings.json the original file names, unchanged + clis.json + custom-model-hosts.json + ... + cfg-20261011-123012/ + cfg-20261011-123012-2/ a second backup in the same second gets -2, -3, ... +``` + +| Part | Meaning | +| ---- | ------- | +| `cfg-YYYYMMDD-HHMMSS` | Date and time (server local time) the backup was made. A collision in the same second gets `-2`, `-3`, … | +| `manifest.json` | The record of the backup: `reason`, `appVersion`, the change-check `hash`, the `origin` id, the retention in force (`keepCount`, `keepDays`), and a SHA-256 and size for every file. | +| `reason` | `startup`, `auto` (a change was detected), `manual` (Back up now) or `pre-restore`. It is in the manifest and the Settings list, not in the folder name. | +| `.tmp-…` | A backup being written. It is renamed into place when complete and is never listed. | + +A folder counts as a backup only if its name matches that pattern **and** it has a valid +`manifest.json`. That is what makes it safe to point the folder at a directory that also holds +other things. + +## Retention + +Two limits, applied after every backup: + +| Setting | Range | Default | Meaning | +| ------- | ----- | ------- | ------- | +| **Keep this many backups** (`configBackupKeepCount`) | 2 – 500 | 20 | Keep the newest N; older ones are removed. (1 is not offered: every change would delete the state before it.) | +| **Delete backups older than (days)** (`configBackupKeepDays`) | 0 – 3650 | 30 | Also remove a backup once it has been replaced for longer than this. `0` means no age limit. | + +Both apply together. Four guarantees: + +- **The newest backup is never removed**, whatever the numbers say, so there is always one to + go back to. +- **Only folders Codeman made are ever removed**, never anything else that happens to live in a + custom folder. +- **Age counts from when a backup was replaced**, not from when it was made. Backups are made on + change, so a config that stays the same for months has one backup whose own date keeps + ageing; it is the copy you need the day something goes wrong, so it stays until a newer one + has existed for longer than the limit. +- Because backups are made on change, 20 backups reach back across 20 *changes*, however long + that took, not 20 timer ticks. + +## Settings → System → Config backups + +| Control | Setting | Default | +| ------- | ------- | ------- | +| Back up my configuration | `configBackupEnabled` | On (absent means on) | +| Backup folder | `configBackupDir` | empty = `/backups/config` | +| Keep this many backups | `configBackupKeepCount` | 20 | +| Delete backups older than (days) | `configBackupKeepDays` | 30 | +| **Back up now** | | | +| The list, with **Restore**, **Compare with current** and **Changes in this backup** per entry | | | +| **Compare** … with … **Show changes** | | | + +The settings are saved server-side and read fresh, so a change applies on the next pass. The +list and **Back up now** use the **saved** folder and retention: apply your edits first. + +## Restoring + +From the list, press **Restore** on a backup and confirm. Codeman then: + +1. **Verifies** every file in the backup against the SHA-256 in its manifest. If any file is + missing or does not match, nothing is changed and you get an error. It also refuses a backup + folder or file that is a symlink or that group or others can read or write (Codeman never + makes one like that, and writing it back as `0600` would turn it into trusted config). The + permission test is skipped when the backup folder's filesystem does not keep file modes + (FAT/exFAT drives, SMB shares, WSL's `/mnt/d` without the `metadata` option report the same + mode for every file, so it would refuse every backup there). The symlink test is not. Compare + applies the same checks to the backups it reads. +2. **Takes a `pre-restore` backup** of the files about to be replaced, so a restore can itself + be undone. +3. **Replaces each file atomically** (written beside it, then renamed) with mode `0600`. + +Only the tracked names are ever written, whatever a manifest lists. + +Restoring `settings.json` keeps the four backup settings (on or off, folder, how many, how many +days) at their current values, so a restore from before a folder change does not point the list +at the old folder and hide the `pre-restore` copy it just made. The exception is the two +retention values: where the live `settings.json` lacks one (a reset file), the restore keeps the +backup's own, key by key, so the next pass does not fall back to 20 copies and 30 days. The +`pre-restore` copy records the retention in force too. + +Settings are re-read on the next request, so reload the page. `.env` and `push-keys.json` are +read once per process: **restart Codeman** for those to take effect (the response lists which). +`clis.json` is reloaded by the restore itself and `users.json` is re-read within a second. Restoring `settings.json` also does not re-run the switches +that saving in Settings applies at once (file watchers and the like), so those apply after a +restart, and `settings.json` is listed with the restart-required files. A restore does not touch +running sessions. + +When the backup folder is shared (a synced folder used by two machines, or prod and a beta), +each snapshot records a random id that the instance made once and keeps in its data directory +(`config-backup-id`; not the hostname, which a Docker container changes on every recreate). Every +instance only snapshots, lists, prunes, compares and restores its own. Deleting that file makes +the instance a new owner, and its earlier snapshots stop being listed (they stay on disk). The +same happens after a reinstall of the data directory. To see them again, copy the `origin` value +from any of their `manifest.json` files into `/config-backup-id`. + +You can also restore by hand: the files in a backup folder are the originals, so copying one back +over the file in the data directory is equivalent (stop Codeman first if you are replacing +`clis.json` or `users.json`). + +## Comparing backups + +Before you restore, or to find out what changed and when, use **Compare** in the list: + +- **Compare with current** on a backup shows what differs between that backup and the config + running now. A restore puts back the files the backup holds; files created since are left as + they are, and the restore response names them (`untouched`). +- **Changes in this backup** shows what differs between the backup before it and this one. +- The **Compare … with … Show changes** pickers compare any two backups, or a backup with the + current config. + +Each file that differs is listed with its changed keys (`theme.mode`, `tags[1]`, a variable name +in `.env`) as before → after, plus files that appeared or vanished between the two states. Files +that are the same are listed on one line. A file that is not JSON shows only that it differs, and +a file with more than 200 changes is cut short, so use the files themselves for a full diff. The +self-rewritten fields listed under "When a backup is made" are not counted as changes. + +**Credential values are never shown.** `.env`, `users.json`, `push-keys.json`, `webhook.json` (the +URL is a bearer secret) and `intents.json` (captured prompts) list only the names that changed, and in every other file a key that looks like a token, secret, password, +credential or key is shown as "changed" without its value. URL values lose any `user:password@`, +query and fragment. + +## Security + +- Some tracked files carry credentials (`.env`, `users.json`, `custom-model-hosts.json`, + `push-keys.json`, `webhook.json`). Each backup folder is mode `0700` and its files `0600`, so the default + location exposes nothing the data directory does not already. +- **A custom folder is your decision.** Putting it on a shared drive, a synced folder or a + network share copies those credentials there, with whatever permissions that location has. + Choose a private location, or turn the feature off if you cannot. +- **Retention survives a bad `settings.json`.** If it does not parse, a pass still takes its + snapshot but deletes nothing. If a retention setting is missing (a reset file), the pass + prunes with the retention the newest backup that recorded one holds, not the 20 copies and 30 + days defaults, and Settings shows those values. Saving sends a retention value only if it was + saved before or you edited it there, so saving other settings (or re-entering a custom folder + after a reset) does not write the displayed 20 and 30 over it; the first pass in the re-entered + folder finds the retention its own backups recorded. While `settings.json` does not + parse, a pass reads it as empty, so its snapshot goes to the default folder even if a custom + folder is set or backups are switched off; nothing is widened (same modes, inside the data + directory). +- **Anyone who can write to a custom backup folder can plant a backup that Restore applies.** + Restore refuses symlinks and files others can write, but a file of your own user's making + passes, so keep the folder private. +- In [multi-user mode](Multi-User-Mode) all of it is admin only: the list, Back up now, + Restore and Compare, and each restore is written to the admin audit log + (`config_backup_restore`). +- Listing returns file names and sizes, never contents. A comparison returns changed values but + never a credential's (see [Comparing backups](#comparing-backups)). + +## HTTP API + +```bash +API=http://localhost:3000 +curl -s "$API/api/config-backups" | jq # settings, folder, list +curl -s -X POST "$API/api/config-backups" | jq # back up now +curl -s -X POST "$API/api/config-backups/cfg-20261011-120305/restore" | jq +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305" | jq # that backup vs the running config +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305&to=cfg-20261012-090000" | jq +``` + +See [HTTP API](HTTP-API) and the endpoint reference in `docs/api-reference.md`. + +## Troubleshooting + +- **The list is empty.** None of the tracked files existed yet, or backups are turned off. Press + **Back up now**. +- **"Backups are not available here".** You are not an admin in multi-user mode. +- **No new backup after I changed a setting.** Changes are picked up within five minutes; **Back up + now** takes one immediately. +- **A restore said a file does not match its checksum.** The backup was altered or damaged on + disk. Nothing was changed; restore an earlier one. +- **The backups folder is growing.** Lower **Keep this many backups**. Each backup is a few + kilobytes, but `custom-model-hosts.json` and `intents.json` can be larger. +- **I want my own off-machine copy.** Point the folder at a location your backup tool already + covers, mind the Security note above, and keep **Keep this many backups** high enough that + your tool sees each one. + +## Read next + +- [Settings Reference](Settings-Reference) - where the controls live. +- [Core Concepts](Core-Concepts) - what is on disk. +- [Security](Security) - what is protected and how. +- [Troubleshooting](Troubleshooting) - "my settings were reset". diff --git a/docs/wiki/Core-Concepts.md b/docs/wiki/Core-Concepts.md index 60da25e6f..bc41e354f 100644 --- a/docs/wiki/Core-Concepts.md +++ b/docs/wiki/Core-Concepts.md @@ -135,6 +135,7 @@ Everything Codeman knows lives under `~/.codeman/`: | `users.json` | Multi-user accounts, mode 0600. | | `push-*.json` | Web push keys and subscriptions. | | `certs/` | Self-signed TLS for `--https`. | +| `backups/config/` | Timestamped copies of the files above that you customized ([Config Backups](Config-Backups)). | None of it needs root, none of it leaves the machine, and deleting `~/.codeman/` resets Codeman to a fresh install without touching your code. diff --git a/docs/wiki/HTTP-API.md b/docs/wiki/HTTP-API.md index 0f940586e..6feb15178 100644 --- a/docs/wiki/HTTP-API.md +++ b/docs/wiki/HTTP-API.md @@ -173,6 +173,10 @@ curl -s "$API/api/subagents" | jq # background agents curl -s "$API/api/search?q=deploy" | jq # cross-session search curl -s "$API/api/mcp-sync" | jq # preview MCP server sync (opt-in: 403 until mcpSyncEnabled is on) curl -s -X POST "$API/api/mcp-sync" | jq # apply it: add missing servers to each CLI config, never edit/remove +curl -s "$API/api/config-backups" | jq # config snapshots: folder, retention, list (admin only in multi-user mode) +curl -s -X POST "$API/api/config-backups" | jq # back up the customized config files now +curl -s -X POST "$API/api/config-backups/cfg-20261011-120000/restore" | jq # put a snapshot back (after a pre-restore copy) +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120000" | jq # what differs from the running config (`to=` another snapshot id to compare two) # with ID set to a session id: curl -s "$API/api/sessions/$ID/last-response" | jq -r '.data.text' # last answer, from the transcript (claude, codex, deepseek) diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index d14c60ea5..50c108bba 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -113,6 +113,7 @@ codeman web # then open http://localhost:3000 | Page | What it answers | | --------------------------------------------- | -------------------------------------------------- | | [Running As A Service](Running-As-A-Service) | How do I keep it up across reboots, and update it? | +| [Config Backups](Config-Backups) | How do I get my settings back after they were changed or lost? | | [Troubleshooting](Troubleshooting) | Why is it doing that? | | [FAQ](FAQ) | The questions that keep coming up. | | [Contributing](Contributing) | How do I send a fix? | diff --git a/docs/wiki/Security.md b/docs/wiki/Security.md index 8dbaaa388..5d72ac5e5 100644 --- a/docs/wiki/Security.md +++ b/docs/wiki/Security.md @@ -107,6 +107,11 @@ browser and your server. Your agent CLI's traffic is its own, on your account. Two features send data outward, both off by default and both stated where they appear: voice dictation through your Claude login, and the Read My Mind prediction call. +[Config Backups](Config-Backups) copies some credential-bearing files (`.env`, `users.json`, +`custom-model-hosts.json`, `push-keys.json`, `webhook.json`) into a backup folder. The default folder is inside +the data directory, and every backup is mode `0700`/`0600`, so nothing new is exposed. Only a +folder you choose yourself can move them somewhere less private; admin only in multi-user mode. + ## Reporting a vulnerability **Never in a public issue.** diff --git a/docs/wiki/Settings-Reference.md b/docs/wiki/Settings-Reference.md index 2e7238ce5..a5287f9b5 100644 --- a/docs/wiki/Settings-Reference.md +++ b/docs/wiki/Settings-Reference.md @@ -190,6 +190,14 @@ Cloudflare tunnel controls including the tunnel URL. The **Diagnostics** group r tools with their versions and install hints (admin only in multi-user mode). In multi-user mode, the **Users** administration entry is injected here. +The **Config backups** group controls timestamped copies of your configuration files: an +on/off switch (on by default), the backup folder (default `~/.codeman/backups/config`), how +many backups to keep (2 to 500, default 20) and how many days to keep them (0 to 3650, default +30; 0 means no age limit). A value that is not saved in `settings.json` (a reset file) is whatever +the newest backup in that folder recorded, and the default only when none did, plus **Back up now** and a list with **Restore**. These are +server-side settings, read fresh. Details, naming and restore behaviour are on +[Config Backups](Config-Backups). + ## Session Options Per session, from the tab. @@ -243,3 +251,4 @@ Some things are configured before the server starts, not in the UI: - [The Dashboard](The-Dashboard) - what each control does once visible. - [Keeping Agents Running](Keeping-Agents-Running) - the Respawn panel in depth. - [Agent CLIs](Agent-CLIs) - model, effort, and permission modes. +- [Config Backups](Config-Backups) - copies of your settings, and how to restore them. diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md index 114ca37de..c63326861 100644 --- a/docs/wiki/Troubleshooting.md +++ b/docs/wiki/Troubleshooting.md @@ -131,6 +131,32 @@ That prompt comes from Claude Code, not Codeman. Codeman's default is to start w permission prompts skipped, which is what the security model is built around. If you would rather it prompted, change **App Settings → Agents & CLIs → Claude → Startup Mode**. +### My settings were reset, or a CLI I enabled is gone + +Something rewrote a file in `~/.codeman/` (a hand edit, a restart that loaded a bad file, a +test run pointed at the real data directory). If [Config Backups](Config-Backups) was on, +which is the default, you have copies: + +1. Open **Settings → System → Config backups** and pick the newest backup from before the + change. The list shows when each was made and what it holds. +2. **Restore** it, reload the page, and restart Codeman if the response's `restartRequired` + list is not empty. + +A reset `settings.json` also drops the backup folder setting. If you use a custom folder, enter +it again and save before looking for the list; until then the list shows the default folder. +After a reinstall of the data directory the old backups are unlisted because the instance has a +new id; see [Config Backups](Config-Backups) for how to make them visible again. + +A backup is never dated earlier than the newest one, so if the clock was once set to a far +future date, every later backup keeps that date. Delete the future-dated backup folder to +return to normal dating. + +Without the UI: the backups are plain folders in `~/.codeman/backups/config/` +(`cfg-YYYYMMDD-HHMMSS`); copy the file you need back over the one in `~/.codeman/`. + +Not covered: `state.json` (cron jobs and respawn presets live there). A backup only exists +from the moment the feature first ran, so a reset that happened before that cannot be undone. + ### Sessions vanished after a reboot Expected. tmux does not survive a reboot, so the sessions are gone. Conversations are not: diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 62e0e7a77..0b85b925c 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -51,6 +51,7 @@ **Operating it** - [Running As A Service](Running-As-A-Service) +- [Config Backups](Config-Backups) - [Troubleshooting](Troubleshooting) - [FAQ](FAQ) - [Contributing](Contributing) diff --git a/src/config-backup.ts b/src/config-backup.ts new file mode 100644 index 000000000..34b4ec864 --- /dev/null +++ b/src/config-backup.ts @@ -0,0 +1,820 @@ +/** + * @fileoverview Config backups: timestamped snapshots of the files that hold what a user has + * customized, kept in a default-but-configurable folder with configurable retention. + * + * Why it exists: those files are plain JSON in the data dir, rewritten by the server, the Settings UI, + * the CLI and (once, by a mis-configured test run) by things that should never touch them, and there + * was no copy to go back to. A snapshot is taken at startup and whenever the tracked files' content + * changes, so the history is one entry per actual change rather than one per timer tick. + * + * Safety rules the code enforces rather than documents: + * - The tracked file list is a fixed allowlist, never "everything in the data dir". + * - Retention only ever deletes folders this module made (name pattern AND a parseable manifest), so + * pointing the backup folder at a directory that holds other things cannot delete them. + * - The newest snapshot is never pruned, whatever the retention numbers say. + * - A snapshot is built in a temp folder and renamed into place, so a crash leaves no half snapshot. + * - Restore verifies each file against the manifest's SHA-256 and takes a `pre-restore` snapshot first. + * - Snapshot folders are 0700 and their files 0600: some tracked files carry credentials. + * + * Pure of the server: every function takes the directories it works in. + * + * @module config-backup + */ + +import { createHash, randomUUID } from 'node:crypto'; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + readdirSync, + readFileSync, + renameSync, + rmSync, + statSync, + writeFileSync, +} from 'node:fs'; +import { homedir } from 'node:os'; +import { isAbsolute, join, resolve, sep } from 'node:path'; + +/** + * The files (relative to the data dir) a snapshot holds: what a user configured, not what the server + * regenerates (session state, logs, the hook secret, docker seeds, caches). + * + * ⚠️ Not tracked: `state.json`. It also holds cron jobs and respawn presets, but it is rewritten with + * every session event, so tracking it would make a snapshot per event and bury the real changes. + * + * ⚠️ `.env`, `users.json`, `custom-model-hosts.json` and `push-keys.json` carry credentials or keys. + * They are already in the data dir with the same permissions; a snapshot folder is 0700 with 0600 + * files, so the default location widens nothing. A custom location is the user's own choice. + */ +export const CONFIG_BACKUP_FILES = [ + 'settings.json', + 'clis.json', + 'custom-model-hosts.json', + 'webhook.json', + 'intents.json', + 'linked-cases.json', + 'remote-hosts.json', + 'remote-cases.json', + 'docker-hosts.json', + 'docker-cases.json', + 'webviews.json', + 'push-keys.json', + 'users.json', + '.env', +] as const; + +export const CONFIG_BACKUP_DEFAULTS = { enabled: true, keepCount: 20, keepDays: 30 } as const; +/** 1 would delete the state before every change, so a bad write could never be undone. */ +export const CONFIG_BACKUP_MIN_KEEP_COUNT = 2; +export const CONFIG_BACKUP_MAX_KEEP_COUNT = 500; +export const CONFIG_BACKUP_MAX_KEEP_DAYS = 3650; + +/** `cfg-YYYYMMDD-HHMMSS`, plus `-2`, `-3`… if two land in one second. */ +const SNAPSHOT_ID_RE = /^cfg-\d{8}-\d{6}(?:-\d{1,3})?$/; +const MANIFEST = 'manifest.json'; + +export interface ConfigBackupSettings { + enabled: boolean; + /** Absolute folder, or null for the default (`/backups/config`). */ + dir: string | null; + /** Keep at most this many snapshots (newest first). 2–500. */ + keepCount: number; + /** Also drop snapshots older than this many days; 0 = no age limit. */ + keepDays: number; +} + +export interface ConfigBackupFile { + name: string; + bytes: number; + sha256: string; +} + +export interface ConfigBackupManifest { + id: string; + createdAt: number; + reason: string; + appVersion: string; + /** Digest of the set of files, so an unchanged state is not snapshotted twice. */ + hash: string; + /** Id of the data dir that made it (see {@link configBackupOrigin}), so instances sharing one folder keep to their own snapshots. */ + origin?: string; + /** + * The retention in force when it was made. If `settings.json` later fails to parse or is reset, + * the next pass prunes with these instead of the defaults, so one bad write cannot shorten a + * history the user asked to keep. + */ + keepCount?: number; + keepDays?: number; + files: ConfigBackupFile[]; +} + +export interface ConfigBackupSummary { + id: string; + createdAt: number; + reason: string; + appVersion: string; + files: Array<{ name: string; bytes: number }>; + bytes: number; +} + +const intIn = (value: unknown, min: number, max: number, fallback: number): number => { + const n = typeof value === 'number' ? value : typeof value === 'string' && value.trim() ? Number(value) : NaN; + if (!Number.isFinite(n)) return fallback; + return Math.min(max, Math.max(min, Math.floor(n))); +}; + +/** `~`-expand and require an absolute path; anything else means "use the default". */ +export function normalizeBackupDir(raw: unknown, home: string = homedir()): string | null { + if (typeof raw !== 'string') return null; + let value = raw.trim(); + if (!value || value.includes('\0')) return null; + if (value === '~') value = home; + else if (value.startsWith('~/')) value = join(home, value.slice(2)); + if (!isAbsolute(value)) return null; + return resolve(value); +} + +/** Read the four `configBackup*` settings out of a settings.json object, clamped and defaulted. */ +export function readConfigBackupSettings(raw: Record | null | undefined): ConfigBackupSettings { + const s = raw ?? {}; + return { + // Absent means ON: the feature protects against an event nobody schedules. + enabled: s.configBackupEnabled !== false, + dir: normalizeBackupDir(s.configBackupDir), + keepCount: intIn( + s.configBackupKeepCount, + CONFIG_BACKUP_MIN_KEEP_COUNT, + CONFIG_BACKUP_MAX_KEEP_COUNT, + CONFIG_BACKUP_DEFAULTS.keepCount + ), + keepDays: intIn(s.configBackupKeepDays, 0, CONFIG_BACKUP_MAX_KEEP_DAYS, CONFIG_BACKUP_DEFAULTS.keepDays), + }; +} + +export function defaultConfigBackupDir(dataDir: string): string { + return join(dataDir, 'backups', 'config'); +} + +/** + * The folder snapshots go to: the configured one, or the default. A custom folder that IS the data dir + * falls back to the default, because snapshot folders and the live files must not share a directory. + */ +export function resolveConfigBackupDir(settings: ConfigBackupSettings, dataDir: string): string { + const fallback = defaultConfigBackupDir(dataDir); + const dir = settings.dir; + if (!dir) return fallback; + const data = resolve(dataDir); + if (dir === data) return fallback; + return dir; +} + +const ORIGIN_FILE = 'config-backup-id'; +const ORIGIN_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; + +/** + * Who owns a snapshot: a random id made once and kept in the data dir. A hostname would not do (a + * Docker container gets a new one on every recreate, which orphaned every earlier snapshot), nor + * would the path alone (two machines can share a `$HOME` path). It is a file of its own, not the + * hook secret, and is not tracked or restored. If it cannot be written the id is not persisted, so + * snapshots made meanwhile are not listed later; the data dir being unwritable breaks far more. + */ +export function configBackupOrigin(dataDir: string): string { + const path = join(dataDir, ORIGIN_FILE); + try { + const existing = readFileSync(path, 'utf8').trim(); + if (ORIGIN_RE.test(existing)) return existing; + } catch { + /* missing: made below */ + } + const id = randomUUID(); + try { + mkdirSync(dataDir, { recursive: true }); + writeFileSync(path, id + '\n', { mode: 0o600 }); + } catch { + /* see above */ + } + return id; +} + +/** The id already made for this data dir, or null. Read-only: listing must not write a file. */ +export function readConfigBackupOrigin(dataDir: string): string | null { + try { + const existing = readFileSync(join(dataDir, ORIGIN_FILE), 'utf8').trim(); + return ORIGIN_RE.test(existing) ? existing : null; + } catch { + return null; + } +} + +const sha256 = (data: Buffer | string): string => createHash('sha256').update(data).digest('hex'); + +/** + * Keys a tracked file rewrites on its own schedule, with no change to anything the user set, so + * they must not make a snapshot look different. `custom-model-hosts.json` carries a + * `lastDiscoveredAt` per host that the model-discovery poll refreshes every 5 minutes: before this, + * the change check never matched and the snapshot folder filled with copies that differed by one + * timestamp, evicting every older snapshot within about 100 minutes at the default keep count. + * The snapshot itself still stores the file exactly as it was, and restore still verifies against + * the file's own digest; only the "has anything changed?" comparison looks through these keys. + */ +const VOLATILE_KEYS: Readonly> = { + 'custom-model-hosts.json': ['lastDiscoveredAt'], + // Rewritten by activity, not by anything the user set: a Basic-auth login (at most once a minute + // per user), a session started in another case, every captured prompt, and a Docker case's + // conversation id after a /clear, and a web tab being opened. + 'users.json': ['lastLoginAt'], + 'settings.json': ['lastUsedCase'], + 'intents.json': ['recentPrompts', 'updatedAt'], + 'docker-cases.json': ['lastClaudeSessionId'], + 'webviews.json': ['lastOpenedAt'], +}; + +function withoutKeys(value: unknown, drop: ReadonlySet): unknown { + if (Array.isArray(value)) return value.map((v) => withoutKeys(v, drop)); + if (value && typeof value === 'object') { + const out: Record = {}; + for (const [k, v] of Object.entries(value as Record)) { + if (!drop.has(k)) out[k] = withoutKeys(v, drop); + } + return out; + } + return value; +} + +/** Digest used only to decide whether two states are the same: the file's own, minus its volatile keys. */ +function changeDigest(name: string, data: Buffer, rawSha: string): string { + const volatile = VOLATILE_KEYS[name]; + if (!volatile) return rawSha; + try { + return sha256(JSON.stringify(withoutKeys(JSON.parse(data.toString('utf8')), new Set(volatile)))); + } catch { + return rawSha; // not JSON (or unreadable as such): compare the bytes, as for any other file + } +} + +function stamp(now: number): string { + const d = new Date(now); + const p = (n: number, w = 2) => String(n).padStart(w, '0'); + return `${d.getFullYear()}${p(d.getMonth() + 1)}${p(d.getDate())}-${p(d.getHours())}${p(d.getMinutes())}${p(d.getSeconds())}`; +} + +function readManifest(dir: string, id: string, origin?: string): ConfigBackupManifest | null { + if (!SNAPSHOT_ID_RE.test(id)) return null; + try { + const m = JSON.parse(readFileSync(join(dir, id, MANIFEST), 'utf8')) as ConfigBackupManifest; + if (!m || m.id !== id || !Array.isArray(m.files) || typeof m.createdAt !== 'number' || typeof m.hash !== 'string') { + return null; + } + if (origin !== undefined && m.origin !== origin) return null; + return m; + } catch { + return null; + } +} + +/** + * Snapshots this module made in `dir`, newest first. Foreign folders and files are ignored, and so + * are other instances' snapshots when `origin` is given (see {@link configBackupOrigin}). + */ +export function listConfigBackups(dir: string, origin?: string): ConfigBackupSummary[] { + let names: string[]; + try { + names = readdirSync(dir); + } catch { + return []; + } + const out: ConfigBackupSummary[] = []; + for (const id of names) { + const m = readManifest(dir, id, origin); + if (!m) continue; + const files = m.files.map((f) => ({ name: f.name, bytes: f.bytes })); + out.push({ + id, + createdAt: m.createdAt, + reason: m.reason, + appVersion: m.appVersion, + files, + bytes: files.reduce((n, f) => n + f.bytes, 0), + }); + } + return out.sort((a, b) => b.createdAt - a.createdAt || (a.id < b.id ? 1 : -1)); +} + +export type ConfigBackupResult = + | { status: 'created'; id: string; files: number; bytes: number } + | { status: 'unchanged'; id: string } + | { status: 'empty' }; + +export interface CreateConfigBackupOptions { + dataDir: string; + backupDir: string; + reason: string; + appVersion?: string; + /** Snapshot even if nothing changed (a manual request, a pre-restore safety copy). */ + force?: boolean; + /** Recorded in the manifest, see {@link ConfigBackupManifest.keepCount}. */ + keepCount?: number; + keepDays?: number; + now?: number; +} + +/** Snapshot the tracked files that exist, unless the latest snapshot already holds exactly this state. */ +export function createConfigBackup(opts: CreateConfigBackupOptions): ConfigBackupResult { + const now = opts.now ?? Date.now(); + const present: Array<{ name: string; data: Buffer; sha: string; change: string }> = []; + for (const name of CONFIG_BACKUP_FILES) { + try { + const path = join(opts.dataDir, name); + if (!statSync(path).isFile()) continue; + const data = readFileSync(path); + const sha = sha256(data); + present.push({ name, data, sha, change: changeDigest(name, data, sha) }); + } catch { + /* absent or unreadable: simply not part of this snapshot */ + } + } + if (present.length === 0) return { status: 'empty' }; + + const hash = sha256(present.map((f) => `${f.name}\0${f.change}`).join('\n')); + const origin = configBackupOrigin(opts.dataDir); + const existing = listConfigBackups(opts.backupDir, origin); + if (!opts.force && existing[0]) { + const latest = readManifest(opts.backupDir, existing[0].id, origin); + if (latest && latest.hash === hash) return { status: 'unchanged', id: latest.id }; + } + + // Never dated earlier than this instance's newest snapshot. One written while the clock read ahead + // (a dual-boot machine, a VM resumed with a stale clock) would otherwise stay newest after the + // correction, differ from every later pass, and have each one write a copy until real time caught up. + const createdAt = existing[0] ? Math.max(now, existing[0].createdAt + 1) : now; + mkdirSync(opts.backupDir, { recursive: true, mode: 0o700 }); + const base = `cfg-${stamp(createdAt)}`; + let id = base; + for (let n = 2; existsSync(join(opts.backupDir, id)); n++) id = `${base}-${n}`; + + const tmp = join(opts.backupDir, `.tmp-${id}-${process.pid}`); + rmSync(tmp, { recursive: true, force: true }); + mkdirSync(tmp, { mode: 0o700 }); + try { + const manifest: ConfigBackupManifest = { + id, + createdAt, + reason: opts.reason, + appVersion: opts.appVersion ?? '', + hash, + origin, + ...(opts.keepCount !== undefined && { keepCount: opts.keepCount }), + ...(opts.keepDays !== undefined && { keepDays: opts.keepDays }), + files: present.map((f) => ({ name: f.name, bytes: f.data.length, sha256: f.sha })), + }; + for (const f of present) writeFileSync(join(tmp, f.name), f.data, { mode: 0o600 }); + writeFileSync(join(tmp, MANIFEST), JSON.stringify(manifest, null, 2) + '\n', { mode: 0o600 }); + renameSync(tmp, join(opts.backupDir, id)); + try { + chmodSync(join(opts.backupDir, id), 0o700); + } catch { + /* best effort on filesystems without modes */ + } + } catch (err) { + rmSync(tmp, { recursive: true, force: true }); + throw err; + } + return { status: 'created', id, files: present.length, bytes: present.reduce((n, f) => n + f.data.length, 0) }; +} + +/** Remove `.tmp-cfg-*` build folders a crashed snapshot left behind (older than an hour, so a live one on a shared folder is safe). */ +export function sweepConfigBackupTemps(dir: string, now: number = Date.now()): void { + let names: string[]; + try { + names = readdirSync(dir); + } catch { + return; + } + for (const n of names) { + if (!/^\.tmp-cfg-\d{8}-\d{6}(?:-\d{1,3})?-\d+$/.test(n)) continue; + const st = lstatSync(join(dir, n), { throwIfNoEntry: false }); + if (st && now - st.mtimeMs > 3_600_000) rmSync(join(dir, n), { recursive: true, force: true }); + } +} + +/** The retention recorded in this instance's newest snapshot, or null when it recorded none. */ +export function latestBackupRetention(dir: string, origin: string): { keepCount: number; keepDays: number } | null { + // The newest manifest that recorded one: a snapshot from before this field existed must not hide it. + for (const b of listConfigBackups(dir, origin)) { + const m = readManifest(dir, b.id, origin); + if (m && typeof m.keepCount === 'number' && typeof m.keepDays === 'number') { + return { keepCount: m.keepCount, keepDays: m.keepDays }; + } + } + return null; +} + +/** + * The settings in effect, with each retention key that `raw` (the parsed settings.json) lacks, as + * after a reset, taken from what the newest backup recorded in `dir` instead of the defaults. It + * looks at the retention keys only (not the folder), so re-entering a custom folder after a reset + * finds that folder's own manifests. One place for the timer pass and for what the GET route + * reports, so they cannot disagree. + */ +export function withRecordedRetention( + settings: ConfigBackupSettings, + raw: Record, + dir: string, + origin: string | null +): ConfigBackupSettings { + const lacksCount = !('configBackupKeepCount' in raw); + const lacksDays = !('configBackupKeepDays' in raw); + if ((!lacksCount && !lacksDays) || !origin) return settings; + const kept = latestBackupRetention(dir, origin); + if (!kept) return settings; + return { + ...settings, + ...(lacksCount && { keepCount: kept.keepCount }), + ...(lacksDays && { keepDays: kept.keepDays }), + }; +} + +export interface PruneOptions { + keepCount: number; + keepDays: number; + /** Only this instance's snapshots are pruned; omitted = every snapshot in the folder. */ + origin?: string; + now?: number; +} + +/** + * Apply retention: keep the newest `keepCount`, and (when `keepDays` > 0) drop anything older than + * that. The newest snapshot always survives. Returns the ids removed. + */ +export function pruneConfigBackups(dir: string, opts: PruneOptions): string[] { + const now = opts.now ?? Date.now(); + const all = listConfigBackups(dir, opts.origin); + const cutoff = opts.keepDays > 0 ? now - opts.keepDays * 86_400_000 : null; + const removed: string[] = []; + all.forEach((b, index) => { + if (index === 0) return; + const tooMany = index >= Math.max(1, opts.keepCount); + // A snapshot's state was live until the next one was made: age it from then. Aging by its own + // createdAt deleted the last good copy on the first change after a quiet month. + const tooOld = cutoff !== null && all[index - 1].createdAt < cutoff; + if (!tooMany && !tooOld) return; + // listConfigBackups already proved this is one of ours (name pattern + manifest). + const target = join(dir, b.id); + if (!target.startsWith(resolve(dir) + sep)) return; + try { + rmSync(target, { recursive: true, force: true }); + removed.push(b.id); + } catch { + /* leave it; the next pass retries */ + } + }); + return removed; +} + +export class ConfigBackupNotFoundError extends Error {} +export class ConfigBackupCorruptError extends Error {} + +/** + * Restore refuses what this module never writes. Snapshots are 0700/0600 real files, and a restore + * writes the file as 0600: a symlink, or a file with a group/world bit, was planted or loosened + * after the snapshot, and rewriting it as 0600 would turn it into trusted config (`clis.json` + * selects the binaries Codeman spawns; `isUnsafePermissions` refuses the same bits there). + * POSIX only, like `isUnsafePermissions`. The mode test is skipped (the symlink test is not) on a + * filesystem that does not keep modes, see {@link folderKeepsModes}. + */ +function assertTrustedSnapshotEntry(path: string, label: string, checkMode: boolean): void { + if (process.platform === 'win32') return; + const st = lstatSync(path, { throwIfNoEntry: false }); + if (!st) return; // missing is reported by the read that follows + if (st.isSymbolicLink()) throw new ConfigBackupCorruptError(`${label} is a symlink, so it is not used`); + if (checkMode && (st.mode & 0o077) !== 0) { + throw new ConfigBackupCorruptError(`${label} is readable or writable by group or others, so it is not used`); + } +} + +/** + * Whether the backup folder's filesystem keeps a file's mode: vfat/exFAT, CIFS shares and WSL's + * `/mnt/d` report 0755 or 0777 for everything, so the group/world test would refuse every backup + * there even though snapshots succeed (`createConfigBackup` already expects mode-less filesystems). + * Probed with a 0600 temp file made with `wx` under a random name; if the probe cannot run, the + * strict test stays on. + */ +function folderKeepsModes(dir: string): boolean { + if (process.platform === 'win32') return false; + // A random name and 'wx' (fail if anything is there): a planted symlink or directory with a + // guessable name must neither be written through nor break the probe. Only a file this call + // created is removed. + const probe = join(dir, `.mode-probe-${randomUUID()}`); + let created = false; + try { + writeFileSync(probe, '', { mode: 0o600, flag: 'wx' }); + created = true; + chmodSync(probe, 0o600); + return (lstatSync(probe).mode & 0o777 & 0o077) === 0; + } catch { + return true; + } finally { + if (created) rmSync(probe, { force: true }); + } +} + +/** The folder's own backup settings stay as they are now: restoring an old `configBackupDir` hides the `pre-restore` copy. */ +const BACKUP_SETTING_KEYS = ['configBackupEnabled', 'configBackupDir', 'configBackupKeepCount', 'configBackupKeepDays']; + +function keepCurrentBackupSettings(snapshot: Buffer, livePath: string): Buffer { + try { + const restored = JSON.parse(snapshot.toString('utf8')) as Record; + const live = JSON.parse(readFileSync(livePath, 'utf8')) as Record; + if (!restored || typeof restored !== 'object' || !live || typeof live !== 'object') return snapshot; + let changed = false; + // A reset live file lacks the retention keys. Taking "absent" for them would bring the defaults + // back (20 copies, 30 days) on the next pass, so the backup's own retention stays. + for (const key of BACKUP_SETTING_KEYS) { + if (!(key in live) && (key === 'configBackupKeepCount' || key === 'configBackupKeepDays')) continue; + if (JSON.stringify(restored[key]) === JSON.stringify(live[key])) continue; + changed = true; + if (key in live) restored[key] = live[key]; + else delete restored[key]; + } + if (!changed) return snapshot; // byte-identical to the snapshot when the folder settings did not move + return Buffer.from(JSON.stringify(restored, null, 2) + '\n'); + } catch { + return snapshot; // not JSON, or no live file to take the values from + } +} + +export interface RestoreResult { + restored: string[]; + /** Tracked files present now that the snapshot did not hold; a restore leaves them as they are. */ + untouched: string[]; + /** The snapshot taken of the live files before they were overwritten. */ + safetySnapshot: string | null; +} + +/** + * Put a snapshot's files back into the data dir. Every file is checked against the manifest's digest + * BEFORE anything is written, a `pre-restore` snapshot of the live files is taken first, and each + * file is replaced atomically (temp + rename) with mode 0600. + */ +export function restoreConfigBackup(opts: { + dataDir: string; + backupDir: string; + id: string; + appVersion?: string; + now?: number; + /** Override the probe of whether the backup folder keeps file modes (tests; a mode-less filesystem). */ + keepsModes?: boolean; +}): RestoreResult { + const origin = readConfigBackupOrigin(opts.dataDir); + const manifest = origin ? readManifest(opts.backupDir, opts.id, origin) : null; + if (!manifest || !origin) throw new ConfigBackupNotFoundError(`No config backup named ${opts.id}`); + + const allowed = new Set(CONFIG_BACKUP_FILES); + const loaded: Array<{ name: string; data: Buffer }> = []; + const checkMode = opts.keepsModes ?? folderKeepsModes(opts.backupDir); + assertTrustedSnapshotEntry(join(opts.backupDir, opts.id), `Backup ${opts.id}`, checkMode); + for (const f of manifest.files) { + if (!allowed.has(f.name)) continue; + let data: Buffer; + assertTrustedSnapshotEntry(join(opts.backupDir, opts.id, f.name), `${f.name} in ${opts.id}`, checkMode); + try { + data = readFileSync(join(opts.backupDir, opts.id, f.name)); + } catch { + throw new ConfigBackupCorruptError(`${f.name} is missing from ${opts.id}`); + } + if (sha256(data) !== f.sha256) + throw new ConfigBackupCorruptError(`${f.name} in ${opts.id} does not match its checksum`); + loaded.push({ name: f.name, data }); + } + if (loaded.length === 0) throw new ConfigBackupCorruptError(`${opts.id} holds no restorable files`); + + const safety = createConfigBackup({ + dataDir: opts.dataDir, + backupDir: opts.backupDir, + reason: 'pre-restore', + appVersion: opts.appVersion, + force: true, + now: opts.now, + // The retention in force, so this newest snapshot does not hide it from the next pass. + ...(latestBackupRetention(opts.backupDir, origin) ?? {}), + }); + + mkdirSync(opts.dataDir, { recursive: true }); + const restored: string[] = []; + for (const f of loaded) { + const target = join(opts.dataDir, f.name); + const tmp = `${target}.restore-${process.pid}`; + try { + writeFileSync(tmp, f.name === 'settings.json' ? keepCurrentBackupSettings(f.data, target) : f.data, { + mode: 0o600, + }); + renameSync(tmp, target); + } catch (err) { + rmSync(tmp, { force: true }); + throw err; + } + restored.push(f.name); + } + // A restore never deletes: tracked files the snapshot did not hold stay as they are. + const untouched = CONFIG_BACKUP_FILES.filter( + (n) => !restored.includes(n) && lstatSync(join(opts.dataDir, n), { throwIfNoEntry: false })?.isFile() + ); + return { restored, untouched, safetySnapshot: safety.status === 'created' ? safety.id : null }; +} + +/** The live files in the data dir, as a side of a diff. */ +export const CONFIG_DIFF_CURRENT = 'current'; + +export interface ConfigDiffChange { + /** Dotted path inside the file (`a.b[0].c`), or the variable name for `.env`. */ + path: string; + kind: 'added' | 'removed' | 'changed'; + /** JSON text of the value; absent when it did not exist on that side or `redacted` is set. */ + before?: string; + after?: string; + /** The value is a credential, so only the fact that it differs is reported. */ + redacted?: boolean; +} + +export interface ConfigDiffFile { + name: string; + /** `added` = only in `to`, `removed` = only in `from`. */ + status: 'added' | 'removed' | 'changed' | 'unchanged'; + changes: ConfigDiffChange[]; + /** More changes than `MAX_DIFF_CHANGES`; the rest are not listed. */ + truncated: boolean; + /** `keys` = per-key changes; `bytes` = the file is not JSON/env, so only "differs" is known. */ + detail: 'keys' | 'bytes'; +} + +export interface ConfigDiff { + from: string; + to: string; + files: ConfigDiffFile[]; + /** Files whose status is not `unchanged`. */ + changed: number; +} + +const MAX_DIFF_CHANGES = 200; +const MAX_DIFF_VALUE_CHARS = 200; + +/** Files whose every value is a credential or key: only names are ever shown. */ +const REDACT_ALL_FILES: ReadonlySet = new Set([ + '.env', + 'users.json', + 'push-keys.json', + 'webhook.json', + 'intents.json', +]); +/** Elsewhere a value is hidden when its own key looks like a credential. */ +const SECRET_KEY_RE = /(token|secret|passw|credential|auth|key$)/i; + +type Leaves = Map; + +function flattenJson(value: unknown, path: string, out: Leaves): void { + if (Array.isArray(value) && value.length > 0) { + value.forEach((v, i) => flattenJson(v, `${path}[${i}]`, out)); + } else if (value && typeof value === 'object' && Object.keys(value).length > 0) { + for (const [k, v] of Object.entries(value as Record)) { + flattenJson(v, path ? `${path}.${k}` : k, out); + } + } else { + out.set(path || '(root)', JSON.stringify(value) ?? 'null'); + } +} + +function parseEnv(text: string): Leaves { + const out: Leaves = new Map(); + for (const line of text.split(/\r?\n/)) { + const m = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_.-]*)\s*=(.*)$/.exec(line); + if (m) out.set(m[1], m[2].trim()); + } + return out; +} + +/** Per-key view of a tracked file, or null when it is not JSON/env text. */ +function leavesOf(name: string, data: Buffer): Leaves | null { + const text = data.toString('utf8'); + if (name === '.env') return parseEnv(text); + try { + const parsed = JSON.parse(text) as unknown; + const volatile = VOLATILE_KEYS[name]; + const out: Leaves = new Map(); + flattenJson(volatile ? withoutKeys(parsed, new Set(volatile)) : parsed, '', out); + return out; + } catch { + return null; + } +} + +/** A URL value loses its userinfo, query and fragment: they carry passwords and tokens. */ +function scrubUrl(jsonText: string): string { + try { + const v = JSON.parse(jsonText) as unknown; + if (typeof v !== 'string' || !/^[a-z][a-z0-9+.-]*:\/\//i.test(v)) return jsonText; + const u = new URL(v); + if (!u.username && !u.password && !u.search && !u.hash) return jsonText; + return JSON.stringify(`${u.protocol}//${u.host}${u.pathname}`); + } catch { + return jsonText; + } +} + +const clip = (s: string): string => (s.length > MAX_DIFF_VALUE_CHARS ? `${s.slice(0, MAX_DIFF_VALUE_CHARS)}…` : s); + +function diffLeaves(name: string, a: Leaves, b: Leaves): { changes: ConfigDiffChange[]; truncated: boolean } { + const all = REDACT_ALL_FILES.has(name); + const changes: ConfigDiffChange[] = []; + for (const path of [...new Set([...a.keys(), ...b.keys()])].sort()) { + const before = a.get(path); + const after = b.get(path); + if (before === after) continue; + const kind = before === undefined ? 'added' : after === undefined ? 'removed' : 'changed'; + if (all || SECRET_KEY_RE.test(path.split('.').pop() ?? '')) { + changes.push({ path, kind, redacted: true }); + } else { + changes.push({ + path, + kind, + ...(before !== undefined && { before: clip(scrubUrl(before)) }), + ...(after !== undefined && { after: clip(scrubUrl(after)) }), + }); + } + } + return { changes: changes.slice(0, MAX_DIFF_CHANGES), truncated: changes.length > MAX_DIFF_CHANGES }; +} + +/** The tracked files of one side, verified: a snapshot's against its manifest, the live ones as they are. */ +function loadDiffSide(side: string, dataDir: string, backupDir: string): Map { + const out = new Map(); + if (side === CONFIG_DIFF_CURRENT) { + for (const name of CONFIG_BACKUP_FILES) { + try { + const path = join(dataDir, name); + if (statSync(path).isFile()) out.set(name, readFileSync(path)); + } catch { + /* absent or unreadable: not part of this side */ + } + } + return out; + } + const origin = readConfigBackupOrigin(dataDir); + const manifest = origin ? readManifest(backupDir, side, origin) : null; + if (!manifest) throw new ConfigBackupNotFoundError(`No config backup named ${side}`); + const allowed = new Set(CONFIG_BACKUP_FILES); + const checkMode = folderKeepsModes(backupDir); + assertTrustedSnapshotEntry(join(backupDir, side), `Backup ${side}`, checkMode); + for (const f of manifest.files) { + if (!allowed.has(f.name)) continue; + let data: Buffer; + assertTrustedSnapshotEntry(join(backupDir, side, f.name), `${f.name} in ${side}`, checkMode); + try { + data = readFileSync(join(backupDir, side, f.name)); + } catch { + throw new ConfigBackupCorruptError(`${f.name} is missing from ${side}`); + } + if (sha256(data) !== f.sha256) + throw new ConfigBackupCorruptError(`${f.name} in ${side} does not match its checksum`); + out.set(f.name, data); + } + return out; +} + +/** + * What differs between two states of the tracked files; a state is a snapshot id or `current` + * (the live files). Never returns the value of a credential: see REDACT_ALL_FILES and SECRET_KEY_RE. + */ +export function diffConfigBackups(opts: { dataDir: string; backupDir: string; from: string; to: string }): ConfigDiff { + const a = loadDiffSide(opts.from, opts.dataDir, opts.backupDir); + const b = loadDiffSide(opts.to, opts.dataDir, opts.backupDir); + const files: ConfigDiffFile[] = []; + for (const name of CONFIG_BACKUP_FILES) { + const before = a.get(name); + const after = b.get(name); + if (!before && !after) continue; + const base = { name, changes: [] as ConfigDiffChange[], truncated: false }; + if (before && after && sha256(before) === sha256(after)) { + files.push({ ...base, status: 'unchanged', detail: 'keys' }); + continue; + } + const la = before ? leavesOf(name, before) : new Map(); + const lb = after ? leavesOf(name, after) : new Map(); + const status = !before ? 'added' : !after ? 'removed' : 'changed'; + if (!la || !lb) { + files.push({ ...base, status, detail: 'bytes' }); + continue; + } + const { changes, truncated } = diffLeaves(name, la, lb); + // Only volatile keys differed (a discovery timestamp): the same state as far as the user is concerned. + files.push({ + name, + status: changes.length || status !== 'changed' ? status : 'unchanged', + changes, + truncated, + detail: 'keys', + }); + } + return { from: opts.from, to: opts.to, files, changed: files.filter((f) => f.status !== 'unchanged').length }; +} diff --git a/src/intent-store.ts b/src/intent-store.ts index d1592bf92..432a318fb 100644 --- a/src/intent-store.ts +++ b/src/intent-store.ts @@ -108,6 +108,11 @@ export class IntentStore { // ----- Public API ----- + /** Drop the in-memory copy so the next call re-reads intents.json (after a config restore replaced it). */ + reload(): void { + this.profiles = null; + } + /** * The profile for a session's case. Never persists on read: an absent * profile returns an empty transient one (`updatedAt: 0`). diff --git a/src/web/public/index.html b/src/web/public/index.html index abbd2cd9d..d2e9b1554 100644 --- a/src/web/public/index.html +++ b/src/web/public/index.html @@ -3242,6 +3242,48 @@

System

+
+

Config backups

server
+
+
+
+ Back up my configuration + Keeps timestamped copies of your settings, CLI overrides, custom endpoints, webhook, linked cases and (in multi-user mode) accounts. A copy is made at startup and whenever one of them changes. Some of these files hold credentials, so each copy is private to this user (mode 0700/0600). +
+ +
+
+
+ Backup folder + An absolute path (a leading ~ is expanded). Leave empty for the default shown. Only folders Codeman created there are ever pruned. +
+ +
+
+
+ Keep this many backups + 2 to 500 (with 1, every change would delete the state before it). The newest backup is always kept. +
+ +
+
+
+ Delete backups older than (days) + A backup is removed once a newer one has existed this many days. 0 keeps backups until the count above pushes them out. +
+ +
+
+
+ Backups + Folder and retention changes apply after you save. Restoring puts a backup's files back and makes a copy of the current ones first. Compare shows what changed between two backups, or between a backup and the running config; credential values are never shown. +
+ +
+
Loading…
+
+
+

Automation

synced
diff --git a/src/web/public/settings-ui.js b/src/web/public/settings-ui.js index d1343edfc..c40dcd75b 100644 --- a/src/web/public/settings-ui.js +++ b/src/web/public/settings-ui.js @@ -551,6 +551,12 @@ Object.assign(CodemanApp.prototype, { this._mcpSyncSavedOn = settings.mcpSyncEnabled === true; document.getElementById('appSettingsMcpSync').checked = this._mcpSyncSavedOn; this.applyMcpSyncVisibility(); + // Config backups: absent means ON, 20 copies, 30 days. + document.getElementById('appSettingsConfigBackup').checked = settings.configBackupEnabled !== false; + document.getElementById('appSettingsConfigBackupDir').value = settings.configBackupDir || ''; + document.getElementById('appSettingsConfigBackupKeepCount').value = settings.configBackupKeepCount ?? 20; + document.getElementById('appSettingsConfigBackupKeepDays').value = settings.configBackupKeepDays ?? 30; + this.loadConfigBackups(); this._applyDoctorAdminGate(); this.loadWebhook(); // Read My Mind: synced, default OFF (opt-in; capture + prediction cost real tokens). @@ -1358,10 +1364,171 @@ Object.assign(CodemanApp.prototype, { * wired to `codeman:me` for the same late-resolving role as the groups above. */ _applyDoctorAdminGate() { - const group = document.getElementById('doctorGroup'); - if (!group) return; const me = window.__codemanUser || {}; - group.style.display = me.multiUser && me.role !== 'admin' ? 'none' : ''; + for (const id of ['doctorGroup', 'configBackupGroup']) { + const group = document.getElementById(id); + if (group) group.style.display = me.multiUser && me.role !== 'admin' ? 'none' : ''; + } + }, + + // ── Config backups ────────────────────────────────────────────────────────── + // Server-side files, so these read the SAVED settings: a folder or retention edit applies after Save. + + /** + * The retention keys a Save sends: a value only once it was saved before or edited here. After a reset + * the form merely DISPLAYS what the passes use (the retention the backups recorded), and writing that back + * would replace it with whatever a different folder or an older default says. + */ + _configBackupRetentionPayload(prev) { + const count = document.getElementById('appSettingsConfigBackupKeepCount'); + const days = document.getElementById('appSettingsConfigBackupKeepDays'); + const out = {}; + if (prev.configBackupKeepCount !== undefined || count.dataset.touched) { + out.configBackupKeepCount = Math.min(500, Math.max(2, parseInt(count.value, 10) || 20)); + } + if (prev.configBackupKeepDays !== undefined || days.dataset.touched) { + const n = parseInt(days.value, 10); + out.configBackupKeepDays = Math.min(3650, Math.max(0, Number.isNaN(n) ? 30 : n)); + } + return out; + }, + + /** Fill the backup list, and show the default folder as the input's placeholder. */ + async loadConfigBackups() { + const out = this.$('configBackupList'); + if (!out) return; + const res = await this._api('/api/config-backups'); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + out.textContent = body?.error || 'Backups are not available here (admin only in multi-user mode).'; + return; + } + const data = body.data; + const dirInput = this.$('appSettingsConfigBackupDir') || document.getElementById('appSettingsConfigBackupDir'); + if (dirInput && data.defaultDir) dirInput.placeholder = `default: ${data.defaultDir}`; + // A reset settings.json has none of the four keys, but the passes keep the retention the newest backup + // recorded: show that, or a Save would write 20 / 30 over it. + const saved = this.loadAppSettingsFromStorage(); + const countInput = document.getElementById('appSettingsConfigBackupKeepCount'); + const daysInput = document.getElementById('appSettingsConfigBackupKeepDays'); + if (countInput && saved.configBackupKeepCount === undefined && data.settings) countInput.value = data.settings.keepCount; + if (daysInput && saved.configBackupKeepDays === undefined && data.settings) daysInput.value = data.settings.keepDays; + if (!data.backups.length) { + out.innerHTML = `No backups yet in ${escapeHtml(data.dir)}.`; + return; + } + const kb = (n) => (n < 1024 ? `${n} B` : `${(n / 1024).toFixed(1)} KB`); + this._configBackups = data.backups; + const rows = data.backups.map((b, i) => { + const when = new Date(b.createdAt).toLocaleString(); + const names = b.files.map((f) => escapeHtml(f.name)).join(', '); + const id = escapeHtml(b.id); + // The list is newest first, so the next row is the backup this one followed. + const prev = data.backups[i + 1]; + const vsPrev = prev ? `` : ''; + return `
  • ${escapeHtml(when)} (${escapeHtml(b.reason)}, ${kb(b.bytes)}) + + + ${vsPrev} +
    ${names}
    • `; + }); + const options = (sel) => [``] + .concat(data.backups.map((b) => ``)) + .join(''); + const picker = data.backups.length + ? `
    Compare with +
    ` + : ''; + out.innerHTML = `
    ${data.backups.length} backup${data.backups.length === 1 ? '' : 's'} in ${escapeHtml(data.dir)}
    ${picker}
      ${rows.join('')}
    `; + }, + + /** What differs between two states of the config files: a backup id or `current` (the running config). */ + async showConfigDiff(from, to) { + const out = this.$('configBackupDiff'); + if (!out) return; + out.textContent = 'Comparing…'; + const res = await this._api(`/api/config-backups/diff?from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}`); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + out.textContent = body?.error || 'Could not compare these'; + return; + } + const label = (id) => { + if (id === 'current') return 'the current config'; + const b = (this._configBackups || []).find((x) => x.id === id); + return b ? new Date(b.createdAt).toLocaleString() : id; + }; + const d = body.data; + const verb = { added: 'added', removed: 'removed', changed: 'changed' }; + const files = d.files.filter((f) => f.status !== 'unchanged').map((f) => { + let detail; + if (f.detail === 'bytes') { + detail = '
    Differs (not a key/value file, so no detail).
    '; + } else { + const rows = f.changes.map((c) => { + const val = (v) => `${escapeHtml(v ?? '')}`; + const text = c.redacted ? `${verb[c.kind]} (value hidden: credential)` + : c.kind === 'added' ? `added ${val(c.after)}` + : c.kind === 'removed' ? `removed (was ${val(c.before)})` + : `${val(c.before)} → ${val(c.after)}`; + return `
  • ${escapeHtml(c.path)}: ${text}
    • `; + }); + const more = f.truncated ? '
  • More changes not shown.
    • ' : ''; + detail = f.changes.length ? `
      ${rows.join('')}${more}
    ` : ''; + } + const whole = f.status === 'added' ? ' (new file)' : f.status === 'removed' ? ' (file gone)' : ''; + return `
    ${escapeHtml(f.name)}${whole}${detail}
    `; + }); + const same = d.files.filter((f) => f.status === 'unchanged').map((f) => escapeHtml(f.name)); + out.innerHTML = `
    ${escapeHtml(label(d.from))} → ${escapeHtml(label(d.to))}: ${ + d.changed ? `${d.changed} file${d.changed === 1 ? '' : 's'} differ` : 'no differences'}
    ${files.join('')}${ + same.length ? `
    Unchanged: ${same.join(', ')}
    ` : ''}`; + }, + + async backupConfigNow() { + const btn = this.$('configBackupNowBtn'); + if (btn) btn.disabled = true; + try { + const res = await this._apiPost('/api/config-backups', {}); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + this.showToast(body?.error || 'Backup failed', 'error'); + } else if (body.data.status === 'empty') { + this.showToast('Nothing to back up yet', 'warning'); + } else { + this.showToast(`Backed up ${body.data.files} file${body.data.files === 1 ? '' : 's'}`, 'success'); + } + } finally { + if (btn) btn.disabled = false; + this.loadConfigBackups(); + } + }, + + async restoreConfigBackup(id) { + const names = ((this._configBackups || []).find((b) => b.id === id)?.files || []).map((f) => f.name); + const secret = names.filter((n) => ['.env', 'users.json', 'push-keys.json', 'custom-model-hosts.json', 'webhook.json'].includes(n)); + const list = names.length ? `\n\nFiles replaced: ${names.join(', ')}` : ''; + const warn = secret.length ? `\n\nThis rolls back credentials and accounts (${secret.join(', ')}): passwords, disabled accounts and keys return to how they were then.` : ''; + if (!confirm(`Restore this backup? Its files replace the current ones (a copy of the current files is made first). Some files only take effect after a restart.${list}${warn}`)) return; + const res = await this._apiPost(`/api/config-backups/${encodeURIComponent(id)}/restore`, {}); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + this.showToast(body?.error || 'Restore failed', 'error'); + return; + } + const d = body.data; + const restart = d.restartRequired?.length ? ` Restart Codeman to apply: ${d.restartRequired.join(', ')}.` : ''; + // The form still holds the pre-restore values, and Save would PUT them over the restored file. + if (d.restored.includes('settings.json')) { + await this.loadAppSettingsFromServer(); + this.openAppSettings(); + } + this.showToast(`Restored ${d.restored.length} file${d.restored.length === 1 ? '' : 's'}.${restart}`, 'success'); + this.loadConfigBackups(); }, /** Preview (apply=false) or run (apply=true) the MCP server sync across enabled CLIs. */ @@ -2737,6 +2904,9 @@ Object.assign(CodemanApp.prototype, { customModelEndpointsEnabled: document.getElementById('appSettingsCustomModelEndpoints').checked, cliManagementEnabled: document.getElementById('appSettingsCliManagement').checked, mcpSyncEnabled: document.getElementById('appSettingsMcpSync').checked, + configBackupEnabled: document.getElementById('appSettingsConfigBackup').checked, + configBackupDir: document.getElementById('appSettingsConfigBackupDir').value.trim(), + ...this._configBackupRetentionPayload(_prev), readMyMindEnabled: document.getElementById('appSettingsReadMyMind').checked, ultracodeFloatingWindows: document.getElementById('appSettingsUltracodeFloatingWindows').checked, showMultiMonitorButton: document.getElementById('appSettingsShowMultiMonitorButton').checked, diff --git a/src/web/routes/config-backup-routes.ts b/src/web/routes/config-backup-routes.ts new file mode 100644 index 000000000..e4ecea1fe --- /dev/null +++ b/src/web/routes/config-backup-routes.ts @@ -0,0 +1,280 @@ +/** + * @fileoverview Config backups (src/config-backup.ts). + * + * GET /api/config-backups — settings in effect, the folder, and the snapshots in it. + * POST /api/config-backups — snapshot now (a manual request always makes one). + * POST /api/config-backups/:id/restore — put a snapshot's files back (after a `pre-restore` snapshot). + * GET /api/config-backups/diff?from=&to= — what differs between two snapshots, or a snapshot and the + * running config (`current`, the default for `to`). + * + * The files include credentials (`.env`, `users.json`, endpoint keys), so in multi-user mode all four + * are admin only. Responses carry file NAMES and sizes, never file content, except the diff, which shows + * changed values but never a credential's (`.env`, `users.json`, `push-keys.json`, `webhook.json`, `intents.json` and any key that looks + * like a token/secret/password are reported as "changed" only). Restore replaces files the running server + * may have cached: `restartRequired` says which ones only apply after a restart. + */ + +import { readFile } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import type { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify'; +import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from '../../types.js'; +import { getAuthUser, isAdmin, readJsonConfig, SETTINGS_PATH } from '../route-helpers.js'; +import { appendAdminAudit } from '../admin-audit.js'; +import { reloadCliRegistry } from '../../config/cli-registry/registry.js'; +import { isMultiUserMode } from '../../config/multiuser.js'; +import { getDataDir } from '../../config/instance.js'; +import { intentStore } from '../../intent-store.js'; +import { describeUnknownPath, probePathKind } from '../../utils/bounded-path-probe.js'; +import { dirname } from 'node:path'; +import { + CONFIG_BACKUP_FILES, + CONFIG_DIFF_CURRENT, + ConfigBackupCorruptError, + ConfigBackupNotFoundError, + createConfigBackup, + defaultConfigBackupDir, + diffConfigBackups, + listConfigBackups, + pruneConfigBackups, + readConfigBackupSettings, + resolveConfigBackupDir, + restoreConfigBackup, + configBackupOrigin, + withRecordedRetention, + readConfigBackupOrigin, + sweepConfigBackupTemps, + type ConfigBackupResult, + type ConfigBackupSettings, + type ConfigBackupSummary, + type ConfigDiff, +} from '../../config-backup.js'; + +// Only recorded in each manifest; a missing package.json just leaves it blank. +const APP_VERSION: string = (() => { + try { + return (createRequire(import.meta.url)('../../../package.json') as { version?: string }).version ?? ''; + } catch { + return ''; + } +})(); + +/** Files whose running-process copy only reloads on restart (the rest are read per request; `users.json` + * is re-read within a second, `clis.json` is reloaded below, `intents.json` by `reload()`). */ +const RESTART_REQUIRED = new Set(['.env', 'push-keys.json', 'settings.json']); + +export interface ConfigBackupStatus { + settings: ConfigBackupSettings; + dir: string; + defaultDir: string; + tracked: readonly string[]; + backups: ConfigBackupSummary[]; +} + +/** + * The settings in effect and the folder they name. A custom folder is probed (bounded, #516) BEFORE + * the recorded-retention fallback reads it, so this throws `BackupDirUnavailableError` for a folder + * that does not answer and never lists it; `present` is false for a custom folder that does not exist yet. + */ +export async function readEffectiveBackupSettings(): Promise<{ + settings: ConfigBackupSettings; + dir: string; + present: boolean; +}> { + const raw = await readJsonConfig>(SETTINGS_PATH, 'settings.json', {}); + const base = readConfigBackupSettings(raw); + const dir = resolveConfigBackupDir(base, getDataDir()); + const present = await probeBackupDir(base, dir); + // The same fallback the timer pass uses, so the form never shows 20 / 30 while a pass keeps 100 / 0. + const settings = present ? withRecordedRetention(base, raw, dir, readConfigBackupOrigin(getDataDir())) : base; + return { settings, dir, present }; +} + +/** A user-chosen backup folder that is not safe to touch (bounded probe, as for case paths, #516). */ +export class BackupDirUnavailableError extends Error {} + +/** + * Probe a custom backup folder without blocking the event loop on a dead mount. The default folder + * needs no probe. Returns whether the folder exists; throws when it did not answer, or is not a + * folder. A missing custom folder is only creatable when its parent is present. + */ +export async function probeBackupDir(settings: ConfigBackupSettings, dir: string): Promise { + if (!settings.dir || dir === defaultConfigBackupDir(getDataDir())) return true; + const kind = await probePathKind(dir, { pastCap: true }); + if (kind === 'unknown') + throw new BackupDirUnavailableError(describeUnknownPath('Backup folder', dir, { pastCap: true })); + if (kind === 'file') throw new BackupDirUnavailableError(`Backup folder is a file, not a folder: ${dir}`); + if (kind === 'directory') return true; + const parent = await probePathKind(dirname(dir), { pastCap: true }); + if (parent === 'unknown') { + throw new BackupDirUnavailableError(describeUnknownPath('Backup folder', dirname(dir), { pastCap: true })); + } + if (parent !== 'directory') throw new BackupDirUnavailableError(`Backup folder's parent is missing: ${dirname(dir)}`); + return false; +} + +async function readSettingsForBackup(): Promise | 'unreadable'> { + try { + const parsed: unknown = JSON.parse(await readFile(SETTINGS_PATH, 'utf8')); + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) + ? (parsed as Record) + : 'unreadable'; + } catch (err) { + return (err as NodeJS.ErrnoException).code === 'ENOENT' ? {} : 'unreadable'; + } +} + +/** Snapshot (when changed) and apply retention; shared by the timer and the manual route. */ +export async function runConfigBackup( + reason: string, + opts: { force?: boolean } = {} +): Promise<{ result: ConfigBackupResult; removed: string[] } | null> { + // Read settings.json directly: readJsonConfig answers {} for ANY failure, so a syntax error or a read + // that lands inside PUT /api/settings's non-atomic write used to prune with the defaults (20 / 30 + // days) and drop the longer history the user asked to keep. + const raw = await readSettingsForBackup(); + const unreadable = raw === 'unreadable'; + const fromFile = readConfigBackupSettings(unreadable ? {} : raw); + if (!fromFile.enabled && !opts.force) return null; + const dir = resolveConfigBackupDir(fromFile, getDataDir()); + await probeBackupDir(fromFile, dir); + sweepConfigBackupTemps(dir); + const origin = configBackupOrigin(getDataDir()); + // No backup settings in the file (reset, or unreadable): keep the retention the newest snapshot recorded. + const settings = withRecordedRetention(fromFile, unreadable ? {} : raw, dir, origin); + const result = createConfigBackup({ + dataDir: getDataDir(), + backupDir: dir, + reason, + appVersion: APP_VERSION, + force: opts.force, + keepCount: settings.keepCount, + keepDays: settings.keepDays, + }); + // An unreadable settings.json still gets its snapshot, but no pass deletes anything on its say-so. + const removed = unreadable + ? [] + : pruneConfigBackups(dir, { keepCount: settings.keepCount, keepDays: settings.keepDays, origin }); + return { result, removed }; +} + +export function registerConfigBackupRoutes(app: FastifyInstance): void { + const deny = (req: FastifyRequest, reply: FastifyReply): ApiResponse | null => { + if (isMultiUserMode() && !isAdmin(req)) { + reply.code(403); + return createErrorResponse(ApiErrorCode.FORBIDDEN, 'Admin only in multi-user mode'); + } + return null; + }; + + app.get('/api/config-backups', async (req, reply): Promise> => { + const denied = deny(req, reply); + if (denied) return denied; + let eff: Awaited>; + try { + eff = await readEffectiveBackupSettings(); + } catch (err) { + reply.code(503); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, getErrorMessage(err)); + } + const { settings, dir, present } = eff; + // Read-only: a listing must not create config-backup-id, and without one nothing can be ours. + const origin = readConfigBackupOrigin(getDataDir()); + return { + success: true, + data: { + settings, + dir, + defaultDir: defaultConfigBackupDir(getDataDir()), + tracked: CONFIG_BACKUP_FILES, + backups: present && origin ? listConfigBackups(dir, origin) : [], + }, + }; + }); + + app.post('/api/config-backups', async (req, reply) => { + const denied = deny(req, reply); + if (denied) return denied; + try { + const out = await runConfigBackup('manual', { force: true }); + return { success: true, data: { ...out!.result, removed: out!.removed } }; + } catch (err) { + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Backup failed: ${getErrorMessage(err)}`); + } + }); + + app.get('/api/config-backups/diff', async (req, reply): Promise> => { + const denied = deny(req, reply); + if (denied) return denied; + const q = req.query as { from?: string; to?: string }; + if (typeof q.from !== 'string' || !q.from) { + reply.code(400); + return createErrorResponse(ApiErrorCode.INVALID_INPUT, '`from` is required (a backup id or "current")'); + } + try { + const { dir } = await readEffectiveBackupSettings(); + const data = diffConfigBackups({ + dataDir: getDataDir(), + backupDir: dir, + from: q.from, + to: typeof q.to === 'string' && q.to ? q.to : CONFIG_DIFF_CURRENT, + }); + return { success: true, data }; + } catch (err) { + if (err instanceof ConfigBackupNotFoundError) { + reply.code(404); + return createErrorResponse(ApiErrorCode.NOT_FOUND, err.message); + } + if (err instanceof ConfigBackupCorruptError) { + reply.code(409); + return createErrorResponse(ApiErrorCode.CONFLICT, err.message); + } + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Diff failed: ${getErrorMessage(err)}`); + } + }); + + app.post('/api/config-backups/:id/restore', async (req, reply) => { + const denied = deny(req, reply); + if (denied) return denied; + const { id } = req.params as { id: string }; + try { + const { dir } = await readEffectiveBackupSettings(); + const out = restoreConfigBackup({ dataDir: getDataDir(), backupDir: dir, id, appVersion: APP_VERSION }); + // IntentStore caches the file and rewrites it whole on the next prompt, which would undo the restore. + if (out.restored.includes('intents.json')) intentStore.reload(); + // restoreConfigBackup writes clis.json directly (the one writer besides mutateRegistryFile). + if (out.restored.includes('clis.json')) reloadCliRegistry(); + // A restore of users.json replaces accounts, password hashes, roles and bypass grants. + if (isMultiUserMode()) { + void appendAdminAudit({ + admin: getAuthUser(req).username, + action: 'config_backup_restore', + target: id, + ip: req.ip, + detail: { restored: out.restored }, + }); + } + return { + success: true, + data: { + restored: out.restored, + untouched: out.untouched, + safetySnapshot: out.safetySnapshot, + restartRequired: out.restored.filter((n) => RESTART_REQUIRED.has(n)), + }, + }; + } catch (err) { + if (err instanceof ConfigBackupNotFoundError) { + reply.code(404); + return createErrorResponse(ApiErrorCode.NOT_FOUND, err.message); + } + if (err instanceof ConfigBackupCorruptError) { + reply.code(409); + return createErrorResponse(ApiErrorCode.CONFLICT, `${err.message}. Nothing was changed.`); + } + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Restore failed: ${getErrorMessage(err)}`); + } + }); +} diff --git a/src/web/routes/index.ts b/src/web/routes/index.ts index aa1d721fd..ad34f34db 100644 --- a/src/web/routes/index.ts +++ b/src/web/routes/index.ts @@ -32,6 +32,7 @@ export { registerVoiceRoutes } from './voice-routes.js'; export { registerWebviewRoutes, tryWebviewRefererFallback } from './webview-routes.js'; export { registerTabLayoutRoutes } from './tab-layout-routes.js'; export { registerMcpSyncRoutes } from './mcp-sync-routes.js'; +export { registerConfigBackupRoutes, runConfigBackup } from './config-backup-routes.js'; export { registerWebhookRoutes } from './webhook-routes.js'; export { registerDoctorRoutes } from './doctor-routes.js'; export { diff --git a/src/web/schemas.ts b/src/web/schemas.ts index ce577394d..4223310b5 100644 --- a/src/web/schemas.ts +++ b/src/web/schemas.ts @@ -1521,6 +1521,18 @@ export const SettingsUpdateSchema = z * opt-in. While OFF, GET/POST /api/mcp-sync answer 403 and the Settings controls are hidden. */ mcpSyncEnabled: z.boolean().optional(), + /** + * Config backups (src/config-backup.ts): timestamped snapshots of the customized config files, + * taken at startup and whenever their content changes. SYNCED. Absent means ON (the feature + * protects against an event nobody schedules); the folder defaults to `/backups/config`. + */ + configBackupEnabled: z.boolean().optional(), + /** Absolute folder (a leading `~` is expanded); empty or absent = the default folder. */ + configBackupDir: z.string().max(1024).optional(), + /** Keep at most this many snapshots. */ + configBackupKeepCount: z.number().int().min(2).max(500).optional(), + /** Also drop snapshots older than this many days; 0 = no age limit. */ + configBackupKeepDays: z.number().int().min(0).max(3650).optional(), /** * Read My Mind predictor model override. Empty/absent = the AI-checker * default (opus: prediction quality is the product and it runs only on an diff --git a/src/web/server.ts b/src/web/server.ts index f9bb8b47b..095f1fbde 100644 --- a/src/web/server.ts +++ b/src/web/server.ts @@ -208,6 +208,8 @@ import { registerWebviewRoutes, registerTabLayoutRoutes, registerMcpSyncRoutes, + registerConfigBackupRoutes, + runConfigBackup, registerWebhookRoutes, registerDoctorRoutes, registerCustomModelRoutes, @@ -229,6 +231,9 @@ const __dirname = dirname(fileURLToPath(import.meta.url)); // while capping growth of `sseClientsById` and blocking pathological inputs. const SSE_CLIENT_ID_RE = /^[A-Za-z0-9_-]{8,64}$/; const CODEX_USAGE_POLL_INTERVAL_MS = 5 * 60_000; +// Config backups: one cheap read-and-hash of a handful of small files; a snapshot is written only when +// their content changed since the latest one, so this is one entry per real change, not per tick. +const CONFIG_BACKUP_INTERVAL_MS = 5 * 60_000; const CUSTOM_MODEL_REDISCOVER_INTERVAL_MS = 5 * 60_000; // Much shorter than the model-LIST refresh above on purpose: this catches an actual // eviction (a session's model no longer loaded, silently swapped out by another @@ -1179,6 +1184,7 @@ export class WebServer extends EventEmitter { registerWebviewRoutes(this.app, ctx, this.basePath); registerTabLayoutRoutes(this.app, ctx); registerMcpSyncRoutes(this.app); + registerConfigBackupRoutes(this.app); registerWebhookRoutes(this.app, { notifier: this.webhookNotifier, configDir: getDataDir(), @@ -3067,6 +3073,18 @@ export class WebServer extends EventEmitter { // $CODEMAN_HOOK_SECRET_FILE — hook curls cat that path at execution time. getHookSecret(); + // Config backups: a snapshot at start (the state the process found) and then on change. + if (!this.testMode) { + const backup = (reason: string) => + void runConfigBackup(reason).catch((err) => + console.warn('[config-backup] snapshot failed:', err instanceof Error ? err.message : err) + ); + backup('startup'); + this.cleanup.setInterval(() => backup('auto'), CONFIG_BACKUP_INTERVAL_MS, { + description: 'Config backup', + }); + } + // Main Codex subscription limits come from the signed-in local CLI. Keep // this read-only and host-scoped; multi-user SSE routing makes it admin-only. if (!this.testMode) { diff --git a/test/config-backup-origin.test.ts b/test/config-backup-origin.test.ts new file mode 100644 index 000000000..4db940e51 --- /dev/null +++ b/test/config-backup-origin.test.ts @@ -0,0 +1,58 @@ +/** + * @fileoverview The snapshot origin must survive a hostname change (a Docker container gets a new + * hostname on every recreate) and still differ between data dirs. + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const host = vi.hoisted(() => ({ name: 'container-a' })); +vi.mock('node:os', async (importOriginal) => ({ + ...(await importOriginal()), + hostname: () => host.name, +})); + +import { + configBackupOrigin, + createConfigBackup, + listConfigBackups, + pruneConfigBackups, + restoreConfigBackup, +} from '../src/config-backup.js'; + +let root: string; +let data: string; +let backups: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-origin-')); + data = join(root, 'data'); + backups = join(root, 'backups'); + mkdirSync(data, { recursive: true }); + writeFileSync(join(data, 'settings.json'), '{"a":1}'); + host.name = 'container-a'; +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +describe('configBackupOrigin', () => { + it('is made once, kept in the data dir, and differs between data dirs', () => { + const a = configBackupOrigin(data); + expect(configBackupOrigin(data)).toBe(a); + expect(readFileSync(join(data, 'config-backup-id'), 'utf8').trim()).toBe(a); + const other = join(root, 'other'); + expect(configBackupOrigin(other)).not.toBe(a); + }); + + it('keeps snapshots listed, restorable and prunable after the hostname changes', () => { + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: 1_000_000 }) as { + id: string; + }; + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'manual', force: true, now: 2_000_000 }); + host.name = 'container-b'; // the container was recreated + const origin = configBackupOrigin(data); + expect(listConfigBackups(backups, origin)).toHaveLength(2); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id })).not.toThrow(); + const removed = pruneConfigBackups(backups, { keepCount: 1, keepDays: 0, origin }); + expect(removed).toHaveLength(2); // 3 now: the restore took a pre-restore copy + }); +}); diff --git a/test/config-backup-restore-ui.test.ts b/test/config-backup-restore-ui.test.ts new file mode 100644 index 000000000..840ee567a --- /dev/null +++ b/test/config-backup-restore-ui.test.ts @@ -0,0 +1,154 @@ +/** + * @fileoverview Restoring settings.json from inside App Settings must refill the form, or Save + * would PUT the pre-restore values back over the restored file. Evaluates the real settings-ui.js + * against a stub app. + */ +import { describe, expect, it } from 'vitest'; +import { readFileSync } from 'node:fs'; +import vm from 'node:vm'; + +const SRC = readFileSync(new URL('../src/web/public/settings-ui.js', import.meta.url), 'utf-8'); + +function makeApp(restored: string[], confirmed = true) { + const calls: string[] = []; + const confirms: string[] = []; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: {}, + confirm: (m: string) => (confirms.push(m), confirmed), + escapeHtml: (s: string) => s, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + Object.assign(app, { + _configBackups: [ + { + id: 'cfg-1', + files: [{ name: 'settings.json' }, { name: 'users.json' }, { name: '.env' }, { name: 'clis.json' }], + }, + ], + _apiPost: async () => ({ + ok: true, + json: async () => ({ success: true, data: { restored, restartRequired: [] } }), + }), + loadAppSettingsFromServer: async () => void calls.push('reload-settings'), + openAppSettings: () => void calls.push('refill-form'), + loadConfigBackups: () => void calls.push('reload-list'), + showToast: () => {}, + }); + return { app, calls, confirms }; +} + +describe('restoreConfigBackup (Settings UI)', () => { + it('re-reads settings into the form when settings.json was restored', async () => { + const { app, calls } = makeApp(['settings.json', 'clis.json']); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual(['reload-settings', 'refill-form', 'reload-list']); + }); + + it('leaves the form alone when settings.json was not part of the restore', async () => { + const { app, calls } = makeApp(['clis.json']); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual(['reload-list']); + }); + + it('names the files being replaced and calls out the credential ones', async () => { + const { app, confirms } = makeApp(['settings.json'], false); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(confirms[0]).toContain('settings.json, users.json, .env, clis.json'); + expect(confirms[0]).toContain('credentials'); + expect(confirms[0]).toMatch(/users\.json, \.env\)/); + }); +}); + +describe('loadConfigBackups (retention fields)', () => { + function load(saved: Record) { + const inputs: Record = { + appSettingsConfigBackupKeepCount: { value: '20', placeholder: '' }, + appSettingsConfigBackupKeepDays: { value: '30', placeholder: '' }, + appSettingsConfigBackupDir: { value: '', placeholder: '' }, + }; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: { getElementById: (id: string) => inputs[id] ?? null }, + escapeHtml: (s: string) => s, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + Object.assign(app, { + $: () => ({ textContent: '', innerHTML: '' }), + loadAppSettingsFromStorage: () => saved, + _api: async () => ({ + ok: true, + json: async () => ({ + success: true, + data: { settings: { keepCount: 100, keepDays: 0 }, dir: '/d', defaultDir: '/d', backups: [] }, + }), + }), + }); + return { app, inputs }; + } + + it('shows the retention the passes use when settings.json has none of the keys (a reset file)', async () => { + const { app, inputs } = load({}); + await (app.loadConfigBackups as () => Promise)(); + expect(inputs.appSettingsConfigBackupKeepCount.value).toBe(100); + expect(inputs.appSettingsConfigBackupKeepDays.value).toBe(0); + }); + + it('leaves the form alone when the saved settings state the retention', async () => { + const { app, inputs } = load({ configBackupKeepCount: 7, configBackupKeepDays: 9 }); + await (app.loadConfigBackups as () => Promise)(); + expect(inputs.appSettingsConfigBackupKeepCount.value).toBe('20'); // the open-settings code owns it + }); +}); + +describe('Save and the displayed retention', () => { + function payload( + prev: Record, + touched: { count?: boolean; days?: boolean }, + count = '100', + days = '0' + ) { + const inputs: Record }> = { + appSettingsConfigBackupKeepCount: { value: count, dataset: touched.count ? { touched: '1' } : {} }, + appSettingsConfigBackupKeepDays: { value: days, dataset: touched.days ? { touched: '1' } : {} }, + }; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: { getElementById: (id: string) => inputs[id] }, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + return (app._configBackupRetentionPayload as (p: Record) => Record).call( + app, + prev + ); + } + + it('leaves out a retention value that was only displayed (a reset settings.json)', () => { + expect(payload({}, {})).toEqual({}); + }); + + it('sends a value that was saved before, or edited here, and only that one', () => { + expect(payload({ configBackupKeepCount: 100 }, {})).toEqual({ configBackupKeepCount: 100 }); + expect(payload({}, { days: true })).toEqual({ configBackupKeepDays: 0 }); + expect(payload({}, { count: true, days: true }, '5', '7')).toEqual({ + configBackupKeepCount: 5, + configBackupKeepDays: 7, + }); + }); + + it('clamps like the form always did', () => { + expect(payload({}, { count: true, days: true }, '1', '99999')).toEqual({ + configBackupKeepCount: 2, + configBackupKeepDays: 3650, + }); + }); +}); diff --git a/test/config-backup.test.ts b/test/config-backup.test.ts new file mode 100644 index 000000000..2cbaa58eb --- /dev/null +++ b/test/config-backup.test.ts @@ -0,0 +1,645 @@ +/** + * @fileoverview Config backups (src/config-backup.ts): what is tracked, when a snapshot is made, + * how retention prunes, and that restore verifies before it writes. Pure fs over temp dirs; + * port: N/A. + */ +import { + chmodSync, + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + symlinkSync, + statSync, + writeFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { + CONFIG_BACKUP_FILES, + ConfigBackupCorruptError, + ConfigBackupNotFoundError, + createConfigBackup, + defaultConfigBackupDir, + diffConfigBackups, + listConfigBackups, + normalizeBackupDir, + pruneConfigBackups, + readConfigBackupSettings, + resolveConfigBackupDir, + restoreConfigBackup, + configBackupOrigin, + latestBackupRetention, + sweepConfigBackupTemps, +} from '../src/config-backup.js'; + +let root: string; +let data: string; +let backups: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-backup-')); + data = join(root, 'data'); + backups = join(root, 'backups'); + mkdirSync(data); +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +const put = (name: string, text: string) => writeFileSync(join(data, name), text); +const DAY = 86_400_000; +const T0 = Date.UTC(2026, 9, 1, 12, 0, 0); + +describe('settings', () => { + it('defaults to ON, 20 copies, 30 days, default folder', () => { + expect(readConfigBackupSettings({})).toEqual({ enabled: true, dir: null, keepCount: 20, keepDays: 30 }); + expect(readConfigBackupSettings(null).enabled).toBe(true); + }); + + it('only an explicit false turns it off, and numbers are clamped to their ranges', () => { + expect(readConfigBackupSettings({ configBackupEnabled: false }).enabled).toBe(false); + const s = readConfigBackupSettings({ configBackupKeepCount: 9999, configBackupKeepDays: -3 }); + expect(s.keepCount).toBe(500); + expect(s.keepDays).toBe(0); + expect(readConfigBackupSettings({ configBackupKeepCount: 0 }).keepCount).toBe(2); + expect(readConfigBackupSettings({ configBackupKeepCount: 'abc' }).keepCount).toBe(20); + expect(readConfigBackupSettings({ configBackupKeepCount: 7.9 }).keepCount).toBe(7); + }); + + it('expands ~ and refuses anything that is not an absolute path', () => { + expect(normalizeBackupDir('~/cfg', '/home/u')).toBe('/home/u/cfg'); + expect(normalizeBackupDir('/mnt/backups/codeman', '/home/u')).toBe('/mnt/backups/codeman'); + for (const bad of ['', ' ', 'relative/dir', './here', undefined, 42, '/x\0y']) { + expect(normalizeBackupDir(bad as never, '/home/u'), String(bad)).toBeNull(); + } + }); + + it('uses /backups/config by default and never the data dir itself', () => { + expect(resolveConfigBackupDir(readConfigBackupSettings({}), '/d')).toBe(defaultConfigBackupDir('/d')); + expect(resolveConfigBackupDir(readConfigBackupSettings({ configBackupDir: '/elsewhere' }), '/d')).toBe( + '/elsewhere' + ); + expect(resolveConfigBackupDir(readConfigBackupSettings({ configBackupDir: '/d' }), '/d')).toBe( + defaultConfigBackupDir('/d') + ); + }); +}); + +describe('createConfigBackup', () => { + it('snapshots only the tracked files that exist, with a manifest', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + put('state.json', '{"runtime":true}'); // not config: must not be copied + put('hook-secret', 'x'); // regenerated: must not be copied + const r = createConfigBackup({ + dataDir: data, + backupDir: backups, + reason: 'startup', + appVersion: '1.2.3', + now: T0, + }); + expect(r.status).toBe('created'); + const [b] = listConfigBackups(backups); + expect(b.files.map((f) => f.name).sort()).toEqual(['clis.json', 'settings.json']); + expect(b.reason).toBe('startup'); + expect(b.appVersion).toBe('1.2.3'); + expect(b.createdAt).toBe(T0); + expect(existsSync(join(backups, b.id, 'state.json'))).toBe(false); + expect(readFileSync(join(backups, b.id, 'settings.json'), 'utf8')).toBe('{"a":1}'); + }); + + it('tracks a fixed allowlist', () => { + expect([...CONFIG_BACKUP_FILES]).toContain('settings.json'); + expect([...CONFIG_BACKUP_FILES]).not.toContain('state.json'); + expect([...CONFIG_BACKUP_FILES]).not.toContain('hook-secret'); + }); + + it('makes no second snapshot while nothing changed, and one when something does', () => { + put('settings.json', '{"a":1}'); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + const again = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(again).toEqual({ status: 'unchanged', id: (first as { id: string }).id }); + expect(listConfigBackups(backups)).toHaveLength(1); + + put('settings.json', '{"a":2}'); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'created' + ); + expect(listConfigBackups(backups)).toHaveLength(2); + }); + + it('does not count a self-refreshed discovery timestamp as a change', () => { + const hosts = (stamp: string, model = 'm1') => + JSON.stringify({ hosts: [{ id: 'h', models: [{ id: model }], lastDiscoveredAt: stamp }] }); + put('custom-model-hosts.json', hosts('2026-10-12T08:44:22.026Z')); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + + // The 5-minute discovery poll rewrites only the timestamp: still the same state. + put('custom-model-hosts.json', hosts('2026-10-12T08:49:22.025Z')); + const again = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(again).toEqual({ status: 'unchanged', id: (first as { id: string }).id }); + expect(listConfigBackups(backups)).toHaveLength(1); + + // A real edit next to a refreshed timestamp is still a change, stored exactly as written. + const edited = hosts('2026-10-12T08:54:22.025Z', 'm2'); + put('custom-model-hosts.json', edited); + const third = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }); + expect(third.status).toBe('created'); + expect(readFileSync(join(backups, (third as { id: string }).id, 'custom-model-hosts.json'), 'utf8')).toBe(edited); + }); + + it.each([ + [ + 'webviews.json', + (n: number) => + JSON.stringify([ + { + id: 'w1', + name: 'Grafana', + url: 'http://localhost:3001', + embedMode: 'proxy', + trusted: false, + createdAt: 1, + lastOpenedAt: n, + }, + ]), + 'name', + ], + [ + 'docker-cases.json', + (n: number) => JSON.stringify([{ name: 'app', hostId: 'h', lastClaudeSessionId: `c${n}` }]), + 'hostId', + ], + ['users.json', (n: number) => JSON.stringify({ users: [{ name: 'a', role: 'admin', lastLoginAt: n }] }), 'role'], + ['settings.json', (n: number) => JSON.stringify({ displayName: 'x', lastUsedCase: `case-${n}` }), 'displayName'], + [ + 'intents.json', + (n: number) => JSON.stringify({ profiles: [{ key: 'k', goals: 'g', updatedAt: n, recentPrompts: [`p${n}`] }] }), + 'goals', + ], + ])('does not count activity in %s as a change, but does count an edit next to it', (name, make, editKey) => { + put(name, make(1)); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put(name, make(2)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 })).toEqual({ + status: 'unchanged', + id: (first as { id: string }).id, + }); + put(name, make(3).replace(new RegExp(`"${editKey}":"[^"]*"`), `"${editKey}":"edited"`)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'created' + ); + }); + + it('compares a tracked file that is not valid JSON by its bytes', () => { + put('custom-model-hosts.json', 'not json lastDiscoveredAt 1'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put('custom-model-hosts.json', 'not json lastDiscoveredAt 2'); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }).status).toBe( + 'created' + ); + }); + + it('force makes a snapshot even when unchanged, and two in one second get distinct ids', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'a', now: T0 }); + const forced = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'manual', force: true, now: T0 }); + expect(forced.status).toBe('created'); + const ids = listConfigBackups(backups).map((b) => b.id); + expect(new Set(ids).size).toBe(2); + }); + + it('reports empty, creating nothing, when no tracked file exists', () => { + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x' })).toEqual({ status: 'empty' }); + expect(existsSync(backups)).toBe(false); + }); + + it('keeps snapshots private: 0700 folder and 0600 files', () => { + put('settings.json', '{}'); + put('.env', 'CODEMAN_PASSWORD=secret'); + const r = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x', now: T0 }); + const id = (r as { id: string }).id; + expect(statSync(join(backups, id)).mode & 0o777).toBe(0o700); + expect(statSync(join(backups, id, '.env')).mode & 0o777).toBe(0o600); + expect(statSync(join(backups, id, 'manifest.json')).mode & 0o777).toBe(0o600); + }); + + it('leaves no temp folder behind', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x', now: T0 }); + const stray = readdirSync(backups).filter((n) => n.startsWith('.tmp-')); + expect(stray).toEqual([]); + }); +}); + +describe('retention', () => { + const makeN = (n: number) => { + for (let i = 0; i < n; i++) { + put('settings.json', JSON.stringify({ i })); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + i * DAY }); + } + }; + + it('keeps the newest keepCount', () => { + makeN(6); + const removed = pruneConfigBackups(backups, { keepCount: 3, keepDays: 0, now: T0 + 6 * DAY }); + expect(removed).toHaveLength(3); + const left = listConfigBackups(backups); + expect(left).toHaveLength(3); + expect(left.map((b) => b.createdAt)).toEqual([T0 + 5 * DAY, T0 + 4 * DAY, T0 + 3 * DAY]); + }); + + it('drops anything older than keepDays, but never the newest', () => { + makeN(4); // T0 .. T0+3d + // At T0+100d everything is older than 30 days: the newest still survives. + const removed = pruneConfigBackups(backups, { keepCount: 50, keepDays: 30, now: T0 + 100 * DAY }); + expect(removed).toHaveLength(3); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 3 * DAY]); + }); + + it('applies both limits together and keepDays 0 means no age limit', () => { + makeN(5); + expect(pruneConfigBackups(backups, { keepCount: 50, keepDays: 0, now: T0 + 999 * DAY })).toEqual([]); + const removed = pruneConfigBackups(backups, { keepCount: 4, keepDays: 3, now: T0 + 5 * DAY }); + // Newest first: T0+4d, +3d, +2d, +1d, T0. A snapshot ages from its successor: T0+1d was live + // until T0+2d, which sits exactly on the cutoff (kept); only T0 is over the count. + expect(removed.length).toBe(1); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([ + T0 + 4 * DAY, + T0 + 3 * DAY, + T0 + 2 * DAY, + T0 + DAY, + ]); + }); + + it('keeps the last good copy when the first change comes after a quiet month', () => { + put('settings.json', '{"theme":"good"}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'startup', now: T0 }); + put('settings.json', '{}'); // the bad write, 40 days later + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 40 * DAY }); + pruneConfigBackups(backups, { keepCount: 20, keepDays: 30, now: T0 + 40 * DAY }); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 40 * DAY, T0]); + }); + + it('never deletes a folder it did not make', () => { + makeN(3); + mkdirSync(join(backups, 'photos')); + writeFileSync(join(backups, 'photos', 'a.jpg'), 'x'); + mkdirSync(join(backups, 'cfg-20200101-000000')); // right name, no manifest: not ours + writeFileSync(join(backups, 'notes.txt'), 'keep'); + pruneConfigBackups(backups, { keepCount: 1, keepDays: 1, now: T0 + 500 * DAY }); + expect(existsSync(join(backups, 'photos', 'a.jpg'))).toBe(true); + expect(existsSync(join(backups, 'cfg-20200101-000000'))).toBe(true); + expect(existsSync(join(backups, 'notes.txt'))).toBe(true); + expect(listConfigBackups(backups)).toHaveLength(1); + }); + + it('does nothing for a folder that does not exist', () => { + expect(pruneConfigBackups(join(root, 'nope'), { keepCount: 1, keepDays: 1 })).toEqual([]); + }); +}); + +describe('restoreConfigBackup', () => { + it('puts the files back, atomically, after taking a pre-restore snapshot', () => { + put('settings.json', '{"good":true}'); + put('clis.json', '{"clis":{"grok":{"enabled":true}}}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + + put('settings.json', '{"polluted":true}'); + rmSync(join(data, 'clis.json')); + + const r = restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id, now: T0 + 1000 }); + expect(r.restored.sort()).toEqual(['clis.json', 'settings.json']); + expect(readFileSync(join(data, 'settings.json'), 'utf8')).toBe('{"good":true}'); + expect(readFileSync(join(data, 'clis.json'), 'utf8')).toBe('{"clis":{"grok":{"enabled":true}}}'); + expect(statSync(join(data, 'settings.json')).mode & 0o777).toBe(0o600); + + // The state it replaced is recoverable. + expect(r.safetySnapshot).toBeTruthy(); + const safety = listConfigBackups(backups).find((b) => b.id === r.safetySnapshot)!; + expect(safety.reason).toBe('pre-restore'); + expect(readFileSync(join(backups, safety.id, 'settings.json'), 'utf8')).toBe('{"polluted":true}'); + }); + + it('refuses a snapshot whose file no longer matches its checksum, changing nothing', () => { + put('settings.json', '{"good":true}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + chmodSync(join(backups, made.id, 'settings.json'), 0o600); + writeFileSync(join(backups, made.id, 'settings.json'), '{"tampered":true}'); + put('settings.json', '{"live":true}'); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id })).toThrow( + ConfigBackupCorruptError + ); + expect(readFileSync(join(data, 'settings.json'), 'utf8')).toBe('{"live":true}'); + expect(listConfigBackups(backups)).toHaveLength(1); // no pre-restore snapshot for a refused restore + }); + + it('rejects unknown ids and anything that is not a snapshot name (no path traversal)', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + for (const id of ['cfg-19990101-000000', '../data', '..', 'photos', 'cfg-20261001-120000/../../x']) { + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id }), id).toThrow( + ConfigBackupNotFoundError + ); + } + }); + + it('only ever writes allowlisted names, even if a manifest lists others', () => { + put('settings.json', '{"a":1}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + const mpath = join(backups, made.id, 'manifest.json'); + const m = JSON.parse(readFileSync(mpath, 'utf8')); + m.files.push({ name: '../escape.json', bytes: 1, sha256: 'x' }, { name: 'state.json', bytes: 1, sha256: 'x' }); + writeFileSync(mpath, JSON.stringify(m)); + restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id }); + expect(existsSync(join(root, 'escape.json'))).toBe(false); + expect(existsSync(join(data, 'state.json'))).toBe(false); + }); +}); + +describe('diffConfigBackups', () => { + const snap = (now: number) => { + const r = createConfigBackup({ dataDir: data, backupDir: backups, reason: 't', force: true, now }); + if (r.status !== 'created') throw new Error('expected a snapshot'); + return r.id; + }; + const diff = (from: string, to: string) => diffConfigBackups({ dataDir: data, backupDir: backups, from, to }); + const file = (d: ReturnType, name: string) => d.files.find((f) => f.name === name)!; + + it('lists key-level changes between two backups', () => { + put('settings.json', JSON.stringify({ displayName: 'a', theme: { mode: 'dark' }, tags: ['x'] })); + const a = snap(T0); + put('settings.json', JSON.stringify({ displayName: 'b', theme: { mode: 'dark' }, tags: ['x', 'y'], extra: 1 })); + const b = snap(T0 + 1000); + const f = file(diff(a, b), 'settings.json'); + expect(f.status).toBe('changed'); + expect(f.changes).toEqual([ + { path: 'displayName', kind: 'changed', before: '"a"', after: '"b"' }, + { path: 'extra', kind: 'added', after: '1' }, + { path: 'tags[1]', kind: 'added', after: '"y"' }, + ]); + }); + + it('compares a backup with the running config, and reports files that appeared or vanished', () => { + put('settings.json', '{"a":1}'); + put('webhook.json', '{"url":"u"}'); + const a = snap(T0); + put('settings.json', '{"a":2}'); + rmSync(join(data, 'webhook.json')); + put('intents.json', '{"i":true}'); + const d = diff(a, 'current'); + expect(d.changed).toBe(3); + expect(file(d, 'settings.json').changes).toEqual([{ path: 'a', kind: 'changed', before: '1', after: '2' }]); + expect(file(d, 'webhook.json').status).toBe('removed'); + expect(file(d, 'intents.json').status).toBe('added'); + expect(diff(a, a).changed).toBe(0); + }); + + it('never returns a webhook URL, captured prompts, or URL userinfo and query', () => { + put('webhook.json', JSON.stringify({ url: 'https://hooks.slack.com/services/T0/B0/oldsecret' })); + put('intents.json', JSON.stringify({ profiles: [{ key: 'k', prompts: ['my password is pw1'] }] })); + put('settings.json', JSON.stringify({ baseUrl: 'https://user:pw@host/p?token=t1#f' })); + const a = snap(T0); + put('webhook.json', JSON.stringify({ url: 'https://hooks.slack.com/services/T0/B0/newsecret' })); + put('intents.json', JSON.stringify({ profiles: [{ key: 'k', prompts: ['my password is pw2'] }] })); + put('settings.json', JSON.stringify({ baseUrl: 'https://user:pw@host/p?token=t2#f' })); + const d = diff(a, 'current'); + const json = JSON.stringify(d); + for (const s of ['secret', 'pw1', 'pw2', 'user:pw', 'token=', 'hooks.slack']) expect(json).not.toContain(s); + expect(file(d, 'webhook.json').changes.every((c) => c.redacted)).toBe(true); + expect(file(d, 'intents.json').changes.every((c) => c.redacted)).toBe(true); + }); + + it('never returns a credential value, only that it differs', () => { + put('.env', 'SECRET_ONE=hunter2\nOTHER=abc\n'); + put('users.json', JSON.stringify({ admin: { name: 'root' } })); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-old', url: 'http://x' }])); + const a = snap(T0); + put('.env', 'SECRET_ONE=hunter3\nOTHER=abd\n'); + put('users.json', JSON.stringify({ admin: { name: 'rooted' } })); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-new', url: 'http://y' }])); + const d = diff(a, 'current'); + const json = JSON.stringify(d); + for (const secret of ['hunter', 'abc', 'abd', 'root', 'sk-old', 'sk-new']) expect(json).not.toContain(secret); + expect(file(d, '.env').changes.every((c) => c.redacted)).toBe(true); + expect(file(d, 'custom-model-hosts.json').changes).toEqual([ + { path: '[0].apiKey', kind: 'changed', redacted: true }, + { path: '[0].url', kind: 'changed', before: '"http://x"', after: '"http://y"' }, + ]); + }); + + it('does not count a self-refreshed discovery timestamp as a difference', () => { + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 1 }])); + const a = snap(T0); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 2 }])); + const d = diff(a, 'current'); + expect(file(d, 'custom-model-hosts.json').status).toBe('unchanged'); + expect(d.changed).toBe(0); + }); + + it('reports a file that is not JSON as differing, without its content', () => { + put('settings.json', 'not json at all'); + const a = snap(T0); + put('settings.json', 'still not json'); + const f = file(diff(a, 'current'), 'settings.json'); + expect(f).toMatchObject({ status: 'changed', detail: 'bytes', changes: [] }); + }); + + it('caps the changes listed per file', () => { + put('settings.json', '{}'); + const a = snap(T0); + put('settings.json', JSON.stringify(Object.fromEntries(Array.from({ length: 250 }, (_, i) => [`k${i}`, i])))); + const f = file(diff(a, 'current'), 'settings.json'); + expect(f.changes).toHaveLength(200); + expect(f.truncated).toBe(true); + }); + + it('refuses unknown ids, path traversal and a tampered snapshot', () => { + put('settings.json', '{"a":1}'); + const a = snap(T0); + expect(() => diff('cfg-20200101-000000', 'current')).toThrow(ConfigBackupNotFoundError); + expect(() => diff('../etc', 'current')).toThrow(ConfigBackupNotFoundError); + writeFileSync(join(backups, a, 'settings.json'), '{"a":99}'); + expect(() => diff(a, 'current')).toThrow(ConfigBackupCorruptError); + }); +}); + +describe('createConfigBackup: a snapshot dated ahead of the clock', () => { + it('is not written again on every pass once the clock is corrected', () => { + put('settings.json', '{"v":0}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put('settings.json', '{"v":1}'); + const ahead = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2 * 3_600_000 }); + expect(ahead.status).toBe('created'); + put('settings.json', '{"v":2}'); // the clock is back at T0 + 1 minute + const corrected = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 60_000 }); + expect(corrected.status).toBe('created'); + const [newest, second] = listConfigBackups(backups); + expect(newest.id).toBe((corrected as { id: string }).id); // dated no earlier than the one before it + expect(newest.createdAt).toBeGreaterThan(second.createdAt); + for (let i = 0; i < 24; i++) { + const r = createConfigBackup({ + dataDir: data, + backupDir: backups, + reason: 'auto', + now: T0 + 120_000 + i * 300_000, + }); + expect(r.status, `pass ${i}`).toBe('unchanged'); + } + expect(listConfigBackups(backups)).toHaveLength(3); + }); +}); + +describe('instances sharing one backup folder', () => { + it('keeps to its own snapshots: no churn, no prune, no restore of the other', () => { + const other = join(root, 'other-data'); + mkdirSync(other, { recursive: true }); + put('settings.json', '{"a":1}'); + writeFileSync(join(other, 'settings.json'), '{"b":2}'); + const mine = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + const theirs = createConfigBackup({ dataDir: other, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(theirs.status).toBe('created'); + // Each side's change check compares with its own newest snapshot, so a second tick is a no-op. + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'unchanged' + ); + expect(createConfigBackup({ dataDir: other, backupDir: backups, reason: 'auto', now: T0 + 3000 }).status).toBe( + 'unchanged' + ); + expect(listConfigBackups(backups, configBackupOrigin(data))).toHaveLength(1); + expect( + pruneConfigBackups(backups, { keepCount: 1, keepDays: 1, now: T0 + 999 * DAY, origin: configBackupOrigin(data) }) + ).toEqual([]); + expect(listConfigBackups(backups)).toHaveLength(2); + const theirId = (theirs as { id: string }).id; + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: theirId })).toThrow( + ConfigBackupNotFoundError + ); + expect((mine as { id: string }).id).not.toBe(theirId); + }); + + it('a restore reports tracked files it left alone, and sweeps only old temp folders', () => { + put('settings.json', '{"a":1}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + put('webhook.json', '{"url":"u"}'); + expect(restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id }).untouched).toEqual(['webhook.json']); + const stale = join(backups, '.tmp-cfg-20200101-000000-99999'); + mkdirSync(stale); + sweepConfigBackupTemps(backups, Date.now() + 2 * 3_600_000); + expect(existsSync(stale)).toBe(false); + }); +}); + +describe('restore trusts only what this module wrote', () => { + const made = () => + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + + it('refuses a symlinked snapshot file and a snapshot file with group or world bits', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + const { id } = made(); + const file = join(backups, id, 'clis.json'); + chmodSync(file, 0o644); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(ConfigBackupCorruptError); + + chmodSync(file, 0o600); + const real = join(root, 'elsewhere.json'); + writeFileSync(real, '{"clis":{}}'); + rmSync(file); + symlinkSync(real, file); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(/symlink/); + // nothing was written and no pre-restore copy was taken for a refused restore + expect(listConfigBackups(backups)).toHaveLength(1); + }); + + it('skips the group/world bit test on a filesystem that does not keep modes, but never the symlink test', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + const { id } = made(); + const file = join(backups, id, 'clis.json'); + chmodSync(file, 0o644); // what vfat, CIFS and WSL /mnt/d report for every file + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id, keepsModes: false })).not.toThrow(); + + const real = join(root, 'elsewhere.json'); + writeFileSync(real, '{"clis":{}}'); + rmSync(file); + symlinkSync(real, file); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id, keepsModes: false })).toThrow(/symlink/); + }); + + it('the mode probe never writes through a planted symlink or trips over a planted directory', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + const target = join(root, 'victim.txt'); + writeFileSync(target, 'precious', { mode: 0o644 }); + symlinkSync(target, join(backups, `.mode-probe-${process.pid}`)); + mkdirSync(join(backups, '.mode-probe-0')); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).not.toThrow(); + expect(() => diffConfigBackups({ dataDir: data, backupDir: backups, from: id, to: 'current' })).not.toThrow(); + expect(readFileSync(target, 'utf8')).toBe('precious'); + expect(statSync(target).mode & 0o777).toBe(0o644); + }); + + it('compare applies the same check to a snapshot file', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + chmodSync(join(backups, id, 'settings.json'), 0o644); + expect(() => diffConfigBackups({ dataDir: data, backupDir: backups, from: id, to: 'current' })).toThrow( + ConfigBackupCorruptError + ); + }); + + it('refuses a snapshot folder that others can write to', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + chmodSync(join(backups, id), 0o777); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(ConfigBackupCorruptError); + chmodSync(join(backups, id), 0o700); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).not.toThrow(); + }); + + it('keeps the backup retention when the live settings.json was reset and has none of the four keys', () => { + put('settings.json', JSON.stringify({ displayName: 'old', configBackupKeepCount: 100, configBackupKeepDays: 0 })); + const { id } = made(); + put('settings.json', '{}'); + restoreConfigBackup({ dataDir: data, backupDir: backups, id }); + expect(JSON.parse(readFileSync(join(data, 'settings.json'), 'utf8'))).toEqual({ + displayName: 'old', + configBackupKeepCount: 100, + configBackupKeepDays: 0, + }); + }); + + it('latestBackupRetention skips a newer snapshot that recorded none, and a pre-restore copy records it', () => { + put('settings.json', '{"a":1}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0, keepCount: 100, keepDays: 0 }); + put('settings.json', '{"a":2}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); // an older build + const origin = configBackupOrigin(data); + expect(latestBackupRetention(backups, origin)).toEqual({ keepCount: 100, keepDays: 0 }); + const id = listConfigBackups(backups, origin).at(-1)!.id; + const r = restoreConfigBackup({ dataDir: data, backupDir: backups, id, now: T0 + 2000 }); + expect(r.safetySnapshot).toBeTruthy(); + const pre = listConfigBackups(backups, origin).find((b) => b.id === r.safetySnapshot)!; + expect(JSON.parse(readFileSync(join(backups, pre.id, 'manifest.json'), 'utf8'))).toMatchObject({ + keepCount: 100, + keepDays: 0, + }); + }); + + it('keeps the current folder and retention when settings.json is restored', () => { + put('settings.json', JSON.stringify({ displayName: 'old', configBackupDir: '/old', configBackupKeepCount: 5 })); + const { id } = made(); + put('settings.json', JSON.stringify({ displayName: 'new', configBackupDir: '/new', configBackupKeepDays: 7 })); + restoreConfigBackup({ dataDir: data, backupDir: backups, id }); + const after = JSON.parse(readFileSync(join(data, 'settings.json'), 'utf8')); + // the folder and days follow the live file; the live file has no keep count (reset), so the backup's 5 stays + expect(after).toEqual({ + displayName: 'old', + configBackupDir: '/new', + configBackupKeepCount: 5, + configBackupKeepDays: 7, + }); + }); +}); diff --git a/test/routes/config-backup-routes.test.ts b/test/routes/config-backup-routes.test.ts new file mode 100644 index 000000000..40dce415f --- /dev/null +++ b/test/routes/config-backup-routes.test.ts @@ -0,0 +1,300 @@ +/** + * @fileoverview /api/config-backups: list, back up now, restore. Settings are read fresh from + * settings.json, so a folder or retention change applies on the next request. + * + * ⚠️ test/setup.ts gives the whole FILE one temp HOME (and so one data dir). Port: N/A (inject()). + */ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { createRouteTestHarness } from './_route-test-utils.js'; +import { registerConfigBackupRoutes, runConfigBackup } from '../../src/web/routes/config-backup-routes.js'; +import { SETTINGS_PATH } from '../../src/web/route-helpers.js'; +import { getDataDir } from '../../src/config/instance.js'; +import { createConfigBackup, defaultConfigBackupDir, listConfigBackups } from '../../src/config-backup.js'; +import { intentStore } from '../../src/intent-store.js'; + +let custom: string; +const settings = (extra: Record = {}) => { + mkdirSync(dirname(SETTINGS_PATH), { recursive: true }); + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'before', ...extra })); +}; + +beforeEach(() => { + custom = mkdtempSync(join(tmpdir(), 'cfg-backup-route-')); + rmSync(defaultConfigBackupDir(getDataDir()), { recursive: true, force: true }); + settings(); +}); +afterEach(() => { + delete process.env.CODEMAN_MULTIUSER; + rmSync(custom, { recursive: true, force: true }); + rmSync(SETTINGS_PATH, { force: true }); + rmSync(defaultConfigBackupDir(getDataDir()), { recursive: true, force: true }); +}); + +describe('/api/config-backups', () => { + it('lists the effective settings, the default folder and the tracked files', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'GET', url: '/api/config-backups' }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.settings).toEqual({ enabled: true, dir: null, keepCount: 20, keepDays: 30 }); + expect(d.dir).toBe(defaultConfigBackupDir(getDataDir())); + expect(d.tracked).toContain('settings.json'); + expect(d.backups).toEqual([]); + }); + + it('backs up now into the default folder and lists it', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const made = await app.inject({ method: 'POST', url: '/api/config-backups' }); + expect(made.statusCode).toBe(200); + expect(made.json().data.status).toBe('created'); + const list = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data; + expect(list.backups).toHaveLength(1); + expect(list.backups[0].reason).toBe('manual'); + expect(list.backups[0].files.map((f: { name: string }) => f.name)).toContain('settings.json'); + // Names and sizes only, never content. + expect(JSON.stringify(list)).not.toContain('before'); + }); + + it('follows a custom folder and retention from settings on the next request', async () => { + const dir = join(custom, 'snap'); + settings({ configBackupDir: dir, configBackupKeepCount: 2, configBackupKeepDays: 0 }); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + for (let i = 0; i < 4; i++) { + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: dir, configBackupKeepCount: 2, i })); + await app.inject({ method: 'POST', url: '/api/config-backups' }); + await new Promise((r) => setTimeout(r, 5)); + } + const d = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data; + expect(d.dir).toBe(dir); + expect(d.backups).toHaveLength(2); + expect(existsSync(defaultConfigBackupDir(getDataDir()))).toBe(false); + }); + + it('restores a backup, reports what needs a restart and keeps a pre-restore copy', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + writeFileSync(join(dirname(SETTINGS_PATH), 'clis.json'), '{"schemaVersion":1,"clis":{}}', { mode: 0o600 }); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'polluted' })); + writeFileSync(join(dirname(SETTINGS_PATH), 'clis.json'), '{"clis":{"evil":{}}}'); + + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.restored).toEqual(expect.arrayContaining(['settings.json', 'clis.json'])); + expect(d.restartRequired).not.toContain('clis.json'); // reloaded by the restore itself + expect(d.restartRequired).toContain('settings.json'); // its watcher toggles are not re-run by a restore + expect(d.safetySnapshot).toBeTruthy(); + expect(JSON.parse(readFileSync(SETTINGS_PATH, 'utf8')).displayName).toBe('before'); + }); + + it('a restored intents.json is not reverted by the next captured prompt', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const dir = mkdtempSync(join(tmpdir(), 'cfg-backup-intent-')); + try { + intentStore.recordPrompt(undefined, dir, 's1', 'first prompt about the build pipeline'); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + intentStore.recordPrompt(undefined, dir, 's1', 'second prompt about the build pipeline'); + await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + intentStore.recordPrompt(undefined, dir, 's1', 'third prompt about the build pipeline'); + expect(intentStore.getProfile(undefined, dir).recentPrompts).toHaveLength(2); + } finally { + intentStore.deleteProfile(undefined, dir); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('refuses a custom folder whose parent is missing instead of creating it', async () => { + settings({ configBackupDir: join(custom, 'no', 'such', 'dir') }); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'POST', url: '/api/config-backups' }); + expect(res.statusCode).toBe(503); + expect(existsSync(join(custom, 'no'))).toBe(false); + }); + + it('answers 404 for an unknown backup and 404 for a traversal attempt', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + for (const id of ['cfg-19990101-000000', '..%2F..%2Fetc', 'photos']) { + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + expect(res.statusCode, id).toBe(404); + } + }); + + it('is admin only in multi-user mode: a non-admin is refused on every verb', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'bob', role: 'user' }, + }); + for (const [method, url] of [ + ['GET', '/api/config-backups'], + ['POST', '/api/config-backups'], + ['POST', '/api/config-backups/cfg-20261001-120000/restore'], + ['GET', '/api/config-backups/diff?from=cfg-20261001-120000'], + ] as const) { + expect((await app.inject({ method, url })).statusCode, `${method} ${url}`).toBe(403); + } + }); + + it('multi-user: a restore is written to the admin audit log', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'root', role: 'admin' }, + }); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + const auditPath = join(getDataDir(), 'admin-audit.jsonl'); + const written = () => existsSync(auditPath) && readFileSync(auditPath, 'utf8').includes('config_backup_restore'); + for (let i = 0; i < 40 && !written(); i++) await new Promise((r) => setTimeout(r, 25)); + const last = JSON.parse(readFileSync(auditPath, 'utf8').trim().split('\n').pop()!); + expect(last).toMatchObject({ admin: 'root', action: 'config_backup_restore', target: id }); + expect(last.detail.restored).toContain('settings.json'); + rmSync(auditPath, { force: true }); + }); + + it('multi-user: an admin is allowed', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'root', role: 'admin' }, + }); + expect((await app.inject({ method: 'GET', url: '/api/config-backups' })).json().success).toBe(true); + }); +}); + +describe('GET /api/config-backups/diff', () => { + it('compares a backup with the running config, defaulting `to` to current', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const made = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data; + settings({ displayName: 'after' }); + const res = await app.inject({ method: 'GET', url: `/api/config-backups/diff?from=${made.id}` }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.to).toBe('current'); + expect(d.files.find((f: { name: string }) => f.name === 'settings.json').changes).toEqual([ + { path: 'displayName', kind: 'changed', before: '"before"', after: '"after"' }, + ]); + }); + + it('400 without `from`, 404 for an unknown backup', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + expect((await app.inject({ method: 'GET', url: '/api/config-backups/diff' })).statusCode).toBe(400); + const missing = await app.inject({ method: 'GET', url: '/api/config-backups/diff?from=cfg-20200101-000000' }); + expect(missing.statusCode).toBe(404); + }); +}); + +describe('retention survives an unreadable or reset settings.json', () => { + const DAY = 86_400_000; + const BASE = Date.UTC(2025, 0, 1); + const dir = () => defaultConfigBackupDir(getDataDir()); + /** 30 snapshots a day apart, made under a long-history retention (100 copies, no age limit). */ + function seed(withKeys = true) { + settings({ configBackupKeepCount: 100, configBackupKeepDays: 0 }); + for (let i = 0; i < 30; i++) { + writeFileSync( + SETTINGS_PATH, + JSON.stringify( + withKeys + ? { displayName: `v${i}`, configBackupKeepCount: 100, configBackupKeepDays: 0 } + : { displayName: `v${i}` } + ) + ); + createConfigBackup({ + dataDir: getDataDir(), + backupDir: dir(), + reason: 'auto', + now: BASE + i * DAY, + keepCount: 100, + keepDays: 0, + }); + } + expect(listConfigBackups(dir())).toHaveLength(30); + } + + it('a settings.json that does not parse still gets a snapshot but prunes nothing', async () => { + seed(); + writeFileSync(SETTINGS_PATH, '{ "displayName": "half-written", }'); + const out = await runConfigBackup('auto'); + expect(out?.result.status).toBe('created'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(dir())).toHaveLength(31); + }); + + it('a reset settings.json prunes with the retention the newest snapshot recorded, not the defaults', async () => { + seed(); + writeFileSync(SETTINGS_PATH, '{}'); + const out = await runConfigBackup('auto'); + expect(out?.result.status).toBe('created'); + // aged manifests (a year old) and keepDays 0: the defaults (20 copies, 30 days) would leave 20 + expect(listConfigBackups(dir())).toHaveLength(31); + }); + + it.each([true, false])( + 'a restore after a reset keeps the retention (the backup %s the four keys)', + async (withKeys) => { + seed(withKeys); + const oldest = listConfigBackups(dir()).at(-1)!.id; + writeFileSync(SETTINGS_PATH, '{}'); // the reset + expect((await runConfigBackup('auto'))?.removed).toEqual([]); + + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${oldest}/restore` }); + expect(res.statusCode).toBe(200); + // the restored file carries the retention, or the recorded one still applies via the pre-restore copy + const out = await runConfigBackup('auto'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(dir()).length).toBeGreaterThanOrEqual(32); + const status = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.settings; + expect(status).toMatchObject({ keepCount: 100, keepDays: 0 }); // what the form shows is what the pass uses + } + ); + + it("re-entering a custom folder after a reset finds that folder's recorded retention", async () => { + const folder = join(custom, 'kept'); + settings({ configBackupDir: folder, configBackupKeepCount: 100, configBackupKeepDays: 0 }); + for (let i = 0; i < 30; i++) { + createConfigBackup({ + dataDir: getDataDir(), + backupDir: folder, + reason: 'auto', + force: true, + now: BASE + i * DAY, + keepCount: 100, + keepDays: 0, + }); + } + writeFileSync(SETTINGS_PATH, '{}'); // the reset drops the folder too + await runConfigBackup('auto'); // goes to the default folder, which holds none of those manifests + // the user enters the folder again and saves; the form no longer sends retention it only displayed + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: folder, configBackupEnabled: true })); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const reported = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.settings; + expect(reported).toMatchObject({ keepCount: 100, keepDays: 0 }); + const out = await runConfigBackup('auto'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(folder).length).toBeGreaterThanOrEqual(30); + }); + + it('only the retention key the file lacks falls back to the recorded one', async () => { + seed(); + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepDays: 3650 })); + await runConfigBackup('auto'); + expect(listConfigBackups(dir())).toHaveLength(31); // keepCount 100 from the manifests, not the default 20 + }); + + it('a settings.json that sets the retention still wins', async () => { + seed(); + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepCount: 5, configBackupKeepDays: 0 })); + await runConfigBackup('auto'); + expect(listConfigBackups(dir())).toHaveLength(5); + }); + + it('listing never creates config-backup-id', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const idFile = join(getDataDir(), 'config-backup-id'); + rmSync(idFile, { force: true }); + expect((await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.backups).toEqual([]); + expect(existsSync(idFile)).toBe(false); + }); +}); diff --git a/test/routes/config-backup-unreachable.test.ts b/test/routes/config-backup-unreachable.test.ts new file mode 100644 index 000000000..d1d997002 --- /dev/null +++ b/test/routes/config-backup-unreachable.test.ts @@ -0,0 +1,76 @@ +/** + * @fileoverview A custom backup folder that stops answering (a network mount) must never be listed or + * read before the bounded probe says it is there: GET, diff and restore answer 503 without touching it. + * With only `configBackupDir` in settings.json the recorded-retention fallback would otherwise + * `readdirSync` the folder first (found in review of #661). + * + * ⚠️ test/setup.ts gives the whole FILE one temp HOME. Port: N/A (inject()). + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; + +const dead = vi.hoisted(() => ({ dir: '', touched: [] as string[] })); + +vi.mock('node:fs', async (importOriginal) => { + const actual = await importOriginal(); + const record = unknown>(name: string, real: F): F => + ((path: unknown, ...rest: unknown[]) => { + if (dead.dir && String(path).startsWith(dead.dir)) dead.touched.push(`${name} ${String(path)}`); + return (real as unknown as (...a: unknown[]) => unknown)(path, ...rest); + }) as unknown as F; + const wrapped = { + ...actual, + readdirSync: record('readdirSync', actual.readdirSync), + readFileSync: record('readFileSync', actual.readFileSync), + statSync: record('statSync', actual.statSync), + lstatSync: record('lstatSync', actual.lstatSync), + existsSync: record('existsSync', actual.existsSync), + }; + return { ...wrapped, default: wrapped }; +}); + +vi.mock('../../src/utils/bounded-path-probe.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + probePathKind: async (path: string, options?: { pastCap?: boolean }) => + dead.dir && path.startsWith(dead.dir) ? 'unknown' : actual.probePathKind(path, options), + }; +}); + +import { createRouteTestHarness } from './_route-test-utils.js'; +import { registerConfigBackupRoutes } from '../../src/web/routes/config-backup-routes.js'; +import { SETTINGS_PATH } from '../../src/web/route-helpers.js'; +import { getDataDir } from '../../src/config/instance.js'; +import { configBackupOrigin } from '../../src/config-backup.js'; + +let root: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-backup-dead-')); + dead.dir = join(root, 'mount'); + dead.touched.length = 0; + configBackupOrigin(getDataDir()); // an instance that already has an id: the fallback has something to look for + mkdirSync(dirname(SETTINGS_PATH), { recursive: true }); + // Only the folder: no retention keys, the shape that makes the recorded-retention fallback want to read it. + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: dead.dir })); +}); +afterEach(() => { + dead.dir = ''; + rmSync(root, { recursive: true, force: true }); + rmSync(SETTINGS_PATH, { force: true }); +}); + +describe('a custom backup folder that does not answer', () => { + it.each([ + ['GET', '/api/config-backups'], + ['GET', '/api/config-backups/diff?from=cfg-20261001-120000'], + ['POST', '/api/config-backups/cfg-20261001-120000/restore'], + ] as const)('%s %s answers 503 without reading the folder', async (method, url) => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method, url }); + expect(res.statusCode).toBe(503); + expect(dead.touched).toEqual([]); + }); +});