Skip to content

feat(apps): local app database mirroring .asc/ (DMN-160..163) - #79

Merged
statebyte merged 1 commit into
mainfrom
feat/daemon-local-db
Oct 18, 2026
Merged

statebyte merged 1 commit into
mainfrom
feat/daemon-local-db

Conversation

@statebyte

@statebyte statebyte commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

📋 Summary

Adds the daemon's local database: an SQLite index of installed apps (<data_dir>/index.db) that mirrors everything in every app's .asc/ and stays in sync with the files. asc ls, ListApps/GetStatus and the other app listings read from it instead of re-reading every meta.json and running git rev-parse per app, and the CLI checks the daemon with a light GET /v1/health instead of a full app listing.

🔗 Related task / issue

🧩 Type of change

  • ✨ Feature (feat)
  • 🐛 Bug fix (fix)
  • 📝 Documentation (docs)
  • ♻️ Refactor (refactor)
  • 🧪 Tests (test)
  • 🔧 Chore / tooling (chore)

🗂️ Affected component

App management, API (REST), CLI, Scheduler

📝 What changed

  • src/daemon/apps/index/ — AppIndex over rusqlite (bundled, WAL, busy_timeout, file created 0600 before SQLite writes to it):
    • schema v1: apps (meta split into columns + meta_json, commit, .git stamp, last observed state) and app_files (content of every top-level .asc/ file with an inode:size:mtime_ns:ctime_ns stamp); not mirrored: op.lock, *.tmp, symlinks, sub-directories; files over 1 MiB keep no content;
    • sync_app reads the files inside BEGIN IMMEDIATE, so concurrent writers (two threads, or the CLI and the daemon) converge on the file;
    • verify — stat-only comparison with the files; rebuild — full scan (same rules as the old listing, including layout::migrate);
    • no database / newer or broken schema / changed apps_dir → rebuild; an unusable file is moved aside as index.db.broken-<ts>;
    • process-wide registry: every AppStore of the same root shares one index; one database file serves one apps root per process; ASC_NO_INDEX=1 keeps the old scan.
  • Sync with the files: write-through from AppStore::save/remove, SettingValues::save and the compose override writer; inotify watcher (Linux, non-recursive watches on the apps root, app dirs, .asc/, repository/.git, 200 ms debounce, overflow → verify); verify every minute in the daemon and before reads in a process that does not maintain the index (the CLI without a daemon).
  • Reads: AppStore::list, name resolution in get_authorized (the match is then loaded from its file), rename, reconcile, prune protection, ownership maps, declared ports and the backup scheduler (AppStore::settings from the mirrored settings.json); commits from the index instead of git rev-parse.
  • Batched observation (AppManager::observe): one Engine container list for all docker/compose apps, one systemctl show -p ActiveState for all systemd units (falls back to is-active per unit), pid check for process apps.
  • API: GET /v1/health; root-only GET /v1/db, POST /v1/db/verify, POST /v1/db/rebuild. The CLI probes /v1/health and falls back to /v1/status on older daemons (404).
  • CLI: asc db status|verify|rebuild (EN + RU messages). Named db, not index: commands/index on the docs site is the command overview page.
  • Docs: docs/english|russian/app-management.md — new "🗃️ Local database" section.

✅ Checklist

  • The change starts from a task, and the relevant doc in docs/ is updated (docs first, then code — see AGENTS.md).
  • task check passes locally (clippy with -D warnings, cargo fmt, tests) — run in WSL: 731 unit tests + all integration tests.
  • New logic has unit tests (index, scan, schema, watcher, systemd batch parser) and integration tests (/v1/health, /v1/db, root-only refusal over the socket).
  • Commits follow Conventional Commits.
  • Does not break builds for x86_64 / ARM64 / ARMv7 — rusqlite bundled compiles SQLite with the target C toolchain cross already provides for ring; the cross builds themselves were not run locally (release CI does).
  • User-facing CLI strings go through the translation system (EN + RU keys); debug/log messages are English-only.
  • proto/ changes are backward compatible — no proto changes.

🔎 How to test

task test
sudo asc ls
sudo asc db status
echo '{"MODE":"dev"}' | sudo tee /asc/apps/<id>/.asc/settings.json
sudo asc db verify

Measured on 50 process apps with git checkouts (debug build, CLI without a daemon): asc ls ~15 ms with the index vs ~80 ms with ASC_NO_INDEX=1. In a container running asc serve: index.db, -wal, -shm are 0600, an app directory written by hand shows up in asc ls within a second (watcher), asc db verify picks up a hand-edited settings.json and a removed app.

📌 Notes for reviewers

  • The files stay the source of truth; the database is always rebuildable and deleting index.db is safe at any time.
  • Still open in DMN-162: Docker Engine events feeding observed_state, a shared Docker client per process, asc ls --live.
  • Pre-existing, not touched here: CLI requests over the socket fail for app names containing spaces (invalid uri character) — paths are not percent-encoded in src/daemon/client.rs.
  • No version bump: that happens at release time per AGENTS.md.

🤖 Generated with Claude Code

Add an SQLite index of installed apps, <data_dir>/index.db (rusqlite
bundled, WAL, 0600), so listings stop re-reading every app on every call.

- The index mirrors every app's .asc/: meta.json split into columns plus
  the content of every other top-level state file (settings.json,
  compose.override.yaml, ...) with an inode:size:mtime:ctime stamp, and
  the repository's checked-out commit. Locks, *.tmp files, symlinks and
  sub-directories are not mirrored; files over 1 MiB keep no content.
- Sync with the files: write-through from AppStore::save/remove,
  SettingValues::save and the compose override writer; an inotify watcher
  (Linux) on the apps root, app dirs, .asc/ and repository/.git; a
  stat-only verify every minute in the daemon and before reads in a
  process nothing keeps the index current for. Syncs read the files
  inside BEGIN IMMEDIATE, so concurrent writers converge on the file.
- Full rebuild when there is no database, the schema or apps_dir changed;
  a broken or newer database is moved aside as index.db.broken-<ts>.
- AppStore::list, name resolution, rename checks, reconcile, prune
  protection, ownership maps, declared ports and the backup scheduler
  read from the index; commits come from it instead of git rev-parse.
- AppManager::list observes runtime states in one batch: one Engine
  container list for all docker/compose apps, one systemctl show for all
  systemd units.
- GET /v1/health for the CLI's connection probe (falls back to
  /v1/status on older daemons), so no command pays for a full listing.
- asc db status|verify|rebuild and root-only /v1/db endpoints.
- ASC_NO_INDEX=1 keeps the old file scan.

On 50 apps `asc ls` (debug build, no daemon) drops from ~80 ms to ~15 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@statebyte
statebyte merged commit 2bf6cb8 into main Oct 18, 2026
3 checks passed
@statebyte
statebyte deleted the feat/daemon-local-db branch November 1, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant