Repository navigation
feat(apps): local app database mirroring .asc/ (DMN-160..163) - #79
Merged
Merged
Conversation
Add an SQLite index of installed apps, <data_dir>/index.db (rusqlite bundled, WAL, 0600), so listings stop re-reading every app on every call. - The index mirrors every app's .asc/: meta.json split into columns plus the content of every other top-level state file (settings.json, compose.override.yaml, ...) with an inode:size:mtime:ctime stamp, and the repository's checked-out commit. Locks, *.tmp files, symlinks and sub-directories are not mirrored; files over 1 MiB keep no content. - Sync with the files: write-through from AppStore::save/remove, SettingValues::save and the compose override writer; an inotify watcher (Linux) on the apps root, app dirs, .asc/ and repository/.git; a stat-only verify every minute in the daemon and before reads in a process nothing keeps the index current for. Syncs read the files inside BEGIN IMMEDIATE, so concurrent writers converge on the file. - Full rebuild when there is no database, the schema or apps_dir changed; a broken or newer database is moved aside as index.db.broken-<ts>. - AppStore::list, name resolution, rename checks, reconcile, prune protection, ownership maps, declared ports and the backup scheduler read from the index; commits come from it instead of git rev-parse. - AppManager::list observes runtime states in one batch: one Engine container list for all docker/compose apps, one systemctl show for all systemd units. - GET /v1/health for the CLI's connection probe (falls back to /v1/status on older daemons), so no command pays for a full listing. - asc db status|verify|rebuild and root-only /v1/db endpoints. - ASC_NO_INDEX=1 keeps the old file scan. On 50 apps `asc ls` (debug build, no daemon) drops from ~80 ms to ~15 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📋 Summary
Adds the daemon's local database: an SQLite index of installed apps (
<data_dir>/index.db) that mirrors everything in every app's.asc/and stays in sync with the files.asc ls,ListApps/GetStatusand the other app listings read from it instead of re-reading everymeta.jsonand runninggit rev-parseper app, and the CLI checks the daemon with a lightGET /v1/healthinstead of a full app listing.🔗 Related task / issue
🧩 Type of change
feat)fix)docs)refactor)test)chore)🗂️ Affected component
App management, API (REST), CLI, Scheduler
📝 What changed
src/daemon/apps/index/—AppIndexoverrusqlite(bundled, WAL,busy_timeout, file created0600before SQLite writes to it):apps(meta split into columns +meta_json, commit,.gitstamp, last observed state) andapp_files(content of every top-level.asc/file with aninode:size:mtime_ns:ctime_nsstamp); not mirrored:op.lock,*.tmp, symlinks, sub-directories; files over 1 MiB keep no content;sync_appreads the files insideBEGIN IMMEDIATE, so concurrent writers (two threads, or the CLI and the daemon) converge on the file;verify—stat-only comparison with the files;rebuild— full scan (same rules as the old listing, includinglayout::migrate);apps_dir→ rebuild; an unusable file is moved aside asindex.db.broken-<ts>;AppStoreof the same root shares one index; one database file serves one apps root per process;ASC_NO_INDEX=1keeps the old scan.AppStore::save/remove,SettingValues::saveand the compose override writer; inotify watcher (Linux, non-recursive watches on the apps root, app dirs,.asc/,repository/.git, 200 ms debounce, overflow → verify); verify every minute in the daemon and before reads in a process that does not maintain the index (the CLI without a daemon).AppStore::list, name resolution inget_authorized(the match is then loaded from its file),rename,reconcile, prune protection, ownership maps, declared ports and the backup scheduler (AppStore::settingsfrom the mirroredsettings.json); commits from the index instead ofgit rev-parse.AppManager::observe): one Engine container list for all docker/compose apps, onesystemctl show -p ActiveStatefor all systemd units (falls back tois-activeper unit), pid check for process apps.GET /v1/health; root-onlyGET /v1/db,POST /v1/db/verify,POST /v1/db/rebuild. The CLI probes/v1/healthand falls back to/v1/statuson older daemons (404).asc db status|verify|rebuild(EN + RU messages). Nameddb, notindex:commands/indexon the docs site is the command overview page.docs/english|russian/app-management.md— new "🗃️ Local database" section.✅ Checklist
docs/is updated (docs first, then code — seeAGENTS.md).task checkpasses locally (clippy with-D warnings,cargo fmt, tests) — run in WSL: 731 unit tests + all integration tests./v1/health,/v1/db, root-only refusal over the socket).rusqlitebundledcompiles SQLite with the target C toolchaincrossalready provides forring; the cross builds themselves were not run locally (release CI does).proto/changes are backward compatible — no proto changes.🔎 How to test
Measured on 50 process apps with git checkouts (debug build, CLI without a daemon):
asc ls~15 ms with the index vs ~80 ms withASC_NO_INDEX=1. In a container runningasc serve:index.db,-wal,-shmare0600, an app directory written by hand shows up inasc lswithin a second (watcher),asc db verifypicks up a hand-editedsettings.jsonand a removed app.📌 Notes for reviewers
index.dbis safe at any time.observed_state, a shared Docker client per process,asc ls --live.invalid uri character) — paths are not percent-encoded insrc/daemon/client.rs.AGENTS.md.🤖 Generated with Claude Code