Skip to content

Data correctness: real backups, empty-lane detection, habits race, and honest docs - #37

Merged
ggapp1 merged 6 commits into
mainfrom
fix/data-correctness
Oct 29, 2026
Merged

ggapp1 merged 6 commits into
mainfrom
fix/data-correctness

Conversation

@ggapp1

@ggapp1 ggapp1 commented Oct 29, 2026

Copy link
Copy Markdown
Owner

The correctness/data findings from the whole-project review. One commit each, each with a regression test that fails on main.

  • The .bak was not a backup (HIGH). One fixed <db>.bak was rewritten at the top of every upgrade, so a no-op db migrate replaced the snapshot, and a crash-restart after a bad partial migration overwrote the only good copy with the half-migrated state. Now:

    • a backup is taken only when something is pending, using the same predicates as doctor and the startup gate;
    • each backup is a new <db>.bak-<UTC timestamp>, and the newest five are kept;
    • upgrade_database takes one backup per run instead of one per tier;
    • a backup is skipped when the newest one already records the db's migration state. Without this, the shipped unit's Restart=always/RestartSec=5 against a migration that always fails takes a backup every 5s, and the prune deletes the clean copy within ~25s. While anything is pending the startup gate refuses to run, so an unchanged migration state means an unchanged database. A crash loop now leaves at most two backups: the clean one, plus one half-migrated copy if a revision did apply.

    README covers the backup location, the retention, and how to restore; both .gitignores match the new names.

  • A suite lane resolving to no revisions raised nothing and read as "up to date". Every shipped suite lane has revisions, so an empty one is a packaging error: it now raises, naming the app and the directory, and doctor and the startup gate surface it.

  • habits.tick_habit's race backstop raised MissingGreenlet because rollback expired habit.name. The name is now read before the try.

  • app.db() read isolation is documented, not changed. Reads are autocommit under pysqlite; the docstring and README now say so and show the atomic-UPDATE / unique-constraint alternatives. I audited the three suite apps' read-modify-writes and all are safe: idempotent writes, deletes of immutable rows, or the unique-constraint backstop.

  • llm/client.py now catches SQLAlchemyError like its siblings, so a shutdown-race InterfaceError no longer fails a completed call.

  • search_notes now says its case-insensitivity is ASCII-only, which is all SQLite's LIKE does. Folding in Python would mean an unbounded scan, since the query has no LIMIT.

Fast suite: 1134 passed locally; ruff check and format --check clean.

@ggapp1
ggapp1 merged commit 1405f00 into main Oct 29, 2026
6 checks passed
@ggapp1
ggapp1 deleted the fix/data-correctness branch October 29, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant