Repository navigation
Unify DB write serialization across sync/server/CLI - #16
Merged
Merged
Conversation
Detailed analysis of how Hoard prevents write conflicts when multiple agents share a single SQLite database, covering WAL mode, the WriteCoordinator single-writer thread, busy timeout, optimistic concurrency checks, and async conflict detection. Identifies potential gaps including lack of concurrent write tests. https://claude.ai/code/session_012Zb8mCrSHJtkuCu9mqZCZ2
Introduces flock-based advisory locks to guarantee that database writes are serialized across all processes, not just within a single server. Changes: - New hoard/core/db/lock.py with DatabaseWriteLock (per-write) and ServerSingletonLock (prevents two hoard serve on same DB) - WriteCoordinator now acquires DatabaseWriteLock around each write task - CLI write commands (memory put, memory prune, db migrate) use write_locked() context manager instead of bare connect() - run_sync_with_lock() holds the write lock during sync operations - MCP server acquires ServerSingletonLock on startup, fails fast if another server is already running Includes 9 new tests covering lock semantics, mutual exclusion, WriteCoordinator integration, and concurrent HTTP writes from multiple agents. All 142 existing tests continue to pass. https://claude.ai/code/session_012Zb8mCrSHJtkuCu9mqZCZ2
…inator BackgroundSync runs inside the hoard serve process. When it called write_locked(), it acquired flock(fd_A) on hoard.db.lock. Any MCP write then hit WriteCoordinator._run which tried flock(fd_B) on the same file. On Linux flock is per-open-file-description not per-process, so fd_B blocked. After the 10s timeout the writer thread died from an unhandled DatabaseLockError, and every submit() caller hung forever on event.wait(). Fixes: - Revert run_sync_with_lock to plain connect() since it runs inside the server process where WriteCoordinator already serializes writes. Cross-process safety relies on SQLite WAL + busy_timeout (original behavior). - Wrap the lock acquisition in WriteCoordinator._run with a try/except so that if the lock cannot be acquired, the error is reported to the caller via task.error and the writer thread keeps running. - Move task.event.set() to an outer finally block so callers are always unblocked, even if lock acquisition fails. Adds a regression test that holds the flock externally, verifies the WriteCoordinator reports the error (not hangs), and then verifies the writer still works after the contention clears. All 143 tests pass. https://claude.ai/code/session_012Zb8mCrSHJtkuCu9mqZCZ2
Updates the analysis to document all seven layers of write protection now in place, including the new flock-based DatabaseWriteLock, ServerSingletonLock, and the flock caveat for same-process usage. Adds flow diagrams for CLI-vs-server write serialization and server singleton rejection. Moves resolved gaps to a "Previously Fixed" table. https://claude.ai/code/session_012Zb8mCrSHJtkuCu9mqZCZ2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Testing: