core/audit/trail_db.py hardcodes sqlite3 throughout: _get_conn,
row_factory = sqlite3.Row, raw SQL with ? placeholders, INSERT OR IGNORE,
and WAL pragmas. There is no interface between the query layer and the API
routes that consume it.
That is the right design for one implementation and the wrong one for two. Any
attempt to add DuckDB or ClickHouse today is a rewrite of the call sites rather
than a new driver.
Task
Extract the seam first, without changing behaviour.
- New
core/audit/store/base.py defining an EventStore Protocol. Derive the
method list from what api/routes/audit.py actually calls, not from what
feels like a complete database interface.
- Move the current implementation to
core/audit/store/sqlite_store.py
unchanged. Do not refactor while extracting. Extract, prove the suite still
passes, then improve in a separate commit.
- Select via
core/config.py, defaulting to sqlite.
Why DuckDB is the right second driver, not ClickHouse
DuckDB is embedded, so it keeps the local-first promise with no server to run
and nothing new for the operator to manage. Its columnar scans suit the query
that gets hard at scale: every credential read across many hosts over a long
window. ClickHouse is the correct answer only for an organisation that already
operates one, which makes it a good fit for a separate extension package rather
than a default.
Constraint
The JSONL sinks are the SIEM contract and are not in scope. This is about the
query path only. Existing databases must keep opening: a migration that requires
users to discard their trail is not acceptable for an audit product.
Acceptance criteria
core/audit/trail_db.pyhardcodessqlite3throughout:_get_conn,row_factory = sqlite3.Row, raw SQL with?placeholders,INSERT OR IGNORE,and WAL pragmas. There is no interface between the query layer and the API
routes that consume it.
That is the right design for one implementation and the wrong one for two. Any
attempt to add DuckDB or ClickHouse today is a rewrite of the call sites rather
than a new driver.
Task
Extract the seam first, without changing behaviour.
core/audit/store/base.pydefining anEventStoreProtocol. Derive themethod list from what
api/routes/audit.pyactually calls, not from whatfeels like a complete database interface.
core/audit/store/sqlite_store.pyunchanged. Do not refactor while extracting. Extract, prove the suite still
passes, then improve in a separate commit.
core/config.py, defaulting to sqlite.Why DuckDB is the right second driver, not ClickHouse
DuckDB is embedded, so it keeps the local-first promise with no server to run
and nothing new for the operator to manage. Its columnar scans suit the query
that gets hard at scale: every credential read across many hosts over a long
window. ClickHouse is the correct answer only for an organisation that already
operates one, which makes it a good fit for a separate extension package rather
than a default.
Constraint
The JSONL sinks are the SIEM contract and are not in scope. This is about the
query path only. Existing databases must keep opening: a migration that requires
users to discard their trail is not acceptable for an audit product.
Acceptance criteria
EventStoreprotocol with no sqlite types in its signatures