Skip to content

Extract an EventStore protocol before adding a second storage backend #35

Description

@blitzcrieg1

core/audit/trail_db.py hardcodes sqlite3 throughout: _get_conn,
row_factory = sqlite3.Row, raw SQL with ? placeholders, INSERT OR IGNORE,
and WAL pragmas. There is no interface between the query layer and the API
routes that consume it.

That is the right design for one implementation and the wrong one for two. Any
attempt to add DuckDB or ClickHouse today is a rewrite of the call sites rather
than a new driver.

Task

Extract the seam first, without changing behaviour.

  • New core/audit/store/base.py defining an EventStore Protocol. Derive the
    method list from what api/routes/audit.py actually calls, not from what
    feels like a complete database interface.
  • Move the current implementation to core/audit/store/sqlite_store.py
    unchanged. Do not refactor while extracting. Extract, prove the suite still
    passes, then improve in a separate commit.
  • Select via core/config.py, defaulting to sqlite.

Why DuckDB is the right second driver, not ClickHouse

DuckDB is embedded, so it keeps the local-first promise with no server to run
and nothing new for the operator to manage. Its columnar scans suit the query
that gets hard at scale: every credential read across many hosts over a long
window. ClickHouse is the correct answer only for an organisation that already
operates one, which makes it a good fit for a separate extension package rather
than a default.

Constraint

The JSONL sinks are the SIEM contract and are not in scope. This is about the
query path only. Existing databases must keep opening: a migration that requires
users to discard their trail is not acceptable for an audit product.

Acceptance criteria

  • EventStore protocol with no sqlite types in its signatures
  • All existing tests pass against the sqlite implementation, unmodified
  • A second implementation can be registered without editing route code

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions