-
Notifications
You must be signed in to change notification settings - Fork 12
Description
Feature Description
Add a MongoDB adapter to the @logtide/reservoir storage abstraction layer, allowing Logtide to use MongoDB (or MongoDB-compatible databases like DocumentDB, FerretDB, or Cosmos DB for MongoDB) as its storage backend.
Problem/Use Case
Many teams — especially in the Node.js ecosystem and in organizations already operating MongoDB clusters — would prefer to run Logtide on infrastructure they already have rather than introducing a new database. Currently the only available reservoir backends are TimescaleDB (default) and ClickHouse (enterprise scale), which means these teams must provision and maintain an entirely new database just to run Logtide.
MongoDB is one of the most widely deployed databases in the world. Supporting it as a reservoir backend removes a significant adoption barrier for teams in that ecosystem, and opens Logtide to cloud environments where managed MongoDB services (AWS DocumentDB, Azure Cosmos DB, MongoDB Atlas) are already the standard.
Proposed Solution
Implement a @logtide/reservoir-mongodb adapter that conforms to the existing @logtide/reservoir interface, enabling transparent swap of the storage backend without any changes to the Logtide core or application code.
The adapter should cover:
- Log storage and retrieval: capped collections or time-based partitioning for efficient time-range queries on log entries
- Full-text search: leverage MongoDB Atlas Search or the native
$textindex for log message search, with fallback to regex for self-hosted deployments without Atlas - Aggregations: implement the same aggregation queries (count by level/service, hourly stats) using MongoDB's aggregation pipeline
- Traces and spans: store trace and span documents with the same correlation fields (trace_id, span_id, parent_span_id) used by other backends
- TTL-based retention: use MongoDB TTL indexes to implement automatic log retention policies, consistent with the TimescaleDB compression/retention model
- Metrics storage (once issue #1 is implemented): time-bucketed documents for metric samples with aggregation pipeline rollups
Configuration via environment variable consistent with the existing reservoir pattern:
LOGTIDE_RESERVOIR=mongodb
LOGTIDE_MONGODB_URI=mongodb://...
Alternatives Considered
- FerretDB as a compatibility layer over TimescaleDB/Postgres: FerretDB speaks the MongoDB wire protocol but stores data in PostgreSQL. This would allow MongoDB-compatible clients but wouldn't actually help teams that want to use their existing MongoDB infrastructure.
- Asking users to run a MongoDB → PostgreSQL sync: rejected because it adds operational overhead and defeats the purpose of native backend support.
Implementation Details (Optional)
- The adapter should live in
packages/reservoir-mongodb/following the same package structure as the existing adapters - Use the official MongoDB Node.js driver (
mongodb) — no ODM (Mongoose or similar) to keep the dependency surface minimal and consistent with the query-builder approach used in the rest of the codebase (Kysely for SQL backends) - Log collections should use time-based document partitioning or MongoDB's native sharding to support large datasets without full collection scans on time-range queries
- The reservoir interface should not be extended — the MongoDB adapter must implement all existing methods and return data in the same normalized shape as other adapters, ensuring complete interchangeability
- Consider adding a
@logtide/reservoir-mongodbsection to the Docker Compose examples with amongo:7service for local testing
Priority
- Critical - Blocking my usage of LogTide
- High - Would significantly improve my workflow
- Medium - Nice to have
- Low - Minor enhancement
Target Users
- DevOps Engineers
- Developers
- Security/SIEM Users
- System Administrators
- All Users
Additional Context
MongoDB compatibility also implies potential support for MongoDB-compatible managed services without additional work: AWS DocumentDB, Azure Cosmos DB for MongoDB API, and FerretDB. This significantly expands the cloud deployment options for teams that want Logtide on managed infrastructure without operating a PostgreSQL or ClickHouse cluster themselves.
This adapter would make Logtide one of the few observability platforms with genuine multi-database backend support — a meaningful differentiator for self-hosted teams with diverse infrastructure constraints.
Contribution
- I would like to work on implementing this feature