Skip to content

[Feature] MongoDB adapter to the @logtide/reservoir #157

@Polliog

Description

@Polliog

Feature Description

Add a MongoDB adapter to the @logtide/reservoir storage abstraction layer, allowing Logtide to use MongoDB (or MongoDB-compatible databases like DocumentDB, FerretDB, or Cosmos DB for MongoDB) as its storage backend.

Problem/Use Case

Many teams — especially in the Node.js ecosystem and in organizations already operating MongoDB clusters — would prefer to run Logtide on infrastructure they already have rather than introducing a new database. Currently the only available reservoir backends are TimescaleDB (default) and ClickHouse (enterprise scale), which means these teams must provision and maintain an entirely new database just to run Logtide.

MongoDB is one of the most widely deployed databases in the world. Supporting it as a reservoir backend removes a significant adoption barrier for teams in that ecosystem, and opens Logtide to cloud environments where managed MongoDB services (AWS DocumentDB, Azure Cosmos DB, MongoDB Atlas) are already the standard.

Proposed Solution

Implement a @logtide/reservoir-mongodb adapter that conforms to the existing @logtide/reservoir interface, enabling transparent swap of the storage backend without any changes to the Logtide core or application code.

The adapter should cover:

  • Log storage and retrieval: capped collections or time-based partitioning for efficient time-range queries on log entries
  • Full-text search: leverage MongoDB Atlas Search or the native $text index for log message search, with fallback to regex for self-hosted deployments without Atlas
  • Aggregations: implement the same aggregation queries (count by level/service, hourly stats) using MongoDB's aggregation pipeline
  • Traces and spans: store trace and span documents with the same correlation fields (trace_id, span_id, parent_span_id) used by other backends
  • TTL-based retention: use MongoDB TTL indexes to implement automatic log retention policies, consistent with the TimescaleDB compression/retention model
  • Metrics storage (once issue #1 is implemented): time-bucketed documents for metric samples with aggregation pipeline rollups

Configuration via environment variable consistent with the existing reservoir pattern:

LOGTIDE_RESERVOIR=mongodb
LOGTIDE_MONGODB_URI=mongodb://...

Alternatives Considered

  • FerretDB as a compatibility layer over TimescaleDB/Postgres: FerretDB speaks the MongoDB wire protocol but stores data in PostgreSQL. This would allow MongoDB-compatible clients but wouldn't actually help teams that want to use their existing MongoDB infrastructure.
  • Asking users to run a MongoDB → PostgreSQL sync: rejected because it adds operational overhead and defeats the purpose of native backend support.

Implementation Details (Optional)

  • The adapter should live in packages/reservoir-mongodb/ following the same package structure as the existing adapters
  • Use the official MongoDB Node.js driver (mongodb) — no ODM (Mongoose or similar) to keep the dependency surface minimal and consistent with the query-builder approach used in the rest of the codebase (Kysely for SQL backends)
  • Log collections should use time-based document partitioning or MongoDB's native sharding to support large datasets without full collection scans on time-range queries
  • The reservoir interface should not be extended — the MongoDB adapter must implement all existing methods and return data in the same normalized shape as other adapters, ensuring complete interchangeability
  • Consider adding a @logtide/reservoir-mongodb section to the Docker Compose examples with a mongo:7 service for local testing

Priority

  • Critical - Blocking my usage of LogTide
  • High - Would significantly improve my workflow
  • Medium - Nice to have
  • Low - Minor enhancement

Target Users

  • DevOps Engineers
  • Developers
  • Security/SIEM Users
  • System Administrators
  • All Users

Additional Context

MongoDB compatibility also implies potential support for MongoDB-compatible managed services without additional work: AWS DocumentDB, Azure Cosmos DB for MongoDB API, and FerretDB. This significantly expands the cloud deployment options for teams that want Logtide on managed infrastructure without operating a PostgreSQL or ClickHouse cluster themselves.

This adapter would make Logtide one of the few observability platforms with genuine multi-database backend support — a meaningful differentiator for self-hosted teams with diverse infrastructure constraints.

Contribution

  • I would like to work on implementing this feature

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions