Skip to content

mongodb: add importer/exporter integration - #78

Open
Abdul-Moiz31 wants to merge 1 commit into
PlakarKorp:mainfrom
Abdul-Moiz31:mongodb-integration
Open

mongodb: add importer/exporter integration#78
Abdul-Moiz31 wants to merge 1 commit into
PlakarKorp:mainfrom
Abdul-Moiz31:mongodb-integration

Conversation

@Abdul-Moiz31

Copy link
Copy Markdown

New integration for MongoDB: back up a single logical database via mongodump/mongorestore --archive and restore it to any MongoDB server. Everything lives under mongodb/, in the same family as the existing postgresql and mysql connectors.

Discussed with @Mathieu-PLKR over Discord — scope (logical dump/restore only, mongodb:// URI scheme) confirmed before implementation.

What it does

  • Importer (mongodb://[user:pass@]host[:port]/database): wraps mongodump --archive, streamed directly as a single record (/dump.archive) rather than written to a temp file first, plus a lightweight /manifest.json (host, port, database, mongodump version — no live server introspection in v1).
  • Exporter: wraps mongorestore --archive, reading straight from the record's io.Reader. Supports drop (maps to mongorestore --drop) as the closest equivalent to Postgres's clean option.
  • Importer/exporter only — no storage connector, since MongoDB isn't a spot to durably store Kloset's own chunks (same reasoning as postgresql/mysql).

Options

Connection: location (mongodb:// URI) or standalone host/port/username/password/database overrides, same override pattern as postgresql's schema. tls / tls_ca_file for TLS. mongo_bin_dir to point at a non-$PATH install of the MongoDB database tools. A database (via location path or database key) is required in v1 — enforced both in the JSON schema (anyOf: [location, database]) and at runtime.

Credentials are never placed on the mongodump/mongorestore command line — the connection URI (including password) is written to a 0600 temp --config YAML file instead, mirroring how postgresql injects PGPASSWORD via env and mysql uses --defaults-extra-file.

Explicitly out of scope for v1

--nsInclude/--nsExclude collection filtering, oplog/point-in-time restore, sharded-cluster flags, Atlas-specific auth, mongodb+srv:// (all noted as fast-follows, not blockers).

Validation

  • go build ./..., go vet ./..., go mod tidy — clean
  • make build — both mongodbImporter and mongodbExporter build successfully
  • Testcontainers e2e test (tests/logical/single_test.go) run against live Docker: seeds a mongo:7 source, backs it up via mongodb://.../testdb, verifies the snapshot contains /dump.archive + /manifest.json, restores into a fresh mongo:7 target, and confirms the document count matches — full roundtrip passes.

@Abdul-Moiz31

Copy link
Copy Markdown
Author

Hi @mathieu-plak

Following up on the MongoDB integration we discussed over Discord.

This PR implements the scope you approved: single-database logical backup/restore via mongodump/mongorestore --archive, mongodb:// URI scheme, no oplog/sharded-cluster/Atlas support in v1.

Ran the full test suite locally, including the testcontainers e2e roundtrip against real MongoDB containers , passes end to end.

Happy to adjust anything that doesn't match what you had in mind for v1.

Looking forward !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant