Add doltlite-go: a database/sql driver with the engine vendored - #2531
Closed
timsehn wants to merge 1 commit into
Closed
Add doltlite-go: a database/sql driver with the engine vendored#2531timsehn wants to merge 1 commit into
timsehn wants to merge 1 commit into
Conversation
Go's sql.Open("doltlite", path) over cgo, with the amalgamation vendored
into the package so no system library is needed. Implements the full
driver set -- Conn, Stmt, Rows, Tx, Result -- so database/sql, and
anything built on it, works unchanged. Version control is plain SQL on
the connection, needing no binding surface of its own.
Why a Go binding at all, given github.com/dolthub/driver embeds Dolt
itself in pure Go: that driver speaks Dolt's on-disk format and cannot
open a DoltLite database, which is a single-file chunk store. Anything
that has to read DoltLite files from Go needs this. The README says so
plainly and points new Go programs at embedded Dolt instead.
Binds text and blobs with SQLITE_TRANSIENT, which cgo requires -- C must
not retain Go memory past the call -- and reads text by length rather
than as a C string, so embedded NULs survive.
test-package.sh stages the module and builds the smoke test against the
staged copy through a replace directive, so a missing vendored
amalgamation fails the test rather than shipping. Build-Test runs it on
ubuntu and macos; Windows is excluded and documented, as for the crate,
since the build links the platform zlib that MSVC lacks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
Closing: speculative. dolthub/doltlite-go already exists and covers the actual need — a pure-Go implementation of the HTTP sync protocol (prollyhash, remoteproto, litestore, remote, litehttp), deliberately without cgo, which is what hosting DoltLite repos calls for. This PR was a different capability: executing SQL against a DoltLite file from Go through the C engine. Nothing needs that today, and the module path here collides with the existing repo's. Recoverable from the packaging/go-driver branch if a Go consumer ever needs to run queries rather than move chunks. |
Contributor
DoltLite source coverage
Merged 201 pooled raw profiles from the distributed Linux correctness jobs. Per-file coverage (98 files)
|
Contributor
DoltLite performance vs PR base
blobpk details
compositepk details
int details
textpk details
vc details
All relative performance gates passed. |
|
SummaryCoverage spans core database lifecycle and transaction flows, external package use, version-control operations, concurrent access, data type and binary-content handling, cleanup after errors, and fail-safe behavior for invalid inputs or incomplete packaging. Overall health is strong across normal workflows, edge cases, concurrency, recovery, and adversarial inputs. Safe to merge — the run found no PR-attributable regressions or failures, and the exercised database, packaging, concurrency, transaction, and input-validation behaviors remain healthy. No merge blocker is indicated; any environment-limited checks are flag-for-later observations rather than evidence of a product defect. Tests run by Ito
Tip Reply with @itoqa to send us feedback on this test run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Go bindings as a
database/sqldriver —sql.Open("doltlite", path)— over cgo, with the amalgamation vendored into the package so there's no system library to install.Why a Go binding, given embedded Dolt exists
I argued earlier in this repo's history that Go didn't need one, because
github.com/dolthub/driverembeds Dolt in pure Go with no cgo. That reasoning holds for new programs wanting versioned SQL — but it doesn't cover the case that motivated this: embedded Dolt speaks Dolt's on-disk format and cannot open a DoltLite database, which is a single-file chunk store. Anything that has to read DoltLite files from Go — the DoltHub API, for instance — needs a real binding.The README states this plainly and sends new Go programs to embedded Dolt instead, so the packages don't compete for the same use case.
Shape
Driver,Conn,Stmt,Rows,Tx,Result— sodatabase/sqland everything layered on it works unchanged.SQLITE_TRANSIENT, which cgo requires (C must not retain Go memory past the call), and reads text by length rather than as a C string so embedded NULs survive.Tests
test-package.shstages the module and builds the smoke test against the staged copy via areplacedirective, so a missing vendored amalgamation fails the test instead of shipping. 22 checks pass locally: every value type (UTF-8 text, blobs with embedded NULs, null, int64, real, empty string),ErrNoRows, column names, error paths, commit/rollback transactions, and a commit → branch → checkout → merge →dolt_loground trip.gofmtandgo vetclean.Build-Test runs it on ubuntu and macos. Windows is excluded and documented, same as the Rust crate: the build links the platform's zlib, which MSVC doesn't provide.
Distribution needs a repo
Go modules resolve from a VCS path, so publishing needs a
dolthub/doltlite-gorepository (same pattern asdoltlite-php) that release wiring pushes the staged module and a matching tag into. Could you create it when convenient? I'll wirerelease-goonce it exists.Until then this is usable directly with a
replacedirective pointing atpackaging/go, which is what the DoltHub API side could do immediately if it's in a hurry.🤖 Generated with Claude Code