Skip to content

Commit 72b2019

Browse files
committed
Enable exact-version default Dolt writes
1 parent 49da0f9 commit 72b2019

12 files changed

Lines changed: 341 additions & 91 deletions

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,7 @@ jobs:
143143
steps: *sql-test-steps
144144

145145
dolt-integration:
146-
name: Dolt ${{ matrix.dolt }} read-only / fail-closed integration
146+
name: Dolt ${{ matrix.dolt }} default-write integration
147147
runs-on: ubuntu-latest
148148
strategy:
149149
fail-fast: false

‎README.md‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -96,25 +96,27 @@ SAV/ZSAV export for the semantics exposed by that engine. SQLite is the local
9696
reference path.
9797
PostgreSQL, MySQL, MariaDB, and Dolt are each covered by separate service-backed CI
9898
conformance checks. Dolt support is an independent core profile for the
99-
canonical stable range `>=2.2.2,<2.3.0`; earlier patches, other families,
100-
noncanonical versions, and unknown MySQL-wire products fail closed.
99+
exact write-supported versions 2.2.2 and 2.2.3; all other versions (including
100+
2.3.0), noncanonical versions, and unknown MySQL-wire products fail closed for writes.
101101

102102
The supported family claims are broader than the deliberately exact CI
103103
evidence points: PostgreSQL 17.x/18.x is exercised at 17.10/18.4, MySQL
104104
8.4.x/9.7.x at 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at
105105
11.4.12/11.8.8/12.3.2. Each service job checks the normalized live server
106106
version against its exact matrix entry before that run can count as evidence.
107-
Dolt claims the conservative 2.2.x range `>=2.2.2,<2.3.0`; its full service
108-
suite is exercised independently at exact versions 2.2.2 and 2.2.3 using
109-
immutable container-image digests.
107+
Dolt's default-write service checks run independently at exact versions 2.2.2
108+
and 2.2.3 using immutable container-image digests. Release/CI owns this evidence;
109+
normal callers need no declaration or evidence files. The optional explicit
110+
`DoltConformanceSource` override remains strict and does not fall back to the
111+
default policy if its declarations are missing, invalid, or mismatched.
110112

111113
| Engine/profile | Runtime supported policy | Exact CI-tested versions |
112114
| --- | --- | --- |
113115
| SQLite core / optional workflow | Core `>=3.24.0,<4.0.0`; optional workflow `>=3.35.0,<4.0.0` | Runtime-provided SQLite on Python 3.11–3.14 runners; not a pinned server image |
114116
| PostgreSQL | 17.x and 18.x | 17.10 and 18.4 |
115117
| MySQL | 8.4.x and 9.7.x | 8.4.11 and 9.7.2 |
116118
| MariaDB | 11.4.x, 11.8.x, and 12.3.x | 11.4.12, 11.8.8, and 12.3.2 |
117-
| Dolt | 2.2.x with `>=2.2.2,<2.3.0` | 2.2.2 and 2.2.3 |
119+
| Dolt writes | Exactly 2.2.2 and 2.2.3 | 2.2.2 and 2.2.3 |
118120

119121
Microsoft SQL Server (MSSQL) remains roadmap-only and is not a supported
120122
runtime profile; see the specification's [MSSQL roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
@@ -124,7 +126,7 @@ Use these explicit SQLAlchemy URLs:
124126
- SQLite: `sqlite:///dataset.sqlite`
125127
- PostgreSQL: `postgresql+psycopg://user:password@host/database`
126128
- MySQL/MariaDB: `mysql+pymysql://user:password@host/database`
127-
- Dolt `>=2.2.2,<2.3.0`: `mysql+pymysql://user:password@host/database` (detected by server identity)
129+
- Dolt 2.2.2 or 2.2.3: `mysql+pymysql://user:password@host/database` (detected by server identity)
128130

129131
The Dolt core profile supports strict wide-table import, validation, and export;
130132
the separate Transformation Workflow is unsupported.
@@ -135,8 +137,12 @@ dictionary semantics cannot be reproduced, it stops until you pass the exact
135137
diagnostic code with `--allow-loss`. Diagnostics are returned to the caller, not
136138
persisted. Reads, validation, and SAV/ZSAV export never write to the database,
137139
including on failure. Export returns no `operation_id` and needs only read
138-
permissions; Dolt reads do not require a write-conformance declaration. Imports
139-
and transformations still require an exact matching Dolt write declaration.
140+
permissions; Dolt reads have no write-version gate. Imports and transformations
141+
use the packaged exact-version policy by default. Active driver/identity checks,
142+
limit preflight, and the clean expected branch/HEAD guard for in-place apply
143+
remain mandatory. Dolt's reported limits are adapter safety budgets, narrowed
144+
by the active packet limit, not proven native server ceilings; full boundary
145+
conformance is not claimed.
140146

141147
The matrix is also available to Python callers as
142148
`openstatspec.capability_matrix()`. It distinguishes supported semantics from

‎docs/release-readiness.md‎

Lines changed: 35 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,17 @@ database contract.
2020
| PostgreSQL | `postgresql+psycopg://…` | 17.x and 18.x | 17.10 and 18.4 |
2121
| MySQL | `mysql+pymysql://…` | 8.4.x and 9.7.x | 8.4.11 and 9.7.2 |
2222
| MariaDB | `mysql+pymysql://…` | 11.4.x, 11.8.x, and 12.3.x | 11.4.12, 11.8.8, and 12.3.2 |
23-
| Dolt | `mysql+pymysql://…` | 2.2.x with `>=2.2.2,<2.3.0` | 2.2.2 and 2.2.3 |
23+
| Dolt writes | `mysql+pymysql://…` | Exactly 2.2.2 and 2.2.3 | 2.2.2 and 2.2.3 |
2424

2525
Separate service matrices test the shared MySQL/MariaDB profile contract while
2626
retaining distinct active-server identities and version claims. Dolt is an
27-
independent core profile that accepts only canonical stable versions in
28-
`>=2.2.2,<2.3.0`; the optional Transformation Workflow remains SQLite-only. This does not
29-
claim coverage for every server configuration. The machine-readable capability
30-
declaration reports both the theoretical profile boundaries and effective
31-
limits observed from the active connection.
27+
independent core profile with a packaged exact-version write policy; normal
28+
callers supply no conformance files. An explicit external declaration source
29+
remains a strict override. Read-only validation/export has no write-version gate.
30+
The optional Transformation Workflow remains SQLite-only. This does not claim
31+
coverage for every server configuration or proven Dolt native limit ceilings.
32+
Dolt capabilities report adapter safety budgets and active packet constraints;
33+
full boundary conformance remains pending.
3234
Every server service job compares the normalized live product version with its
3335
exact matrix entry, so a moved or mismatched image cannot substantiate the CI
3436
declaration. SQLite's optional Transformation Workflow retains its narrower
@@ -102,6 +104,27 @@ implemented openstatspec.frontends.spss package. Stata and SAS remain empty
102104
source-tree placeholders and must expose no compiler, apply API, CLI choice,
103105
capability claim, or implied support.
104106

107+
## Default Dolt write verification
108+
109+
Local release verification used the exact 2.2.2 and 2.2.3 image digests in
110+
`.github/workflows/ci.yml`, isolated ports 13482/13483, and disposable `/tmp`
111+
database directories. With `PYTHONPATH=src:../pyspssio`,
112+
`OPENSTATSPEC_SPECIFICATION_DIR=/tmp/oss-php-spec-cd8f198`, and each matching
113+
`OPENSTATSPEC_DOLT_URL` / `OPENSTATSPEC_EXPECTED_DOLT_VERSION`:
114+
115+
- `../python/.venv/bin/python -m pytest`: **366 passed, 49 skipped per pin**.
116+
- CI selection `-m 'services and not candidate_evidence'`: **7 passed,
117+
41 skipped per pin**; other database services were not configured locally.
118+
- Dolt 2.3.0 on port 13387: read-only export and unknown-version write rejection
119+
checks passed (**19 passed**) using disposable fixture databases/users.
120+
- `python -m compileall -q src tests` and `git diff --check` passed.
121+
122+
Live failures exposed and now cover two previously dormant write blockers:
123+
owned table additions being misclassified as unrelated diffs, and Dolt retaining
124+
`@@autocommit=1` despite the driver's setting. Writes now explicitly begin their
125+
transaction. The candidate storage/identifier/column smoke probe also passed on
126+
both pins; it does **not** establish full value/row/statement limit conformance.
127+
105128
## Maintainer checks before tagging
106129

107130
1. Publish the pinned `openstatspec-pyspssio==0.5.1.post2` engine distribution
@@ -110,7 +133,12 @@ capability claim, or implied support.
110133
2. Run `python -m pytest -m "not services"`.
111134
3. Confirm the GitHub Actions matrix is green for exact PostgreSQL 17.10/18.4,
112135
MySQL 8.4.11/9.7.2, MariaDB 11.4.12/11.8.8/12.3.2, and exact Dolt
113-
2.2.2/2.2.3 service evidence from the immutable image pins in CI.
136+
2.2.2/2.2.3 service evidence from the immutable image pins in CI. The Dolt
137+
jobs must perform default import/validate/SAV+ZSAV export, in-place recode
138+
and labels with unchanged dataset/table counts and HEAD, injected data,
139+
catalog, and audit rollback checks, and failed-import cleanup. Release/CI
140+
owns this evidence, not per-user runtime declaration files. Candidate limit
141+
probes remain non-claiming and separate from these write gates.
114142
4. Build with `python -m build` and install the generated wheel in a clean
115143
environment.
116144
5. Confirm `openstatspec capabilities` reflects the intended support boundary.

‎src/openstatspec/sql/capabilities.py‎

Lines changed: 43 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,10 @@
2424

2525
DOLT_WRITE_CONFORMANCE = {
2626
"declaration_schema_id": "openstatspec-dolt-adapter-declaration-v1",
27-
"write_enabled": False,
28-
"status": "adapter_owned_concrete_declarations_required",
27+
"write_enabled": True,
28+
"status": "packaged_exact_version_policy",
29+
"declaration_count": 0,
30+
"declarations_available": False,
2931
}
3032

3133

@@ -82,7 +84,7 @@ def _dolt_driver_eligible(database_url: str) -> bool:
8284
def dolt_operational_write_enabled(
8385
active: Mapping[str, Any], *, declaration_matched: bool,
8486
) -> bool:
85-
"""Require both exact declaration and claimed active driver for writes."""
87+
"""Require an exact write-policy match and the claimed active driver."""
8688
return declaration_matched and bool(active.get("driver_eligible", False))
8789

8890

@@ -91,7 +93,12 @@ def _dolt_write_enabled(
9193
*,
9294
active_product_version: str | None = None,
9395
) -> bool:
94-
conformance = _conformance_source(source)
96+
if source is None:
97+
return (
98+
True if active_product_version is None
99+
else active_product_version.strip() in SERVER_POLICIES["dolt"]["claimed"]
100+
)
101+
conformance = source
95102
if active_product_version is None:
96103
return bool(conformance.status()["write_enabled"])
97104
conformance.require_exact_match(
@@ -115,8 +122,8 @@ def _dolt_write_enabled(
115122
"ci": ["MariaDB 11.4.12", "MariaDB 11.8.8", "MariaDB 12.3.2"],
116123
},
117124
"dolt": {
118-
"claimed": [],
119-
"ci": [],
125+
"claimed": ["2.2.2", "2.2.3"],
126+
"ci": ["2.2.2", "2.2.3"],
120127
},
121128
"postgresql": {
122129
"claimed": ["PostgreSQL 17.x", "PostgreSQL 18.x"],
@@ -131,13 +138,13 @@ def profile_declarations(
131138
dolt_conformance_source: DoltConformanceSource | None = None,
132139
) -> dict[str, dict[str, Any]]:
133140
"""Declare every profile and, optionally, one active connection."""
134-
source = _conformance_source(dolt_conformance_source)
141+
source = dolt_conformance_source
135142
active = (
136143
active_connection(database_url, dolt_conformance_source=source)
137144
if database_url else None
138145
)
139146
dolt_declaration = None
140-
if active and active["profile"] == "dolt":
147+
if source is not None and active and active["profile"] == "dolt":
141148
try:
142149
dolt_declaration = source.require_exact_match(
143150
active_product_version=active["raw_product_version"],
@@ -290,7 +297,7 @@ def effective_profile(
290297
*,
291298
dolt_conformance_source: DoltConformanceSource | None = None,
292299
) -> tuple[SqlProfile, dict[str, Any]]:
293-
"""Resolve and enforce the write profile, including exact Dolt conformance."""
300+
"""Enforce the write profile and exact Dolt policy or explicit declaration."""
294301
return _effective_profile(database_url, dolt_conformance_source, for_write=True)
295302

296303

@@ -307,7 +314,7 @@ def _effective_profile(
307314
database_url: str, dolt_conformance_source: DoltConformanceSource | None,
308315
*, for_write: bool,
309316
) -> tuple[SqlProfile, dict[str, Any]]:
310-
source = _conformance_source(dolt_conformance_source)
317+
source = dolt_conformance_source
311318
configured = validate_connection_url(database_url)
312319
active = active_connection(database_url, dolt_conformance_source=source)
313320
if configured is not MYSQL and active["profile"] != configured.name:
@@ -326,7 +333,7 @@ def _effective_profile(
326333
# write-conformance envelope or INSERT statement/packet limits.
327334
return DOLT, active
328335
dolt_declaration = None
329-
if active["profile"] == "dolt":
336+
if active["profile"] == "dolt" and source is not None:
330337
dolt_declaration = source.require_exact_match(
331338
active_product_version=active["raw_product_version"],
332339
specification_commit=_bound_specification_commit(),
@@ -392,13 +399,18 @@ def _profile(
392399
dolt_declaration: Mapping[str, Any] | None = None,
393400
) -> dict[str, Any]:
394401
dolt_envelope = name == "dolt"
402+
default_dolt = dolt_envelope and dolt_conformance_source is None
403+
dolt_supported = bool(
404+
default_dolt and active and active["profile"] == "dolt"
405+
and server_version_supported("dolt", active["raw_product_version"])
406+
)
395407
identifier = {
396408
"value": profile.identifier_limit,
397409
"unit": "characters" if name in {"mysql", "mariadb"} else "bytes",
398410
"source": (
399411
"MySQL/MariaDB native identifier limit" if name in {"mysql", "mariadb"}
400412
else "PostgreSQL NAMEDATALEN minus one native byte limit" if name == "postgresql"
401-
else "OpenStatSpec Dolt adapter envelope pending pinned live boundary evidence"
413+
else "OpenStatSpec Dolt adapter safety envelope (not a proven server ceiling)"
402414
if dolt_envelope
403415
else "OpenStatSpec profile boundary; SQLite has no fixed native identifier limit"
404416
),
@@ -416,7 +428,7 @@ def _profile(
416428
if dolt_envelope:
417429
adapter_envelope = {
418430
**profile_limits,
419-
"limit_basis": "proposed_adapter_envelope",
431+
"limit_basis": "adapter_safety_envelope",
420432
"evidence_status": "pending_pinned_live_conformance",
421433
}
422434
theoretical = {
@@ -435,15 +447,16 @@ def _profile(
435447
effective = None
436448
status = "not_connected"
437449
if active and active["profile"] == name and (
438-
not dolt_envelope or dolt_declaration is not None
450+
not dolt_envelope or dolt_declaration is not None or dolt_supported
439451
):
440452
effective = (
441453
dolt_effective_limits(dolt_declaration)
442454
if dolt_envelope and dolt_declaration is not None
455+
else dict(adapter_envelope) if default_dolt
443456
else dict(theoretical)
444457
)
445458
default_source = (
446-
"proposed adapter envelope pending pinned live conformance"
459+
"adapter safety envelope; full boundary conformance not claimed"
447460
if dolt_envelope else "profile theoretical engine ceiling"
448461
)
449462
sources = {
@@ -482,7 +495,9 @@ def _profile(
482495
effective["maximum_value_bytes"] = min(
483496
effective["maximum_value_bytes"], payload,
484497
)
485-
effective["maximum_statement_bytes"] = payload
498+
effective["maximum_statement_bytes"] = min(
499+
effective["maximum_statement_bytes"] or payload, payload,
500+
)
486501
sources["maximum_value_bytes"] = "active @@max_allowed_packet worst-case payload"
487502
sources["maximum_statement_bytes"] = "active @@max_allowed_packet worst-case payload"
488503
status = "active_connection_mixed"
@@ -494,20 +509,25 @@ def _profile(
494509
policy = SERVER_POLICIES[name]
495510
dolt_declarations = (
496511
_validated_dolt_declarations(dolt_conformance_source)
497-
if dolt_envelope else ()
512+
if dolt_envelope and not default_dolt else ()
498513
)
499514
dolt_claimed_versions = sorted({
500515
version
501516
for declaration in dolt_declarations
502-
for version in declaration["claimed_product_versions"]
517+
for version in declaration.get(
518+
"claimed_product_versions", [declaration["active_product_version"]],
519+
)
503520
})
504521
dolt_tested_versions = sorted({
505522
version
506523
for declaration in dolt_declarations
507-
for version in declaration["tested_product_versions"]
524+
for version in declaration.get(
525+
"tested_product_versions", [declaration["active_product_version"]],
526+
)
508527
})
509528
dolt_status = (
510-
_conformance_source(dolt_conformance_source).status()
529+
DOLT_WRITE_CONFORMANCE if default_dolt
530+
else _conformance_source(dolt_conformance_source).status()
511531
if dolt_envelope else None
512532
)
513533
return {
@@ -520,10 +540,10 @@ def _profile(
520540
"specification_commit": SPECIFICATION_COMMIT,
521541
"driver": "psycopg" if name == "postgresql" else "PyMySQL" if name in MYSQL_WIRE_PROFILES else "sqlite3",
522542
"claimed_server_versions": (
523-
dolt_claimed_versions if dolt_envelope else policy["claimed"]
543+
dolt_claimed_versions if dolt_envelope and not default_dolt else policy["claimed"]
524544
),
525545
"ci_tested_server_versions": (
526-
dolt_tested_versions if dolt_envelope else policy["ci"]
546+
dolt_tested_versions if dolt_envelope and not default_dolt else policy["ci"]
527547
),
528548
"write_conformance": (
529549
{
@@ -543,7 +563,7 @@ def _profile(
543563
),
544564
"operational_write_enabled": (
545565
dolt_operational_write_enabled(
546-
active, declaration_matched=dolt_declaration is not None,
566+
active, declaration_matched=dolt_declaration is not None or dolt_supported,
547567
)
548568
if dolt_envelope and active and active["profile"] == name
549569
else (
@@ -599,7 +619,6 @@ def server_version_supported(
599619
"postgresql": {(17,), (18,)},
600620
"mysql": {(8, 4), (9, 7)},
601621
"mariadb": {(11, 4), (11, 8), (12, 3)},
602-
"dolt": {(2, 2, 2)},
603622
}[profile]
604623
width = len(next(iter(allowed)))
605624
return version[:width] in allowed

‎src/openstatspec/sql/dolt_conformance.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
"""Adapter-owned validation for language-neutral Dolt declarations."""
1+
"""Optional strict validation for external, adapter-owned Dolt declarations.
2+
3+
Default writes use the exact-version policy in capabilities, not this registry.
4+
"""
25

36
from __future__ import annotations
47

@@ -280,7 +283,7 @@ class DoltConformanceSource:
280283

281284
@classmethod
282285
def packaged(cls) -> "DoltConformanceSource":
283-
"""Return the empty built-in registry; no Dolt write claim is packaged."""
286+
"""Return the legacy empty registry, not the default write-version policy."""
284287
return cls()
285288

286289
@classmethod

‎src/openstatspec/sql/inplace_transform.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -984,6 +984,9 @@ def _run_in_place_submission(
984984
# catalog mutations roll back together before the write lock
985985
# is released.
986986
connection.exec_driver_sql("BEGIN")
987+
if profile.name == "dolt":
988+
# Dolt may retain @@autocommit=1 despite PyMySQL's setting.
989+
connection.exec_driver_sql("BEGIN")
987990
require_verified_catalog(connection)
988991
if profile.name == "dolt":
989992
if not expected_branch or not expected_head:

0 commit comments

Comments
 (0)