Skip to content

Add owned runtime migration targets and guarded application (PR A) - #97

Open
Dastari wants to merge 11 commits into
mainfrom
feat/runtime-migration-targets-20260930
Open

Dastari wants to merge 11 commits into
mainfrom
feat/runtime-migration-targets-20260930

Conversation

@Dastari

@Dastari Dastari commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Runtime hosts can convert a validated schema into the existing canonical physical migration target without leaked static index storage, preview scoped changes without writes, then separately apply a reviewed immutable plan. Explicit managed ownership bounds DDL; static/system composition and full incoming-FK checks establish the physical dependency environment for later runtime mutations.

Reconciled with current released main while preserving the approved A contract:

  • Base: df474c761c4057cb643e27a6582dd0376f56e31f (released ORM/macros 0.39.0).
  • Previous reviewed head: e41866d582c36fd920044f6461cb15c6bd446f17.
  • Current head: f2665ce68ed7fdbccb272cb8a783b6379ae5fd82.
  • ORM/macros: 0.40.0, unpublished; macros alignment only. SQLite/PostgreSQL features remain explicit. No new dynamic GraphQL feature in A.
  • The reviewed owned-schema, runtime-migration, FK capability, migration apply/history and schema-manager implementation modules are retained. R3 additionally changes the shared planner's PostgreSQL timestamp type comparison. Main's released timestamp, projection, count and portable-group changes are retained. Macro source is byte-identical to main.
  • Original A and unrelated worktrees remain untouched. This merge reconciliation is in an isolated worktree, with the reviewed head retained on a preservation ref. Commit uses [skip ci]; Actions are paused and hosted checks for this head are not run.

The legacy MigrationStep remains a real enum, with variant/glob imports, qualified constructors, exhaustive matches, inferred values, struct literals and function pointers supported. Static/owned storage borrows into one validation, hash, diff, risk, rendering and application engine. No replacement IR or SQL engine is added.

Owned validation/planning is read-only. apply_owned_migration enforces explicit ownership, backend/target/plan integrity, destructive/additive guards and a pinned transactional baseline recheck. DDL/history commit together. Existing PostgreSQL RLS flags, policies, owners/grants, helper functions and unrelated system objects are preserved; owned apply does not reconcile an empty RLS target. SQLite rebuilds fail closed for preservation cases the physical model cannot retain, including relevant unsupported incoming FK actions/update semantics/deferral from system or unowned sources. Rechecks occur inside apply. Unsupported live index/default semantics remain structured diagnostics.

The approved legacy DateTime default correction is unchanged: static legacy epoch-second DateTime defaults reject conversion; Integer epoch-second defaults remain supported; canonical runtime DateTime defaults remain canonical text/native timestamps. The new cross-slice test converts released repository host-managed Integer timestamps (including a physical updated_at alias), preserves explicit and implicit defaults/hash, applies the static target, then proves owned no-op replanning and environment certification on both owned backends.

Public host example

Compiled example: crates/graphql-orm/examples/runtime_owned_migration.rs (owned SQLite memory database, preview by default, --apply commits and replans to no-op). PostgreSQL execution and composed system/journal dependencies are covered by runtime_migrations.

let target = schema.physical_schema::<SqliteBackend>(RuntimeMigrationLimits::default())?
    .with_static_entities(&[HostJournal::metadata()])?;
let ownership = ManagedTableSet::new(["notes".to_owned(), "host_journal".to_owned()])?;
let plan = db.schema().plan_owned_migration(
    "schema-42", "reviewed target", &target, &ownership, PlanOptions::strict(),
).await?;
// Separate host review of exact statements, risks and hashes precedes apply.
db.schema().apply_owned_migration(&plan, ApplyOptions {
    additive_only: true,
    expected_current_schema_hash: Some(plan.source_schema_hash().to_owned()),
    ..Default::default()
}).await?;

R3 correction

The minimal PostgreSQL owned DateTime collection reproduced the review failure before the fix: successful apply followed by a Risky timestamp with time zone → TIMESTAMPTZ alteration. The shared static/owned planner now equates those unmodified PostgreSQL built-in spellings. Physical metadata, hashes, defaults and storage semantics are unchanged. Timestamp without time zone, precision modifiers, arrays and qualified/quoted names remain distinct; this is a bounded alias comparison, not arbitrary type normalization.

Executable SQLite/PostgreSQL tests cover required and nullable DateTime fields, no default and CurrentTimestamp, minimal and composed static/runtime targets, plan/apply/no-op and mutation-environment certification. A pure static-planner regression checks both alias directions, unchanged spelling-sensitive hashes and genuine type changes. No supported DateTime fields are rejected to avoid the failure.

R3 verification at f2665ce68ed7fdbccb272cb8a783b6379ae5fd82:

  • SQLite migration/schema/history/guards: 101 passed, 0 ignored. Released timestamp/UI/projection/count/group regressions: 25 passed, 0 ignored.
  • PostgreSQL combined migration/schema/history and timestamp/projection/count/group lane: 108 passed, 0 ignored. Actual disposable PostgreSQL execution includes required/nullable DateTime no-op/certification, composed targets, host Integer timestamps, FK rechecks, RLS and rollback.
  • External dependency-minimal repository consumers: 2 passed per backend; legacy enum namespace compatibility: 1 passed per SQLite/PostgreSQL/MSSQL lane. All external consumer examples compile per backend.
  • Macros: 9 unit tests per backend, six existing documentation examples ignored per backend. MSSQL unsupported runtime migration rejection: 1 passed before I/O; no live MSSQL execution.
  • Strict patch-level source compatibility against released 0.39.0: 223 passed, 30 inapplicable checks skipped; no source break.
  • Scoped warnings-denied Clippy and Rustdoc on SQLite/PostgreSQL/MSSQL, explicit backend checks/dependency trees (including SQLite+MSSQL), formatting, documentation (192 files), inventory, dependency architecture, package/semver/release-state/manifest policy checks: passed.
  • The SQLite host example compiled and executed --apply, then verified no-op replanning and certification. Release-manifest tests: 6 passed; router notice tests: 13 passed.

R3 reran the focused commands below, plus cargo check/trees for each backend and SQLite+MSSQL. The SQLite migration command includes the new DateTime cases; the PostgreSQL combined command includes them too. Every command uses CARGO_BUILD_JOBS=2, disk scratch and sequential backend builds. PostgreSQL tests verify test-owned container identity before cleanup. The earlier full SQLite/all-target run below was at the preceding head and is not represented as rerun for R3. No new public API, schema migration, automatic DDL or hash-format change is required.

R3 fixes the shared planner only; it does not certify B or implement B–D. #112 is still open at 01d4134da18028ec58e318e0ffd30ab244d0d8e1; owner merge/release of #112 should precede A’s final reconciliation against its actual merged revision. Hosted checks for the R3 head are not run while Actions is paused. Both merge and publication remain owner gates.

Previous reconciliation verification

The previous reconciliation checks below passed at 75765e71610dbc9a4fb6de6cceeda5b6d7a590f1. These earlier broad results are retained as historical evidence; R3 reruns are reported separately. No merge or release has occurred. Hosted checks for this head are not run because Actions is paused.

Executed results:

  • Full SQLite package suite: 428 passed, zero failed, 11 ignored, counting child-fixture result blocks. Ignored cases are the existing large pagination benchmark and ten documentation examples; no ignored PostgreSQL tests are counted as executed. Backend coexistence, federation end-to-end, cross-crate relationships, ordinary SDL and dependency-minimal repository fixtures executed successfully.

  • SQLite migration/schema/history/guards lane: 99 passed, zero ignored.

  • SQLite released timestamp, UI, projection, count and portable-group regression lane: 25 passed, zero ignored.

  • PostgreSQL migration/schema/history plus released timestamp/projection/count/group regression lane: 106 passed, zero ignored; actual disposable database execution, including system RLS and incoming FK preservation.

  • External repository consumer: SQLite 2 passed plus all examples compiled; PostgreSQL 2 executed timestamp/event-enumeration tests. It has no direct async-graphql dependency.

  • Legacy migration namespace external fixture: 1 passed on each SQLite/PostgreSQL/MSSQL feature lane (source/compile evidence, no database).

  • Macros: 9 unit tests passed per backend, six existing documentation examples ignored per lane.

  • MSSQL runtime migration capability: 1 passed, proves rejection before pool acquisition; no live MSSQL execution.

  • Canonical owned runners: SQLite 87 passed, PostgreSQL 69 passed, zero ignored in both. PostgreSQL includes the plain repository aggregate/field-denial external consumer.

  • Warnings-denied Clippy across all SQLite targets, plus scoped PostgreSQL/MSSQL Clippy and all three Rustdoc lanes: passed. Explicit SQLite/PostgreSQL/MSSQL/SQLite+MSSQL cargo check and dependency trees, plus workspace duplicate inspection: passed.

  • Formatting, governed documentation (192 files), inventory, dependency architecture, package release policy, ordinary semver gate and release-state/manifest checks: passed. Manifest tests 6 passed, router notice checks 13 passed.

  • Strict source compatibility against released 0.39.0 with --release-type patch: 223 passed, 30 inapplicable checks skipped, no source break. The external enum namespace fixture separately proves Rust variant/glob imports, matches, inferred values, literals and function pointers.

  • Owned SQLite host example: compiled/executed, committed two statements, verified no-op replan and dependency certification.

Reproducible focused commands:

CARGO_BUILD_JOBS=2 scripts/run-owned-database-lanes.sh sqlite
CARGO_BUILD_JOBS=2 scripts/run-owned-database-lanes.sh postgres
cargo test -p graphql-orm --locked --no-default-features --features sqlite --lib \
  --test runtime_migrations --test runtime_schema_ir --test runtime_migration_memory \
  --test migration_planner --test migration_apply --test schema_policy --test legacy_migration_history \
  -- --test-threads=1
cargo test -p graphql-orm --locked --no-default-features --features sqlite \
  --test host_timestamps --test host_timestamp_ui --test portable_group_pages \
  --test projection_visibility --test repository_counts -- --test-threads=1
cargo test -p graphql-orm --locked --no-default-features --features postgres --lib \
  --test runtime_migrations --test runtime_schema_ir --test runtime_migration_memory \
  --test migration_planner --test migration_apply --test legacy_migration_history \
  --test host_timestamps --test portable_group_pages --test projection_visibility \
  --test repository_counts -- --include-ignored --test-threads=1
cargo test --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml \
  --locked --no-default-features --features postgres --test host_timestamps --test portable_groups \
  -- --include-ignored --test-threads=1
cargo run -p graphql-orm --locked --no-default-features --features sqlite \
  --example runtime_owned_migration -- --apply
for backend in sqlite postgres mssql; do
  cargo test --manifest-path crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.toml \
    --locked --no-default-features --features "$backend"
  cargo test -p graphql-orm-macros --locked --no-default-features --features "$backend"
  RUSTDOCFLAGS=-Dwarnings cargo doc -p graphql-orm -p graphql-orm-macros \
    --locked --no-default-features --features "$backend" --no-deps
done
cargo test -p graphql-orm --locked --no-default-features --features mssql \
  --test runtime_migrations_unsupported

Additional verification commands (all passed):

cargo test -p graphql-orm --locked --no-default-features --features sqlite
cargo clippy -p graphql-orm -p graphql-orm-macros --locked --no-default-features \
  --features sqlite --all-targets -- -D warnings
cargo clippy -p graphql-orm -p graphql-orm-macros --locked --no-default-features \
  --features postgres --lib --test runtime_migrations --test runtime_schema_ir \
  --test runtime_migration_memory --test migration_planner --test migration_apply \
  --test legacy_migration_history --example runtime_owned_migration -- -D warnings
cargo clippy -p graphql-orm -p graphql-orm-macros --locked --no-default-features \
  --features mssql --lib --test runtime_migrations_unsupported \
  --example runtime_owned_migration -- -D warnings
for backend in sqlite postgres mssql sqlite,mssql; do
  cargo check -p graphql-orm -p graphql-orm-macros --locked --no-default-features --features "$backend"
  cargo tree -p graphql-orm --locked --no-default-features --features "$backend"
done
cargo tree --duplicates --workspace --locked
cargo semver-checks -p graphql-orm --manifest-path crates/graphql-orm/Cargo.toml \
  --baseline-rev df474c761c4057cb643e27a6582dd0376f56e31f --default-features --release-type patch
cargo fmt --all -- --check
cargo fmt --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml -- --check
python3 scripts/check-documentation.py --base df474c761c4057cb643e27a6582dd0376f56e31f
python3 scripts/generate-workspace-inventory.py --check
scripts/check-workspace-dependencies.sh
scripts/check-package-release-policy.sh df474c761c4057cb643e27a6582dd0376f56e31f
scripts/check-semver.sh df474c761c4057cb643e27a6582dd0376f56e31f
python3 scripts/check-release-state.py
scripts/check-release-manifest.sh
python3 scripts/test-release-manifest.py
python3 scripts/test-router-notices.py
python3 scripts/check-pr-documentation-impact.py
# PR_BODY supplied from this description for the final documentation-impact gate.

Existing CI/release definitions now select the owned PostgreSQL migration suite, MSSQL pre-I/O rejection and legacy external imports for future authorized runs. YAML parsing and shell syntax checks passed. Both added commits contain [skip ci]; the GitHub run listing for this exact head is empty. No workflow was dispatched or enabled.

The new cross-slice test initially had fixture API/trait-bound compile errors, corrected before these successful runs. Its hash-equivalence fixture now explicitly declares the same id ASC default-order spelling on both sides, as required by the existing metadata-sensitive hash contract; no production planner change was made. Existing MSSQL external host-timestamp example has an unused-import warning; it was compiled without a warnings-denied claim. Broader non-SQLite all-target gating in #89 is outside this change.

Every build uses CARGO_BUILD_JOBS=2, Rust 1.97.1, disk-backed scratch/targets, and sequential backend lanes. SQLite resources are test-owned; PostgreSQL fixtures own labelled loopback containers and verify immutable ownership before cleanup. Ambient application databases are not used. MSSQL evidence is compilation/rendering or pre-I/O capability rejection; live MSSQL execution was not run.

Compatibility and limitations

No static API, ordinary GraphQL SDL, cursor format or host timestamp write/default behavior changes. Runtime conversion still has its documented representability limits; no implicit stored-data migration is introduced. Unrepresentable live FK semantics reject even no-op certification. Arbitrary external DDL requires a host fence/fresh environment; ORM fingerprints are separate from the host's complete public/policy revision. Existing static transaction cancellation remains unchanged: catching an inner mutation timeout then returning Ok does not guarantee rollback. B provides the stronger runtime operation/hook guard separately.

Independent string predicate/binding work is in #112 (ORM/macros 0.39.1) and is not folded into A. These candidates both currently base released 0.39.0. If #112 merges first, reconcile A again with that reviewed commit and rerun affected combined checks before release; owner merge/release of #112 first is the requested order. Neither branch may move package versions backward or reuse a published source identity. B–D remain approved proposals with unimplemented interfaces, gated on exact committed, reviewed predecessors. No catalog, policy language, transport, SDK, durable replay, product/sibling changes, merge or release publication is implied. Broader PostgreSQL/MSSQL all-target gating remains tracked separately in #89.

Digibase handoff

After owner review/merge/release, pin the immutable release identity/full source revision with aligned ORM/macros and default-features = false, selecting sqlite or postgres. Validate the owned RuntimeSchema, compose trusted system entities, provide the host's persisted managed table set (including intentionally removed tables), plan/review, then explicitly apply with the source-hash and risk guards. Replan to no-op and establish the environment after verified application, under the host's external-DDL fence. Preserve host RLS and validate policy-only revisions separately before future B writes. Run downstream SQLite/PostgreSQL contracts before adoption. A alone does not enable runtime mutations or GraphQL registration. No release identity is assigned while owner gates and Actions pause remain in force.

Documentation impact

  • Documentation updated
  • No documentation impact

Canonical runtime-migration reference, compiled example, active A–D checkpoint, changelog/migration notes, aligned README/manifests/locks and generated package inventory. Accepted ADRs are unchanged.

Toby Martin added 5 commits September 30, 2026 23:28
…n-targets-20260930

# Conflicts:
#	CHANGELOG.md
#	Cargo.lock
#	Cargo.toml
#	MIGRATION.md
#	crates/graphql-orm-macros/Cargo.toml
#	crates/graphql-orm-macros/README.md
#	crates/graphql-orm/Cargo.toml
#	crates/graphql-orm/README.md
#	crates/graphql-orm/tests/fixtures/backend-coexistence/Cargo.lock
#	crates/graphql-orm/tests/fixtures/federation-end-to-end/Cargo.lock
#	crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.lock
#	crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.lock
#	docs/reference/workspace-packages.md
@Dastari
Dastari marked this pull request as ready for review October 1, 2026 02:04
@Dastari
Dastari changed the base branch from main to fix/policy-aware-projections-20261001 October 1, 2026 07:34
@Dastari
Dastari changed the base branch from fix/policy-aware-projections-20261001 to main October 1, 2026 07:56
@Dastari

Dastari commented Oct 2, 2026

Copy link
Copy Markdown
Owner Author

Version coordination: independent repository host timestamp PR #109 is open at efa2cd9, based directly on main 809d894. It proposes aligned ORM/macros 0.37.0, leaving this PR's 0.36.0 untouched. It neither depends on nor implements A–D. Timestamp fields remain Integer with unchanged physical/default metadata; approved DateTime rejection semantics are unchanged. SQLite/PostgreSQL no-op regressions passed.

Before merging the later branch, reconcile versions/inventory/locks and rerun combined owned backend lanes after rebase, preserving both independent scopes. No workspace release identity was assigned or release published for #109. Broader target-gating (#89) and pre-existing expired documentation review dates (#108) remain separate.

@Dastari

Dastari commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Version coordination for the accepted independent RepositoryEntity host-timestamp capability: #109 now incorporates reviewed main 3bff96b90ce19efa2bfc9cc71d3c95f45cad8c39 (published ORM/macros 0.38.0) at 75361295392d090d8de65192f2be27fc1905ad0f, with aligned ORM/macros 0.39.0, root/fixture locks and generated inventory. #108 was resolved independently through merged documentation review #110. No #97 source or worktree is incorporated or changed. If #97 follows this merge, reconcile its former 0.36.0 reservation with actual main and regenerate locks/inventory under its separate acceptance review; keep B gated on corrected/reviewed A. No timestamp release identity is reserved or published.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant