Skip to content

feat(sqlite): row-per-element CRUD against position-keyed schema (PR-009) - #1001

Open
AnthonyMDev wants to merge 5 commits into
cache-rewrite/phase-1-planfrom
cache-rewrite/phase-1a-row-per-field-crud
Open

feat(sqlite): row-per-element CRUD against position-keyed schema (PR-009)#1001
AnthonyMDev wants to merge 5 commits into
cache-rewrite/phase-1-planfrom
cache-rewrite/phase-1a-row-per-field-crud

Conversation

@AnthonyMDev

@AnthonyMDev AnthonyMDev commented May 27, 2026

Copy link
Copy Markdown
Contributor

Goal

Land the amended PR-009 scope per ADR 0006: row-per-element write + delete CRUD against the position-keyed schema introduced in PR-008b (#1005). The JSON list_value path is gone; nested lists recurse via synthetic sub-records at <parent>.<field>.$[<position>].

Cascading deletion of synthetic sub-records is split into a follow-up PR with extensive cascade-correctness tests. This PR establishes the foundation; the cascade PR ships the cleanup walks.

What's in this PR

Public API on SQLiteDatabase

Method Behavior
insertOrUpdate(records: [Record]) For each field: scalar → one row at position = -1; list → N rows at positions 0..N-1. Each write atomically clears the field's prior rows before inserting the new ones, so list rewrites and scalar↔list transitions never leave in-field orphans.
deleteRecord(forKey:) Direct DELETE WHERE cache_key = ?. Does not cascade synthetic sub-records (see below).
deleteRecords(matchingKey:) LIKE … COLLATE NOCASE ESCAPE '\\' with \, %, _ in the user-supplied pattern escaped so literal substrings don't act as wildcards. Does not cascade synthetic sub-records.

Synthetic sub-records for nested lists

Per ADR 0006 + the design conversation:

  • Outer-list rows for a list-of-list field hold child_key_value = "<parent>.<field>.$[N]" pointing at a synthetic sub-record.
  • The sub-record's element rows live under field_name = "$" (sentinel — $ can't start a GraphQL Name, so no collision with real fields).
  • Deeper nesting ([[[T]]]) appends .$[N] per level: User:1.matrix3d.$[0].$[0].$[0] is the level-3 sub-record's key.
  • Detection regex: \.\$\[[0-9]+\]$.

Encoder (SQLiteFieldEncoding)

Per-value, never per-array. Lists are exploded into rows at the database layer.

Swift value Column
NSNumber (boolean via CFGetTypeID == CFBooleanGetTypeID) bool_value
NSNumber (integer kinds via CFNumberGetType) int_value
NSNumber (other numeric kinds) float_value
String string_value
CacheReference child_key_value (stores .key)
NSNull and any other JSON-shaped value custom_scalar_value (.sortedKeys + .fragmentsAllowed JSON)
[Record.Value] precondition violation — caller must explode it

The custom-scalar fallback gates on JSONSerialization.isValidJSONObject([value]) before encoding so an unsupported type surfaces a Swift error rather than an NSException abort. The $reference dict-wrapper is recognized only when the dict has exactly one key, so a legitimate custom scalar that happens to include $reference alongside other keys round-trips intact.

Test-only read path

A non-public internal selectRecords(forKeys:) reassembles records from rows and resolves synthetic child_key_value pointers to materialize nested lists. Used by the test suite to verify what insertOrUpdate wrote. The production read path is a projection-aware API to be introduced in PR-009b–h per ADR 0007 — not on the public protocol.

Tests

SQLiteRowPerElementCRUDTests.swift (new file) — 32 tests, all passing:

  • Per-type round-trip ×6 (string, int, double, bool ×2, cache reference)
  • Lists: scalars, cache references, mixed scalars, empty
  • Nested lists: 2D, 3D, list-of-list-of-references
  • Multi-field record preserves all fields including writtenAt
  • UPSERT: writing twice overwrites; changing value type clears prior typed column
  • Atomic list rewrite shrinks length cleanly (no orphan rows for the field itself)
  • Delete by exact key: removes only matching rows; non-existent key is no-op
  • Delete by pattern: matches prefixes, empty pattern is no-op, _ and % escaped
  • Transactional rollback: scalar encoding failure, list-element encoding failure
  • Custom-scalar hardening: NSNull top level, generic dict, $reference with extra keys treated as generic dict
  • NSNumber bool/int/double routing keeps each value's type identity
  • Empty record produces no rows (documented behavior)

What's deferred to the follow-up cascade PR

Three places where synthetic sub-record orphans can accumulate under this PR's behavior:

  1. deleteRecord(forKey:) — deletes the record's own rows but leaves any synthetic sub-records the record's nested-list fields pointed at. They become unreachable but persistent.
  2. insertOrUpdate atomic-rewrite over a nested-list field — clears the field's own rows but not the synthetic sub-records the prior rows pointed at. They orphan.
  3. deleteRecords(matchingKey:) + synthetic sub-records — the flat LIKE filter may or may not match synthetic sub-records depending on the pattern. Behavior is currently ambiguous; the cascade PR pins it down.

The cascade PR will:

  • Add cascadeDeleteFromCacheKey and cascadeDeleteFromField CTEs (and a LIKE-pattern variant for deleteRecords(matchingKey:)).
  • Wire them into all three call sites above.
  • Add a test-only helper that bypasses the synthetic-key filter in selectRecords so orphan detection actually verifies the database state (the previous cascade tests on this PR's branch passed regardless of cascade behavior because selectRecords filtered synthetic keys before the assertions ran — a real test bug surfaced during review).
  • Ship 8+ cascade-correctness tests:
    • depth-1 / depth-2 / depth-3 cascade
    • cascade isolation: deleting record A doesn't disturb record B's synthetic sub-records
    • multiple list-typed fields on one record → each field's synthetic sub-records all cascade
    • mixed list contents (synthetic + real CacheReference) — cascade follows synthetic only
    • real CacheReference inside a synthetic sub-record — that ref's target NOT cascaded
    • atomic-rewrite cascade for nested → scalar transition
    • atomic-rewrite cascade for nested → differently-typed nested
    • deleteRecords(matchingKey:) + synthetic interaction (with a pinned-down behavior)

The deferred cascade and its tests don't gate this PR's correctness for the operations it ships — selectRecords never returns orphan synthetic sub-records as top-level records, so the read surface is unaffected. The cascade is about storage hygiene + the eventual clearDatabase cleanliness contract.

Acceptance criteria

  • Encoder + CRUD + tests compile.
  • All 32 new tests pass.
  • Full Apollo-UnitTestPlan is green.
  • Zero navigator errors.
  • selectRecords is not on the public protocol — only an internal helper.
  • Doc comments on insertOrUpdate and deleteRecord(forKey:) clearly call out the deferred cascade.

Stacks on

cache-rewrite/phase-1-plan at e06a2b859 (the PR-008b merge commit).

Followup

  1. (next) Cascading-delete PR — adds the cascade walks + extensive cascade-correctness tests.
  2. Per ADR 0007's sub-phase 1A.5: PR-009b (FieldProjection types), PR-009c–h.

@apollo-librarian

apollo-librarian Bot commented May 27, 2026

Copy link
Copy Markdown

✅ Docs preview ready

The preview is ready to be viewed. View the preview

File Changes

0 new, 1 changed, 0 removed
* (developer-tools)/ios/(latest)/tutorial/tutorial-define-additional-mutations.mdx

Build ID: 53b673f6969547f6647e2feb
Build Logs: View logs

URL: https://www.apollographql.com/docs/deploy-preview/53b673f6969547f6647e2feb


✅ AI Style Review — No Changes Detected

No MDX files were changed in this pull request.

Review Log: View detailed log

This review is AI-generated. Please use common sense when accepting these suggestions, as they may not always be accurate or appropriate for your specific context.

@AnthonyMDev AnthonyMDev changed the title feat(sqlite): row-per-field CRUD against new schema (PR-009) feat(sqlite): row-per-field writes + deletes + encoder (PR-009) May 28, 2026
AnthonyMDev added a commit that referenced this pull request May 28, 2026
Status flipped from Proposed to Accepted. The sibling `list_items` table
moves out of the active option space into "Alternatives considered"
alongside the JSON `list_value` rejection. The chosen layout dominates
the sibling table on every dimension that matters — read locality (one
table, clustered via WITHOUT ROWID), schema surface (one table, one
migration target), nested-list handling (reuses existing CacheKey
indirection rather than requiring a new depth strategy), and Phase 2
cascade walker source — so the benchmark-driven lock no longer carries
its weight.

The list-heavy benchmark scenarios still land in PR-011a, but as
permanent regression coverage of the chosen design rather than as
decision-gating evidence.

Restructured to the canonical ADR format used by ADR 0001-0005:
Context, Decision, Alternatives considered, Consequences, References.
Separate Trade-offs and Deciding evidence sections collapse into the
Decision section's rationale and the Alternatives considered
rejections.

Implementation impact documented inline:

- §7.1 DDL and §7.2 operations need a follow-up doc PR to match the
  new schema (column changes, PK extension to (cache_key, field_name,
  position), DEFAULT -1 sentinel for scalar rows).
- PR-009 (#1001, open) scope amended: position-aware row writers
  replace the JSON list-encoding branches; decoder grows a
  position = -1 / position >= 0 split. PR-009 review-findings hardening
  is retained where it applies to custom_scalar_value.
- 3.0-alpha is untagged, so the §7.1 schema change carries zero
  migration cost — the drop-and-rebuild path on first 3.0 launch
  carries this ADR's schema directly.
- Execution plan §8 needs a follow-up to reflect the amended PR-009
  scope.

120 lines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AnthonyMDev
AnthonyMDev force-pushed the cache-rewrite/phase-1a-row-per-field-crud branch from 87e76bf to f5f78b7 Compare May 29, 2026 19:34
@AnthonyMDev AnthonyMDev changed the title feat(sqlite): row-per-field writes + deletes + encoder (PR-009) feat(sqlite): row-per-element CRUD against position-keyed schema (PR-009) May 29, 2026
@AnthonyMDev
AnthonyMDev force-pushed the cache-rewrite/phase-1a-row-per-field-crud branch from f5f78b7 to 96e39f4 Compare June 1, 2026 18:31
cacheKey: CacheKey,
fieldName: String,
position: Int64,
column: SQLiteFieldEncoding.Column,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling this parameter column is confusing to me. What about typedValue?

/// into `Record` instances. Follows synthetic `child_key_value`
/// pointers to materialize nested lists.
private func loadRecordBatch(_ keys: Set<CacheKey>) throws -> [Record] {
let placeholders = keys.map { _ in "?" }.joined(separator: ", ")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a repeating count to do this instead of mapping? Should be faster.

Comment thread apollo-ios/Sources/ApolloSQLite/ApolloSQLiteDatabase.swift Outdated
AnthonyMDev added a commit that referenced this pull request Jun 1, 2026
…aceholders

Addresses review comments on PR #1001:

- Rename SQLiteFieldEncoding.Column → TypedValue, and upsertRow's
  `column:` parameter → `typedValue:`. "Column" was overloaded
  with the SQL sense (a database column); the renamed type makes
  it clearer that the enum is a Record.Value already classified
  into the destination column slot it'll be bound to.
- Replace `keys.map { _ in "?" }.joined(separator: ", ")` with
  `Array(repeating: "?", count: keys.count).joined(separator: ", ")`
  in loadRecordBatch. The bulk-fill initializer skips the per-
  element closure invocation.

Doc comments updated where they referred to "the column slot" /
"selects the column slot" to match the new name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AnthonyMDev added a commit that referenced this pull request Jun 1, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment on lines +30 to +39
private func totalRowCount(_ db: ApolloSQLiteDatabase) throws -> Int {
let dbURL = SQLiteTestCacheProvider.temporarySQLiteFileURL()
_ = dbURL // unused; rowCount runs via the internal-test select helper
// The cheapest way without exposing more internals is to load
// every known key and inspect — but we don't know all keys.
// Instead, use clearDatabase as a destructive end-of-test check
// where appropriate. For row-count assertions, the tests
// explicitly select the keys they wrote and check the returned
// structure.
fatalError("Use selectRecords to assert structure; this helper is unused")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If unused, why is this not deleted?

AnthonyMDev added a commit that referenced this pull request Jun 1, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AnthonyMDev added a commit that referenced this pull request Jun 2, 2026
…aceholders

Addresses review comments on PR #1001:

- Rename SQLiteFieldEncoding.Column → TypedValue, and upsertRow's
  `column:` parameter → `typedValue:`. "Column" was overloaded
  with the SQL sense (a database column); the renamed type makes
  it clearer that the enum is a Record.Value already classified
  into the destination column slot it'll be bound to.
- Replace `keys.map { _ in "?" }.joined(separator: ", ")` with
  `Array(repeating: "?", count: keys.count).joined(separator: ", ")`
  in loadRecordBatch. The bulk-fill initializer skips the per-
  element closure invocation.

Doc comments updated where they referred to "the column slot" /
"selects the column slot" to match the new name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AnthonyMDev
AnthonyMDev force-pushed the cache-rewrite/phase-1a-row-per-field-crud branch from ef28685 to b63c35a Compare June 2, 2026 21:02
AnthonyMDev added a commit that referenced this pull request Jun 2, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AnthonyMDev added a commit that referenced this pull request Jun 9, 2026
…aceholders

Addresses review comments on PR #1001:

- Rename SQLiteFieldEncoding.Column → TypedValue, and upsertRow's
  `column:` parameter → `typedValue:`. "Column" was overloaded
  with the SQL sense (a database column); the renamed type makes
  it clearer that the enum is a Record.Value already classified
  into the destination column slot it'll be bound to.
- Replace `keys.map { _ in "?" }.joined(separator: ", ")` with
  `Array(repeating: "?", count: keys.count).joined(separator: ", ")`
  in loadRecordBatch. The bulk-fill initializer skips the per-
  element closure invocation.

Doc comments updated where they referred to "the column slot" /
"selects the column slot" to match the new name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AnthonyMDev
AnthonyMDev force-pushed the cache-rewrite/phase-1a-row-per-field-crud branch from b63c35a to 4a05d29 Compare June 9, 2026 18:26
AnthonyMDev added a commit that referenced this pull request Jun 9, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AnthonyMDev and others added 3 commits June 29, 2026 13:32
…009)

Lays down the row-per-element write + delete CRUD against the
position-keyed schema from PR-008b (#1005). The JSON list-encoding
path is gone; each list element is a typed-column row at
`position = 0..N-1` keyed under its parent's cache_key + field_name,
with `WITHOUT ROWID` clustering keeping elements physically adjacent
to their parent's scalar rows.

Public API on `SQLiteDatabase`:

- `insertOrUpdate(records: [Record])` — for each field, scalar values
  produce one row at `position = -1`, list-typed values produce N
  rows at positions `0..N-1`. Each write atomically clears the
  field's prior rows before inserting the new ones, so list rewrites
  and scalar↔list transitions never leave in-field orphans.

- `deleteRecord(forKey:)` — direct `DELETE WHERE cache_key = ?`.

- `deleteRecords(matchingKey:)` — `LIKE … ESCAPE '\'` with `\`, `%`,
  `_` in the user-supplied pattern escaped so literal substrings
  like `User_` don't act as wildcards.

Nested-list elements (`[[Int]]`, `[[CacheReference]]`) recurse via
synthetic sub-records at `<parent>.<field>.$[<position>]`. Deeper
nesting appends `.$[<position>]` per level. Synthetic sub-records
hold their inner list's element rows under the sentinel
`field_name = "$"`. The `$` character cannot start a GraphQL Name,
so the suffix shape cannot collide with any cache key produced from
real GraphQL schemas.

**Deferred to a follow-up PR**: cascading deletion of synthetic sub-
records. The current implementation:

- `deleteRecord(forKey:)` deletes only the record's own rows; the
  record's nested-list synthetic sub-records remain in the database
  as unreachable orphans.
- `insertOrUpdate` atomic-rewrite clears the field's own rows but
  not the synthetic sub-records the prior rows pointed at; those
  also orphan.
- `deleteRecords(matchingKey:)` uses a flat `LIKE` filter and may
  or may not match synthetic sub-records depending on the pattern.

Reads are unaffected — orphans never surface through `selectRecords`
because they're unreachable from any non-synthetic cache key. The
cleanup walk lands in a follow-up PR with extensive cascade-
correctness tests (depth-1/2/3 cascade, cascade isolation across
records, cascade-doesn't-follow-real-CacheReferences, mixed-content
lists, pattern-delete + synthetic interaction).

`SQLiteFieldEncoding` is the per-value encoder/decoder. Single
(non-array) values dispatch to one of the six typed columns:
`NSNumber`-bool via `CFGetTypeID` → `bool_value`; integer NSNumber
→ `int_value`; floating NSNumber → `float_value`; `String` →
`string_value`; `CacheReference` → `child_key_value`; everything
else (incl. `NSNull` and generic dicts) JSON-encodes to
`custom_scalar_value` with `.sortedKeys + .fragmentsAllowed`. Array
inputs are precondition-violated — they belong at the database
layer's iteration, not in the encoder.

A test-only `internal selectRecords(forKeys:)` reads back what
`insertOrUpdate` wrote, follows synthetic `child_key_value`
pointers to reassemble nested lists, and filters out synthetic
sub-records from the top-level result. Not on the public protocol
— the public read path will be the projection-aware API introduced
in PR-009b–h per ADR 0007.

`SQLiteRowPerElementCRUDTests` (new): 32 tests covering per-type
round-trip, list (scalars / references / mixed / empty), 2D / 3D /
list-of-list-of-references nested lists, multi-field record, UPSERT
semantics (write-twice overwrite, scalar value-type change clearing
prior column), atomic list rewrite that shrinks length, delete by
key (matching + non-existent), delete by pattern (incl. `_` and
`%` escapes), transactional rollback (scalar + list element
encoding failures), NSNull at top level, generic dict round-trip,
`$reference` with extra keys treated as generic dict, NSNumber
bool/int/double routing, empty-record produces no rows.

Cascade-delete-specific tests are deferred to the follow-up PR
along with the implementation.

Full Apollo-UnitTestPlan: 1037 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…aceholders

Addresses review comments on PR #1001:

- Rename SQLiteFieldEncoding.Column → TypedValue, and upsertRow's
  `column:` parameter → `typedValue:`. "Column" was overloaded
  with the SQL sense (a database column); the renamed type makes
  it clearer that the enum is a Record.Value already classified
  into the destination column slot it'll be bound to.
- Replace `keys.map { _ in "?" }.joined(separator: ", ")` with
  `Array(repeating: "?", count: keys.count).joined(separator: ", ")`
  in loadRecordBatch. The bulk-fill initializer skips the per-
  element closure invocation.

Doc comments updated where they referred to "the column slot" /
"selects the column slot" to match the new name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Aligns the enum case name with the schema column name
(`SQLiteSchema.Records.floatValue` = `float_value`). SQLite's
storage type name is REAL, but the column-level name we use
throughout the codebase is `float_value`/`floatValue`; matching
the case to that name keeps the read/write code self-consistent.

Affects the case declaration in `SQLiteFieldEncoding.TypedValue`,
the `NSNumber` floating-point routing in `encode(_:)`, and the
single consumer site (the `switch typedValue` binding loop in
`ApolloSQLiteDatabase.upsertRow`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AnthonyMDev
AnthonyMDev force-pushed the cache-rewrite/phase-1a-row-per-field-crud branch from 4a05d29 to b715e5e Compare June 29, 2026 20:38
AnthonyMDev added a commit that referenced this pull request Jun 29, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AnthonyMDev and others added 2 commits July 9, 2026 10:30
Dead code — unconditionally fatalError'd and was never called; tests
assert structure via selectRecords directly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Empty lists previously wrote zero rows, making a cached `[]`
indistinguishable from a never-written field — and an empty *nested*
list left a dangling synthetic reference that read back as a raw
CacheReference. Both now round-trip: an empty list writes a single
marker row at position = -2 with no value column populated, and the
read path decodes it back to `[]` (including through synthetic
sub-record resolution).

Also lands the reserved-key audit promised by ADR 0006: insertOrUpdate
rejects cache keys containing the reserved synthetic token `.$[`
before opening the write transaction, so the synthetic-key classifiers
(regex and SQL LIKE) can never match a stored user record.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AnthonyMDev added a commit that referenced this pull request Jul 9, 2026
Adds the cascade walks deferred from PR-009 (#1001), wired into all
three places that need them, plus extensive correctness tests.

Three recursive-CTE walks share the same shape (seed → follow
`child_key_value` LIKE `%.$[%]` → repeat → DELETE everything reached):

- `cascadeDeleteSyntheticDescendants(seedCacheKeys:)` — full-record
  walk for `deleteRecord(forKey:)`.
- `cascadeDeleteSyntheticDescendantsOfField(cacheKey:fieldName:)` —
  scoped to one field, used by `insertOrUpdate`'s atomic rewrite
  before re-writing the field's element rows.
- `cascadeDeletePatternMatchedSyntheticDescendants(escapedLikePattern:)`
  — seeds from records matching the pattern, used by
  `deleteRecords(matchingKey:)` to clean up synthetic descendants of
  every matched record.

Real (non-synthetic) `CacheReference` targets are never followed —
each walk filters seed children on the synthetic-suffix `LIKE`
pattern, so a real ref to an independent record is left alone.

Test-only helper `rowCount(forCacheKey:)` is added to
`ApolloSQLiteDatabase` so cascade tests can verify orphan removal
directly against the database. The previous draft of these tests
ran assertions through `selectRecords`, which filters synthetic
keys before returning — the filter masked orphans and made the
assertions pass regardless of cascade behavior. `rowCount` bypasses
the filter via a direct `COUNT(*)` query and surfaces the real DB
state.

`SQLiteRowPerElementCascadeDeleteTests.swift` (new file) — 11 tests:

deleteRecord(forKey:) cascade:
- depth-1 nested list (`[[Int]]`) cascade
- depth-3 nested list (`[[[Int]]]`) cascade — recursive CTE must
  reach the level-3 sub-record
- cascade isolation — deleting record A doesn't affect record B's
  synthetic sub-records
- multiple list-typed fields on one record — both fields' synthetic
  sub-records cascade
- real CacheReference targets in a list are NOT cascaded
- real CacheReference INSIDE a synthetic sub-record is NOT cascaded

insertOrUpdate atomic-rewrite cascade:
- nested-list → scalar rewrite cleans synthetic sub-records
- 3D nested-list → scalar rewrite cleans all synthetic descendants
- rewriting one field doesn't disturb the OTHER field's synthetic
  sub-records on the same record

deleteRecords(matchingKey:) cascade:
- pattern-matched records' synthetic sub-records cascade
- unmatched records' synthetic sub-records survive (cascade
  isolation through the pattern boundary)

Full Apollo-UnitTestPlan: 1045 passed, 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant