docs(cache): ADR 0006 — list storage strategy - #1002
Merged
AnthonyMDev merged 4 commits intoMay 29, 2026
Conversation
Scopes the §7.1 `list_value TEXT` choice and the evidence-collection plan that resolves it. PR-009 review surfaced an asymmetry: scalar fields get typed columns; list-typed fields collapse to a single JSON blob. The benchmark cited by §7.1 (Confluence 1585152147) doesn't measure list paths — only exact-key, type+selection, single-row CRUD, and CTE-join sort. There is no published evidence that JSON-encoded `list_value` is the right choice for list-heavy workloads. Status is Proposed, not Accepted. The two options under evaluation are (1) ratify the current `list_value TEXT` shape or (2) migrate to a sibling `list_items` table. The trade-off table covers read/write cost across list lengths, type symmetry, nested-list handling (`[[Int]]`/`[[CacheReference]]`), child-reference indexing for Phase 2 `@onDelete` cascade, and implementation effort. The decision is locked before PR-012 (3.0-alpha tag) based on list-heavy benchmark scenarios added to PR-011 / PR-011a. Locking before alpha matters because §7.1's drop-and-rebuild migration only runs once per 3.0 upgrade — changing the list shape post-alpha would force a second migration on installed bases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
✅ Docs preview has no changesThe preview was not built because there were no changes. Build ID: e3b2c2cccdff5eff339021aa ✅ AI Style Review — No Changes DetectedNo MDX files were changed in this pull request. Review Log: View detailed log
|
…ndidate Adds a third option to the list-storage ADR — Option 3, in-place rows with a `position` column extending the records-table PK, recursing via `child_key_value` indirection for nested lists at depth ≥ 2. Names this as the leading candidate; the final lock still happens before PR-012 based on PR-011a perf data. Option 3 is the industry-standard adjacency-list-with-position pattern applied to a domain where depth is bounded and known at codegen time. The depth-1 case (≈99% of GraphQL lists in practice) is handled by a single SELECT against the parent record, with list-element rows clustered physically next to scalar field rows on disk thanks to WITHOUT ROWID. Nested lists recurse via the existing CacheKey indirection rather than introducing a new mechanism; heavier hierarchical-data patterns (closure table, nested set, materialized path) exist to answer queries we never ask because the executor walks a compiled-in selection set. Changes in this commit: - Title updated to enumerate the three options. - Status line names Option 3 as leading candidate. - §2 Decision: now lists three options. Option 3 includes the extended PK DDL, the `position = -1` scalar sentinel, the depth ≥ 2 recursion story, and the explicit reference to the adjacency-list pattern. - §3 Trade-offs: redone as a three-column table covering read cost, write cost, order, type symmetry, nested lists, Phase 2 cascade, and schema surface + implementation effort. - §4 Deciding evidence: adds `mutate-element-at-position-K-in-list-of-N` to isolate Option 3's UPDATE-by-position win over Option 1's full-JSON rewrite. Decision rule expanded to three branches with Option 3 as the expected outcome. - §5 Consequences: adds the sentinel-decoder cost as a negative and the leading-candidate framing as a positive. - §7 References: cites the industry pattern source (Djellouli) for the adjacency-list-with-position framing. 152 lines, still well under the 200-line budget. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-element queryability — filtering list elements by typed column at the SQL level, watcher observation of single-element changes, and the Phase 2 `@onDelete` cascade walk — is a hard capability requirement for the cache. JSON-blob storage forecloses it: any per-element operation must load and parse the entire blob in Swift, with no opportunity for SQLite's query planner to participate. This is a capability constraint, not a performance preference. No benchmark outcome rescues the JSON shape. Option 1 (was the §7.1 default) moves out of the active option space into "Alternatives considered" with that rejection reason. What remains is choosing between the two row-per-element shapes: - Option 1 (was Option 2): sibling `list_items` table. - Option 2 (was Option 3): in-place rows in `records` with an extended PK `(cache_key, field_name, position)`; depth ≥ 2 recurses via `child_key_value` indirection. Still the leading candidate. The PR-009 JSON encoder is now interim code; PR-009b replaces it before PR-012 regardless of which option wins. The benchmark-driven lock is still meaningful for the Option 1 vs Option 2 choice. Changes in this commit: - Title narrowed to the two surviving options. - §1 Context: adds the capability-requirement paragraph that motivates the rejection; flags PR-009's encoder as interim. - §2 Decision: now two options; commits to rejecting JSON outright. - §3 (new) Alternatives considered: documents the JSON rejection with the capability reason. - §4 Trade-offs: shrunk to a two-column table. Depth-1 vs depth ≥ 2 read costs broken out so Option 2's clustering win and Option 1's depth-strategy variance are both visible. - §5 Deciding evidence: decision rule reduced to two branches plus edge-case escalation. Adds `filter-list-elements-by-typed-column` scenario to lock the capability requirement into perf coverage. - §6 Consequences: adds "PR-009's encoder is now interim code" as a negative and the capability-locked framing as a positive. 160 lines, well under the 200-line budget. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Status flipped from Proposed to Accepted. The sibling `list_items` table moves out of the active option space into "Alternatives considered" alongside the JSON `list_value` rejection. The chosen layout dominates the sibling table on every dimension that matters — read locality (one table, clustered via WITHOUT ROWID), schema surface (one table, one migration target), nested-list handling (reuses existing CacheKey indirection rather than requiring a new depth strategy), and Phase 2 cascade walker source — so the benchmark-driven lock no longer carries its weight. The list-heavy benchmark scenarios still land in PR-011a, but as permanent regression coverage of the chosen design rather than as decision-gating evidence. Restructured to the canonical ADR format used by ADR 0001-0005: Context, Decision, Alternatives considered, Consequences, References. Separate Trade-offs and Deciding evidence sections collapse into the Decision section's rationale and the Alternatives considered rejections. Implementation impact documented inline: - §7.1 DDL and §7.2 operations need a follow-up doc PR to match the new schema (column changes, PK extension to (cache_key, field_name, position), DEFAULT -1 sentinel for scalar rows). - PR-009 (#1001, open) scope amended: position-aware row writers replace the JSON list-encoding branches; decoder grows a position = -1 / position >= 0 split. PR-009 review-findings hardening is retained where it applies to custom_scalar_value. - 3.0-alpha is untagged, so the §7.1 schema change carries zero migration cost — the drop-and-rebuild path on first 3.0 launch carries this ADR's schema directly. - Execution plan §8 needs a follow-up to reflect the amended PR-009 scope. 120 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
AnthonyMDev
added a commit
that referenced
this pull request
May 29, 2026
ADR 0006 (#1002) ratified the in-place row-per-element layout with a `position` column extending the records-table PK. This commit propagates that decision into the engineering plan and execution plan. cache-rewrite-phase1-plan.md changes: - §7.1 DDL: removes `list_value TEXT`; adds `position INTEGER NOT NULL DEFAULT -1`; PK becomes `(cache_key, field_name, position)`. Adds a brief preamble explaining the row-per-element semantics and citing ADR 0006. Updates `child_key_value` comment to acknowledge its dual use for nested-list sub-record indirection. Schema version bumped from 3 to 4. - §7.2 Operations: `selectRecords` now orders by `(cache_key, field_name, position)` and the decoder branches on `position` to dispatch scalar vs list-element rows. `addOrUpdate` produces one row per scalar field and N rows per list-typed field; list-element rows for a field are rewritten atomically. `deleteRecord` handles the depth ≥ 2 cascading reachability walk for nested-list sub-records. - §7.3 Migration: trigger updated from "< 3" to "< 4". Adds a sentence noting that the trigger absorbs both 2.x upgrades and any local-dev databases on v3; v3 never tagged externally. cache-rewrite-phase1-execution.md changes: - §8 PR-009 entry: title updated to "feat(sqlite): row-per-element CRUD with position-keyed schema"; tests-required rewritten to cover position-keyed CRUD, nested-list recursion at depth ≥ 2, atomic list-element rewrite, and the v3 → v4 schema version bump. Adds a note that this PR supersedes the prior "JSON list_value" scope but retains the row-per-field harness landed in the original implementation. LoC estimate bumped from ~600 to ~700, which crosses execution plan §6 trigger 1's 600-LoC threshold. Reviewer awareness is flagged in the PR description; the amended scope is genuinely larger than the original because it covers the schema update, the nested-list recursion, and the atomic-rewrite contract. If the actual diff materially exceeds this estimate, PR-009 should escalate per §6 trigger 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ADR 0006 ratifies the on-disk shape for list-typed cache fields: each list element becomes its own row in the
recordstable, addressed by an extended primary key with apositioncolumn. Scalars use the sentinelposition = -1; list elements useposition = 0..N-1. Nested lists ([[Int]],[[CacheReference]]) recurse via the existingCacheKeyindirection — depth ≥ 2 is the only path that incurs an extraSELECT. Industry-standard adjacency-list-with-position, applied to a domain where depth is bounded and known at codegen time.Status: Accepted.
The §7.1 DDL (with
list_value TEXT) is replaced. Thelist_valuecolumn is removed;PRIMARY KEY (cache_key, field_name)becomesPRIMARY KEY (cache_key, field_name, position). The new shape is in §2 of the ADR.Why JSON
list_valueis rejectedThe cache must support per-element queries against list-typed fields — filtering and indexing list elements at the SQL level, watcher observation of single-element changes, and walking list elements during the Phase 2
@onDeletecascade. JSON-blob storage forecloses all of these (every per-element operation requires loading and parsing the entire blob in Swift). This is a capability constraint, not a performance preference; no benchmark outcome rescues the JSON shape.Why the sibling
list_itemstable is rejectedThe chosen in-place layout dominates the sibling table on every dimension that matters — read locality (clustered with parent via
WITHOUT ROWID), schema surface (one table vs. two), nested-list handling (reuses existingCacheKeyindirection vs. requires a new depth strategy), Phase 2 cascade walker source (one table vs. two). There is no scenario where the second table produces a capability or performance advantage the chosen layout lacks. Both rejected alternatives sit under "Alternatives considered."Implementation impact
SQLiteFieldEncoding.swiftare replaced with position-aware row writers; the decoder grows aposition = -1/position >= 0split. The PR-009 review-findings hardening (NSNull,$reference,sortedKeys,JSONSerialization.isValidJSONObject) is retained where it applies tocustom_scalar_value.Performance coverage
The list-heavy benchmark scenarios discussed during this ADR's drafting (
read-record-with-list-of-N,write-list-of-N,mutate-element-at-position-K-in-list-of-N, nested[[T]]reads, andfilter-list-elements-by-typed-column) still land in PR-011 / PR-011a — but as permanent regression coverage of the chosen design, not as decision-gating evidence. The 3.0-alpha tag's existing gating (SQLite gates + noregressedverdict in the published dataset) absorbs them without any new lock criterion.120 lines. Follow-up to #1001 (PR-009). Stacks directly on
cache-rewrite/phase-1-plan, not on the PR-009 stack — this is an out-of-stack governance/docs PR per the "Supporting PRs" pattern.Revision history
b056afded— initial draft with two options (JSONlist_valuevs siblinglist_itemstable), decision deferred to PR-011a perf data.272cd70bb— added in-place rows withpositionas a third option (leading candidate); revised trade-off table to three columns; addedmutate-element-at-position-Kscenario; expanded decision rule to three branches.e16bbd844— JSONlist_valuerejected on capability grounds (per-element queryability) and moved to "Alternatives considered"; remaining options renumbered (sibling table = new Option 1; in-place position = new Option 2, still leading candidate); trade-off table shrunk to two columns; decision rule reduced to two branches.89774cfd1— ratified. Status flipped to Accepted; sibling table moved into "Alternatives considered" alongside the JSON rejection; restructured to the canonical ADR format used by ADR 0001-0005; Trade-offs and Deciding evidence sections collapsed into the Decision rationale and Alternatives rejections. Implementation impact documented inline.Test plan
🤖 Generated with Claude Code