Maintain the violation cache per transaction by delta (opt-in) - #449
Open
stefjoosten wants to merge 1 commit into
Open
Maintain the violation cache per transaction by delta (opt-in)#449stefjoosten wants to merge 1 commit into
stefjoosten wants to merge 1 commit into
Conversation
Phase 4 of Ampersand#1684 on the framework side. For conjuncts whose compiler emitted candidate queries (deltaQueries in conjuncts.json, deltaTable in relations.json, delta tables in database.sql), the transaction close can maintain __conj_violation_cache__ by delta-scoped re-evaluation: MysqlDB records the pairs a transaction touches in the relation's delta table (inside the open DB transaction only, so an autocommit write such as lastAccess never leaves a row behind), and at close only the candidate rows of each affected conjunct are deleted and re-inserted. Conservative fallbacks keep full evaluation: a conjunct affected via a touched concept, a bulk mutation (deleteAllLinks, emptyRelation), or a relation without a candidate query. The setting transactions.deltaConjunctMaintenance knows three modes: off (default; behaviour and performance identical to before, no delta recording), shadow (delta protocol AND full evaluation; the full result stays authoritative and any difference is logged as DELTA SHADOW MISMATCH) and on. With generics from a compiler without deltaQueries the setting is a no-op: every close reports "0 delta-maintained". Measured before merging (memorybank/incremental-evaluation in the Ampersand repo): an FC5 shadow run of 1142 replay transactions showed zero mismatches and pairwise identical commit decisions on vs off; on the current population sizes the delta protocol does not win yet (FC5 4.2 vs 3.1 ms median close; RAP 47.0 vs 40.4 ms), the win has to come from database growth — which is why the default stays off and the cost gate (#1692) will route per conjunct. Composes with transactions.skipCleanConjuncts (#443): the clean-since-evaluation check is one helper that both evaluation paths honour, so a clean conjunct needs neither evaluation nor delta maintenance. Guarded by test/projects/delta-conjunct-maintenance: the booking scenario of skip-clean-conjuncts gives an identical digest under off, shadow and on, and the close's summary line proves the non-off modes ran and had nothing to maintain with the bundled compiler. The scenario and API client moved to test/shared/conjunct-parity.mjs, shared with the skip-clean-conjuncts spec. Once a compiler that emits deltaQueries is bundled, that spec is the place to guard the delta path itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 of AmpersandTarski/Ampersand#1684 on the framework side.
What this does
For conjuncts whose compiler emitted candidate queries (
deltaQueriesin conjuncts.json,deltaTablein relations.json, delta tables in database.sql), the transaction close can maintain__conj_violation_cache__by delta-scoped re-evaluation:MysqlDBrecords the pairs a transaction touches in the relation's delta table (inside the open DB transaction only), and at close only the candidate rows of each affected conjunct are deleted and re-inserted. Conservative fallbacks keep full evaluation: a conjunct affected via a touched concept, a bulk mutation, or a relation without a candidate query.New setting
transactions.deltaConjunctMaintenance:off(default — behaviour and performance identical to today),shadow(delta protocol and full evaluation; the full result stays authoritative, any difference is logged asDELTA SHADOW MISMATCH) andon. With generics from a compiler withoutdeltaQueries— every published compiler image today — the setting is a no-op: each close reports0 delta-maintained.Composes with
transactions.skipCleanConjuncts(#443): the clean-since-evaluation check is one helper that both evaluation paths honour.Measurements (before merging)
FC5 shadow run of 1142 replay transactions: zero mismatches, pairwise identical commit decisions on vs off. On current population sizes the delta protocol does not win yet (FC5 4.2 vs 3.1 ms median close; RAP 47.0 vs 40.4 ms); the win has to come from database growth, which is why the default stays off and the cost gate (Ampersand#1692) will route per conjunct. Details:
memorybank/incremental-evaluation/in the Ampersand repo.Verification
test/run-regression.sh delta-conjunct-maintenance→ PASS: the booking scenario gives an identical digest under off, shadow and on; the close's summary line proves the non-off modes ran and had nothing to maintain.test/run-regression.sh skip-clean-conjuncts→ PASS (3 skipped evaluations in the on-run). Its scenario and API client moved totest/shared/conjunct-parity.mjs, shared by both specs.deltaQueriesis bundled, the delta project's spec is the place to guard the delta path itself (noted in its README).🤖 Generated with Claude Code