Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions docs/release-database-fence-prototype.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Inactive database-fence prototype: not a deployable fix

Reference SQL lives outside active migrations. It extends actual content tables
with per-write operation/owner/epoch stamps and an authoritative admission row.
The current runtime does not emit stamps; this schema intentionally rejects its
legacy writes rather than silently allowing an unfenced fallback. Reads retain
old rows. No hosted schema or permissions changed.

The promotion primitive is a single admitted history INSERT. Its trigger advances
the exact head and consumes admission inside the same SQL statement/transaction.
Failure at consumption rolls everything back. This avoids a detached history
INSERT falsely claiming publication. Stop or owner changes before the transaction
reject; a stop after committed promotion does not undo history. Response loss must
be reconciled by operation ID/event/head, not by replaying insert.

All five existing publication tables reject DELETE. Records, provenance and
history reject UPDATE. Revision payload identity is immutable; only completion
0→1 is permitted under preparation admission. Duplicate natural-key or hidden-rowid INSERT is
rejected before conflict handling, preventing INSERT OR REPLACE from deleting old
rows even when recursive_triggers is off. These stricter semantics mean current
ON CONFLICT DO NOTHING callers must be rewritten to read/reconcile first.

Stamps are integrity bindings, not secrets/authentication. Any actor with arbitrary
D1/DDL access can edit admission or remove triggers. Runtime admission provisioning
must remain behind verified actor/operation authority; it is not implemented here.
The prototype does not promise protection from database administrators or malicious
code possessing the entire binding. SQL checks also do not replace canonical bundle
hash validation, full record integrity, semantic review or artifact provenance.

## Remaining implementation before a deployable PR

- Explicit stamped statement construction for every revision/record/provenance
write in stagePublication, stagePublisherBatch, resumeSeedBatch,
stageBundledRevision/importBundledRevision/ensureRevision and bootstrap.
- Replace two-statement activation with admitted event primitive, preserving
complete canonical payload checks and idempotent event readback.
- Authenticate admission creation/state transitions; bind expected source/tree,
environment, exact candidate, run/attempt and reviewed budget. No caller bool.
- Test bootstrap absent-head, rollback history rules, supported D1 trigger
behavior, all insert/update/UPSERT variants, stage caps, stop during each phase,
concurrent actors and crash/readback on the real hosted isolated database.

## Migration/rollback strategy to review separately

Schema is additive but write compatibility changes. Keep publication off; qualify
existing immutable data and backup/rollback evidence; apply stamped schema only
with a compatible runtime ready. Old data remains readable; old unstamped binaries
must fail writes. A runtime-only rollback does not remove the fence or reverse a
migration. Safe rollback preserves rows/history and keeps writes disabled until a
separately reviewed compatible repair. Do not automatically drop triggers/tables
or restore unfenced legacy writes to make a rollback look successful.

This prototype is local SQLite evidence, not hosted D1, CI integration, automatic
publishing or authority approval. Existing enable denial remains in force.

The SQL additionally requires new revisions incomplete, exact event operation stamp and publish/rollback history eligibility. Canonical payload hashes and exact rows still require the trusted active validator; matching digest labels alone do not prove bytes. An existing rowid=-1 causes future inserts to hold because SQLite does not distinguish its implicit NEW.rowid sentinel from explicit -1 in BEFORE INSERT; it is not silently allowed.

## Current inactive adapter scope

`fenced-preparation-reference.mjs` now implements one bounded preparation call:
mandatory explicit operation/owner/epoch, maximum 50 inserted rows per call,
maximum 1,000-row candidate, canonical bundle validation, exact existing payload
and row reconciliation, and atomic stamped provenance/completion. It does not
provision or admit itself, expose an HTTP route, change a head, or authenticate
provenance supplied by its trusted future caller. The production caller must
acquire original independently verified repository/run/attempt/artifact evidence;
well-shaped JSON and the reference admission table are not that proof.

Both permitted UPDATE triggers preserve rowid, closing UPDATE OR REPLACE's
implicit-deletion path as well as the INSERT OR REPLACE regression. Promotion
still uses the single-event SQL primitive. No active core function has been
replaced and no hosted migration has been registered.

## Mandatory writer integration inventory

| Existing entry | Mutations requiring stamped integration or retirement |
| --- | --- |
| content-store stageBundledRevision / private importBundledRevision | revision, record batches, completion |
| content-store ensureRevision | bundled import plus initial pointer |
| content-store resumeSeedBatch | revision, at most 50 rows, completion |
| publication-core stagePublication | revision, batches, provenance/completion |
| publication-core activatePublication | replace old event+pointer batch with single admitted event |
| publisher resumePublisherSeed | delegates seed resume |
| publisher stagePublisherBatch | delegates resume, then separately inserts provenance |
| publisher bootstrapPublication | delegates full staging, then initial pointer/event |

HTTP stage, stage-batch, resume-seed, bootstrap, activate routes reach these
writers. Maintenance-token and OIDC authorities differ; the dispatch-only limits
must remain intact. This inventory is source evidence, not proof that no older
binary or privileged SQL actor exists. No visitor-read initialization exception
may silently bypass the fence.

## Required authenticated admission contract (design, not implemented)

A separately authorized issuer must bind repository/source/tree, original artifact
bytes and run/attempt provenance, target/engine/environment, expected head and
generation, budget, operation ID and monotonic owner epoch. It must retain an
append-only decision history before installing a channel's current pointer.
Preparing to admitted requires full exact candidate/record verification and fresh
control/head checks. The writer cannot grant itself admission. Stop is sticky;
owner transfer, target replacement and recovery require a new explicit decision,
not resetting the old stopped bit. Unknown outcomes reconcile the same operation;
no fresh operation is minted merely to retry. A consumed admission establishes
only the atomic database transition, never public delivery verification.

Deployment requires qualifying all five tables, constraints, indexes and triggers
on actual isolated D1, with interruption at each migration/runtime rollout step.
Admission table protection cannot be achieved against unrestricted D1-binding
code by readable stamps. The chosen execution authority must prevent ordinary
writers from arbitrary SQL/DDL/admission mutation, or explicitly exclude that
privileged actor from the threat model. Such authority and migration decisions
remain separate approvals, not consequences of this inactive reference patch.

Local coverage now includes a 105-record candidate prepared in 50/50/5 calls,
and stop/epoch changes after successful preflight but immediately before the
next actual batch; both reject atomically and preserve the first 50 rows. SQL
readback uses LIMIT recordCount+1. This is deterministic transaction-boundary
injection in SQLite, not simultaneous hosted D1 clients or a production race test.
84 changes: 84 additions & 0 deletions tests/publication-core.test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -266,3 +266,87 @@ test('read-only preflight CLI rejects malformed arguments before accessing artif
const result = require('node:child_process').spawnSync(process.execPath, [path.join(__dirname, '../scripts/verify-publication.mjs'), '--commit', 'main'], { encoding: 'utf8' });
assert.equal(result.status, 1); assert.match(result.stderr, /Usage:/); assert.equal(result.stdout, '');
});

async function fenceFixture(t,stopped=0){
const fixture=await prepared(t),{db,a,b}=fixture;
db.sql.exec(fs.readFileSync('tools/release-coordination/database-fence-reference.sql','utf8'));
db.sql.prepare('INSERT INTO release_admission VALUES(?,?,?,?,?,?,?,?,?,?,?,?)').run('engine:'+engine,'pub-'+ '9'.repeat(32),'owner-a',1,b.manifest.revision,b.manifest.digest,commit,a.manifest.revision,0,'publish','admitted',stopped);
return fixture;
}
function insertFencedEvent(db,a,b,owner='owner-a',epoch=1){db.sql.prepare('INSERT INTO content_publication_events(operation_id,channel,generation,from_revision,to_revision,provenance_commit,intent,created_at,writer_operation,writer_owner,writer_epoch) VALUES(?,?,?,?,?,?,?,?,?,?,?)').run('pub-'+ '9'.repeat(32),'engine:'+engine,1,a.manifest.revision,b.manifest.revision,commit,'publish','synthetic', 'pub-'+ '9'.repeat(32),owner,epoch)}
test('reference event statement atomically promotes and consumes admission',async t=>{const {db,a,b,m}=await fenceFixture(t);insertFencedEvent(db,a,b);assert.equal((await m.readPublicationHead(db,engine)).revision,b.manifest.revision);assert.equal(db.sql.prepare('SELECT state FROM release_admission').get().state,'consumed');assert.throws(()=>insertFencedEvent(db,a,b),/fence-/)});
for(const wrong of ['stopped','owner','epoch'])test('reference blocks '+wrong+' at actual SQL mutation',async t=>{const {db,a,b,m}=await fenceFixture(t,wrong==='stopped'?1:0);assert.throws(()=>insertFencedEvent(db,a,b,wrong==='owner'?'owner-b':'owner-a',wrong==='epoch'?2:1),/fence-/);assert.equal((await m.readPublicationHead(db,engine)).revision,a.manifest.revision);assert.equal(db.sql.prepare('SELECT COUNT(*) AS n FROM content_publication_events').get().n,0)});
test('reference failure after head write rolls back event head and consumption',async t=>{const {db,a,b,m}=await fenceFixture(t);db.sql.exec("CREATE TRIGGER injected_failure BEFORE UPDATE ON release_admission WHEN NEW.state='consumed' BEGIN SELECT RAISE(ABORT,'injected'); END");assert.throws(()=>insertFencedEvent(db,a,b),/injected/);assert.equal((await m.readPublicationHead(db,engine)).revision,a.manifest.revision);assert.equal(db.sql.prepare('SELECT COUNT(*) AS n FROM content_publication_events').get().n,0);assert.equal(db.sql.prepare('SELECT state FROM release_admission').get().state,'admitted')});
test('reference denies legacy actual activation and direct old-row deletion/replacement',async t=>{const {db,a,b,m}=await fenceFixture(t);await assert.rejects(m.activatePublication(db,action(1,a,b),{engineRevision:engine}),/fence-/);for(const table of ['content_revisions','content_records','content_publication_provenance','content_publication_events','content_publication']){if(table==='content_publication_events')continue;assert.throws(()=>db.sql.exec('DELETE FROM '+table),/fence-/);assert.throws(()=>db.sql.exec('INSERT OR REPLACE INTO '+table+' SELECT * FROM '+table+' LIMIT 1'),/fence-/)}assert.equal((await m.readPublicationHead(db,engine)).revision,a.manifest.revision)});
test('reference old rows cannot be rewritten or replaced, including history and head',async t=>{const {db,a,b}=await fenceFixture(t);insertFencedEvent(db,a,b);for(const table of ['content_revisions','content_records','content_publication_provenance','content_publication_events','content_publication']){assert.throws(()=>db.sql.exec('DELETE FROM '+table),/fence-/);assert.throws(()=>db.sql.exec('INSERT OR REPLACE INTO '+table+' SELECT * FROM '+table+' LIMIT 1'),/fence-/);assert.throws(()=>db.sql.exec("UPDATE "+table+" SET writer_owner='owner-b'"),/fence-/)}});
test('reference wrong target and stopped transition cannot consume admission',async t=>{const {db,a,b,c,m}=await fenceFixture(t);assert.throws(()=>insertFencedEvent(db,a,c),/fence-/);db.sql.exec('UPDATE release_admission SET stopped=1');assert.throws(()=>insertFencedEvent(db,a,b),/fence-/);assert.equal(db.sql.prepare('SELECT state FROM release_admission').get().state,'admitted');assert.equal((await m.readPublicationHead(db,engine)).revision,a.manifest.revision)});
test('reference rejects hidden rowid REPLACE collision with new natural key and valid stamps',async t=>{const {db,b}=await fenceFixture(t);db.sql.exec("UPDATE release_admission SET state='preparing'");const old=db.sql.prepare('SELECT rowid AS rid,* FROM content_records LIMIT 1').get();db.sql.exec('PRAGMA recursive_triggers=OFF');assert.throws(()=>db.sql.prepare('INSERT OR REPLACE INTO content_records(rowid,revision_id,record_id,kind,dataset,payload_json,writer_operation,writer_owner,writer_epoch) VALUES(?,?,?,?,?,?,?,?,?)').run(old.rid,b.manifest.revision,'new-record','claim','platform','{}','pub-'+'9'.repeat(32),'owner-a',1),/fence-/);assert.equal(db.sql.prepare('SELECT record_id FROM content_records WHERE rowid=?').get(old.rid).record_id,old.record_id)});
test('reference rejects UPDATE OR REPLACE hidden rowid collision on valid completion',async t=>{
const {db,a}=await fenceFixture(t); const fresh=await bundle(99), m=fresh.manifest, op='pub-'+'9'.repeat(32);
db.sql.exec('PRAGMA recursive_triggers=OFF');
db.sql.prepare("UPDATE release_admission SET state='preparing',target_revision=?,target_digest=?").run(m.revision,m.digest);
db.sql.prepare('INSERT INTO content_revisions(id,engine,schema_version,digest,manifest_json,bundle_json,record_count,completed,writer_operation,writer_owner,writer_epoch) VALUES(?,?,?,?,?,?,?,0,?,?,?)').run(m.revision,engine,1,m.digest,JSON.stringify(m),JSON.stringify(fresh),m.recordCount,op,'owner-a',1);
const old=db.sql.prepare('SELECT rowid AS rid FROM content_revisions WHERE id=?').get(a.manifest.revision);
assert.throws(()=>db.sql.prepare('UPDATE OR REPLACE content_revisions SET rowid=?,completed=1,writer_operation=?,writer_owner=?,writer_epoch=? WHERE id=?').run(old.rid,op,'owner-a',1,m.revision),/fence-/);
assert.equal(db.sql.prepare('SELECT id FROM content_revisions WHERE rowid=?').get(old.rid).id,a.manifest.revision);
assert.equal(db.sql.prepare('SELECT completed FROM content_revisions WHERE id=?').get(m.revision).completed,0);
});
test('inactive stamped preparation validates bytes, completes atomically and reconciles repeats',async t=>{
const {db}=await fenceFixture(t), fresh=await bundle(99), m=fresh.manifest;
db.sql.prepare("UPDATE release_admission SET state='preparing',target_revision=?,target_digest=?").run(m.revision,m.digest);
const {prepareFencedBatch}=await import('../tools/release-coordination/fenced-preparation-reference.mjs');
const identity={operationId:'pub-'+'9'.repeat(32),owner:'owner-a',epoch:1};
const provenance={repository:(await core()).REPOSITORY,commit,workflowRun:7};
const result=await prepareFencedBatch(db,fresh,identity,provenance);
assert.equal(result.status,'prepared');assert.equal(result.count,2);
assert.equal((await prepareFencedBatch(db,fresh,identity,provenance)).writes,0);
const corrupt=clone(fresh);corrupt.catalog.records[1].payload.number=777;
await assert.rejects(prepareFencedBatch(db,corrupt,identity,provenance));
db.sql.exec('UPDATE release_admission SET stopped=1');
await assert.rejects(prepareFencedBatch(db,fresh,identity,provenance),/admission mismatch/);
});
test('stamped preparation refuses ambiguous batch result and reconciles committed rows on next call',async t=>{
const {db}=await fenceFixture(t), fresh=await bundle(100), m=fresh.manifest;
db.sql.prepare("UPDATE release_admission SET state='preparing',target_revision=?,target_digest=?").run(m.revision,m.digest);
const {prepareFencedBatch}=await import('../tools/release-coordination/fenced-preparation-reference.mjs');
const identity={operationId:'pub-'+'9'.repeat(32),owner:'owner-a',epoch:1};
const provenance={repository:(await core()).REPOSITORY,commit,workflowRun:7};
const realBatch=db.batch.bind(db);db.batch=async statements=>{await realBatch(statements);return undefined};
await assert.rejects(prepareFencedBatch(db,fresh,identity,provenance),/outcome uncertain/);
db.batch=realBatch;
const recovered=await prepareFencedBatch(db,fresh,identity,provenance);assert.equal(recovered.status,'prepared');assert.equal(recovered.writes,0);
});
test('stamped preparation rolls back provenance when completion fails',async t=>{
const {db}=await fenceFixture(t), fresh=await bundle(101), m=fresh.manifest;
db.sql.prepare("UPDATE release_admission SET state='preparing',target_revision=?,target_digest=?").run(m.revision,m.digest);
db.sql.exec("CREATE TRIGGER reject_completion BEFORE UPDATE ON content_revisions WHEN NEW.completed=1 BEGIN SELECT RAISE(ABORT,'injected-completion'); END");
const {prepareFencedBatch}=await import('../tools/release-coordination/fenced-preparation-reference.mjs');
await assert.rejects(prepareFencedBatch(db,fresh,{operationId:'pub-'+'9'.repeat(32),owner:'owner-a',epoch:1},{repository:(await core()).REPOSITORY,commit,workflowRun:7}),/injected-completion/);
assert.equal(db.sql.prepare('SELECT completed FROM content_revisions WHERE id=?').get(m.revision).completed,0);
assert.equal(db.sql.prepare('SELECT COUNT(*) AS n FROM content_publication_provenance WHERE revision_id=?').get(m.revision).n,0);
});
async function largeFencedFixture(t) {
const fixture=await fenceFixture(t), fresh=await bundle(201), m=await core();
const template=fresh.catalog.records[1];
fresh.catalog.records=[fresh.catalog.records[0],...Array.from({length:104},(_,i)=>({...clone(template),id:'vc-'+i.toString(16).padStart(20,'0')}))];
fresh.manifest.recordCount=fresh.catalog.records.length;
fresh.manifest.digest=await m.sha256({schemaVersion:1,globals:fresh.globals,catalog:fresh.catalog});
fresh.manifest.revision='content-'+(await m.sha256({digest:fresh.manifest.digest,engineRevision:engine,datasetHashes:fresh.manifest.datasetHashes})).slice(0,24);
fixture.db.sql.prepare("UPDATE release_admission SET state='preparing',target_revision=?,target_digest=?").run(fresh.manifest.revision,fresh.manifest.digest);
return {...fixture,fresh,identity:{operationId:'pub-'+'9'.repeat(32),owner:'owner-a',epoch:1},provenance:{repository:m.REPOSITORY,commit,workflowRun:7},prepare:(await import('../tools/release-coordination/fenced-preparation-reference.mjs')).prepareFencedBatch};
}
test('stamped preparation needs three bounded calls for 105 records',async t=>{
const {db,fresh,identity,provenance,prepare}=await largeFencedFixture(t);
const results=[];for(let i=0;i<3;i++)results.push(await prepare(db,fresh,identity,provenance));
assert.deepEqual(results.map(r=>r.count),[50,100,105]);assert.deepEqual(results.map(r=>r.writes),[50,50,5]);
assert.deepEqual(results.map(r=>r.status),['preparing','preparing','prepared']);
assert.equal(db.sql.prepare('SELECT state FROM release_admission').get().state,'preparing');
});
for(const race of ['stop','epoch'])test('stamped batch atomically rejects '+race+' after successful preflight',async t=>{
const {db,fresh,identity,provenance,prepare}=await largeFencedFixture(t);
await prepare(db,fresh,identity,provenance);
db.beforeNextBatch(()=>db.sql.exec(race==='stop'?'UPDATE release_admission SET stopped=1':'UPDATE release_admission SET epoch=2'));
await assert.rejects(prepare(db,fresh,identity,provenance),/fence-/);
assert.equal(db.sql.prepare('SELECT COUNT(*) AS n FROM content_records WHERE revision_id=?').get(fresh.manifest.revision).n,50);
assert.equal(db.sql.prepare('SELECT completed FROM content_revisions WHERE id=?').get(fresh.manifest.revision).completed,0);
});
Loading
Loading