Raise transactor throughput and add a Cloudflare write benchmark

[tvanhens]

Primary outcome

Per-transaction work in the transactor no longer grows with the size of unindexed novelty, operations group-commit alongside transactions, and the operation path costs one transactor request per Worker batch instead of two per operation. Every existing commit guarantee is unchanged: one ordered writer, dense t, all-or-nothing batches, receipt claim before body execution, authorization freshness checked at apply and again at commit, ack only after the durable log row, ordered subscriber frames, and the client-tx replay cache.

Transactor loop

• SortedNovelty keeps log-structured sorted runs instead of rebuilding one array on every flush. A staged transaction's post-commit view layers only its own datoms over the live novelty; the full copy remains only for transactions that define schema attributes.
• takeBatch no longer isolates an operation into a batch of one. A resolve-time budget (RAMOSE_BATCH_BUDGET_MS, default 20) bounds how long a batch runs before the commit-time freshness recheck. A repeated invocation whose receipt was claimed earlier in the same batch is deferred until after that batch commits.
• The indexer merges the four index trees concurrently and remembers the armed alarm instead of reading it on every commit.
• expandTx probes each unique value once per transaction.
• Log rows commit in multi-row inserts sized under the Durable Object parameter limit.

Measured with bench:transactor at 64 concurrent writers over fsync'd SQLite:

	before	after
Group commit tx/s	1239	~7000
One tx per write tx/s	141	499
Ack p50 (group commit)	48 ms	8.5 ms
Core tx cost at 48k unindexed datoms	4.06 ms	0.07 ms

Operation path

Catalog preparation and per-unit policy tables are memoized by descriptor identity, so admission no longer revalidates the catalog on every invocation. Receipt claims reuse the inspected row and complete with one guarded update. Catalog resolution and receipt preparation run without Effect wrappers, with the principal scope digest cached. The replay fence keeps the admission witness when an operation's datoms touch nothing that witness observed, and recomputes otherwise. bun run bench:op drives the bench catalog through the in-process transactor; locally these take operations from 3,577/s to 4,907/s.

On Cloudflare a no-op route into the transactor Durable Object tops out near 1,000 requests/s, and each operation cost two of them: a catalog provisioning call, then the invoke. The transactor now provisions a routed catalog once per object lifetime when the invoke request names its route root, and exposes /invoke-batch, which settles each entry with the same error mapping as a single invoke. Each Worker isolate groups operations for the same database that arrive within RAMOSE_OP_COALESCE_MS (default 2 ms, 0 disables) into one transactor request; every waiting request keeps its own timer so the Workers runtime never cancels it as hung.

Cloudflare benchmark

bun run bench:cf deploys a throwaway stage with test hooks, generates load from inside Cloudflare through a service binding to the stage itself, and drives the raw transaction path and the /op path with lanes × parallel concurrent writers. It reports client-side throughput and latency, the peer's warn and error telemetry captured inside the stage, and the transactor's batch counters as wall-clock cadence. BENCH_PING=1 adds a no-op request phase that measures the per-object request ceiling, and BENCH_SOCKET=1 adds ping and write phases over a WebSocket into the same object. bun run bench:cf:compare runs it against master and the current branch. Both Cloudflare scripts accept an Alchemy profile login in place of explicit credentials.

Measured at 256 concurrent writers, with the no-op info lane as the request ceiling on the same stage:

stage	raw tx/s	info req/s	ops/s
before, typical	600–890	~980	240–370
after, stage A	482	628	552
after, stage B	345	535	566

Operations now exceed both the single-request write path and the per-request ceiling on their stage, with zero errors. Stage-to-stage variance is large, so compare phases within one run.

The socket phases show where the next ceiling is. On one stage: 667 no-op requests/s with overload errors against 2,004 no-op frames/s with none, and 534 raw writes/s over requests against 878 over frames. The production write protocol is unchanged here; the direction is tracked in #642.

Public API additions

None. Two new environment variables, RAMOSE_BATCH_BUDGET_MS and RAMOSE_OP_COALESCE_MS, forwarded through alchemy.run.ts.

Test lanes

• Unit: test/internal/core/novelty-runs.test.ts covers sorted runs, the overlay view, and validator visibility; test/worker/operation-coalescer.test.ts covers batching, the size cap, and failure propagation; operations-runtime.test.ts gains clean and dirty replay fence cases. Typecheck, unit, browser, and build pass.
• Local: not run in the authoring environment (no Docker).
• Cloudflare: the e2e suite passes on a live stage, and the benchmark numbers above come from live stages.

🤖 Generated with Claude Code

[github-actions]

Reef preview: https://ramose-reef-api-pr-641-ftjn6v6zlcs2sfzw.tvanhens.workers.dev

Deployed from 253d4dd as stage pr-641. Torn down when this PR closes.