Skip to content

test(perf): add independent native SQLite benchmarking and profiles - #2152

Open
n30nex wants to merge 9 commits into
Kpa-clawbot:masterfrom
n30nex:codex/sqlite-performance
Open

n30nex wants to merge 9 commits into
Kpa-clawbot:masterfrom
n30nex:codex/sqlite-performance

Conversation

@n30nex

@n30nex n30nex commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #2150; related to #2146.

The standalone SQLite harness now qualifies upstream v3.15.0 (7fa86fe1) with separately pinned baseline, candidate and harness revisions. Upstream was merged normally, preserving review history; production code matches that revision.

The manifest covers all 22 application tables and 211 selected columns, including observer rowid and client_node_names. Added tests cover the legacy 18/21-table layouts and shifting receipt timestamps while preserving repeater clocks. Existing resource, data-parity, durability, endpoint-coverage and no-publish/no-deploy gates remain intact.

Local validation on 28f91906:

  • All 56 Python controls, executable workflow/release protections and standalone frontend suites passed.
  • Native Go 1.27.2 build, generator, endpoint and profiler controls passed.
  • A 100-transmission/300-observation startup preserved every table digest and passed integrity, orphan-row and WAL/FULL/foreign-key checks.
  • The baseline regression fails at 163341aa and passes at 28f91906.

Fresh Linux S qualification and normal CI passed on this head. Post-push Chromium passed 148 checks with three explicit fixture-dependent skips. The small A/A run validates the controller; it is not a speed comparison. No operator/customizer changes or larger measurements are included.

Historical evidence retains its original revisions: October 20 qualification, its CI, original qualification, retention comparison, and optional-backend comparison. These are not relabelled as v3.15.0 measurements.

@n30nex
n30nex marked this pull request as ready for review October 9, 2026 17:16
@n30nex
n30nex marked this pull request as draft October 9, 2026 17:51
@n30nex

n30nex commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Benchmark coverage corrected (commit 1c5eb9d)

  1. The failed B run returned 240 empty upstream path responses; it remains excluded from performance conclusions.
  2. Paths now use a deterministic repeater with retained canonical routes under both observation ordering cases. Reach/RX targets, data rows, rates and validation remain unchanged.
  3. Native S/B regressions reproduced missing coverage and pass after the fix. All 41 Python controls, native overlay controls and complete Go suites pass locally.

Fresh Linux qualification is running before the retention comparison is retried.

@n30nex
n30nex marked this pull request as ready for review October 9, 2026 21:43
@n30nex

n30nex commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Review/update plan after checking current upstream:

This remains useful as the independent measurement tool for #2153 and #2145. It also carries an older copy of #2160 (9d800035), so the rebase needs to drop that copy and preserve the current deployment workflow, authoritative test runner and configuration tests.

The next useful gate is a fresh native SQLite smoke qualification with baseline, candidate and harness revisions pinned separately. Retain complete logical-data parity, equivalent cache and endpoint coverage, zero-drop/error checks, resource accounting and durability checks. Diagnostic/profile runs should stay separate from timing results, and benchmark-only workflows must retain their no-publish/no-deploy guards.

The comparison markers currently in the body identify historical revisions. They should be updated only when a new comparison is actually requested and run. The successful qualification and comparisons remain valid for their recorded revisions; the excluded failed benchmark run is not evidence for a speed claim.

I would land this tool-only change separately, then use it to measure the focused retention change and a narrower storage candidate against contemporary upstream. No new benchmark or rebase was performed during this discussion review.

Reconstruct f29c443, 1c5eb9d and 4de4194 on b9593f7. Drop the duplicate 9d80003 changes now superseded by upstream Kpa-clawbot#2160; preserve current workflows, test inventory and configuration tests. The harness sources and portable controls remain unchanged.
@n30nex

n30nex commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

I’m rebasing the harness onto current upstream b9593f71 and dropping its duplicate #2160 commit. The small automatic qualification will compare the current application with itself using the rebased harness; the historical comparisons remain separately labelled.

I’m preserving the old 4de41941 harness revision in the fork before updating this branch with an explicit lease. This update does not request a large primary or diagnostic run. I’ll post the new head and qualification results when available.

@n30nex
n30nex force-pushed the codex/sqlite-performance branch from 4de4194 to 6a924cc Compare October 20, 2026 13:37
@n30nex

n30nex commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

Review feedback addressed (commit 6a924cc2)

  1. Rebuilt the harness on upstream b9593f71, dropping the duplicated 9d800035 test fix and preserving the current workflow and test inventory.

  2. Pinned automatic smoke qualification to contemporary upstream, keeping explicit baseline, candidate and harness revisions independent.

  3. Added complete current-schema coverage for all 21 application tables, with guards for unexpected table/column changes and regression coverage for legacy 18-table data, quoted identifiers, row mutations, observer layout adoption and index-only changes.

  4. All 55 Python controls, workflow protections and native Go overlay controls pass. A native startup fixture loaded 100 transmissions and 300 observations and preserved every application-table digest.

  5. The fresh Linux S smoke qualification passed on this head with 30,000 transmissions and 90,000 observations, all 21 tables and unchanged source databases. Its sanitized evidence records the exact three pins. It is an A/A controller qualification, not evidence of a speedup.

Normal CI has also passed on this head. Post-push Chromium validation passed 148 checks with three explicit fixture-dependent skips against the unchanged server/frontend. No primary/diagnostic benchmark was requested or run, and no new speed claim is made.

@n30nex

n30nex commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Review feedback addressed (commit 28f91906)

  1. Merged upstream v3.15.0 without rewriting the review history and refreshed the independent A/A qualification baseline to 7fa86fe1.
  2. Added coverage for all 22 current application tables and 211 manifest columns, including client-name receipt timestamps while preserving repeater clocks. Production code matches upstream.
  3. All 56 Python controls, workflow protections, standalone frontend suites and native build/startup controls pass. Post-push Chromium passed 148 checks with three explicit fixture-dependent skips.
  4. Fresh Linux S qualification and normal CI passed on this head. The artifact records the exact pins and verifies 22-table parity and unchanged source databases.

This is controller qualification with identical production code, not a new speed comparison. Historical measurements remain labelled with their original revisions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: extract a standalone native SQLite performance harness

1 participant