Eliminate 15,400 zero products from the complete q-Toeplitz replay (no κ)

research/q-toeplitz-zero-products/BENCHMARK.json

@@ -0,0 +1,93 @@
+{
+  "cache_policy": "-B and one fresh private bytecode namespace for both sides; no result cache",
+  "commands": {
+    "candidate": [
+      "{python}",
+      "-W",
+      "error",
+      "-B",
+      "research/q-toeplitz-zero-products/replay.py"
+    ],
+    "reference": [
+      "{python}",
+      "-W",
+      "error",
+      "-B",
+      "research/q-toeplitz-factorization/verify.py"
+    ]
+  },
+  "errors": 0,
+  "files_sha256": {
+    "SOURCES.json": "38ad84f66c83309e7b8026743da726b485fd946776a385ca8f7dbd125be86505",
+    "benchmark.py": "0b2fc011781a233e3f028686e47b712a3895a22fce72be3c0e50115151318652",
+    "kernel.py": "027c2b36af33f815832dcbf3aa21ab86534e32e5a0aa62a93a658b89ffce0d4b",
+    "replay.py": "290b86e0f9f714b7b24a4b8aaeabd911e82f54bbcb4d290ef7162ee9757c234c",
+    "verify.py": "712ac913ddd03fa3138ed4fd4dcff5b2c9509ebfe908199a661ee7e71fa1d1b8"
+  },
+  "platform": "macOS-27.0.1-arm64-arm-64bit",
+  "python": "/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12",
+  "rounds_per_side": 15,
+  "samples_seconds": {
+    "candidate": [
+      0.059804582968354225,
+      0.0606410838663578,
+      0.06102329189889133,
+      0.0580701248254627,
+      0.05884737498126924,
+      0.05794099997729063,
+      0.059726875042542815,
+      0.057848958065733314,
+      0.06589725008234382,
+      0.06919579184614122,
+      0.06005770806223154,
+      0.05729866703040898,
+      0.0582977500744164,
+      0.05817358405329287,
+      0.05767741706222296
+    ],
+    "reference": [
+      0.0674856249243021,
+      0.06571291689760983,
+      0.0695430829655379,
+      0.06346737500280142,
+      0.06429062504321337,
+      0.06382962502539158,
+      0.09916904103010893,
+      0.06311595905572176,
+      0.06551783299073577,
+      0.0824238748755306,
+      0.06682491698302329,
+      0.062343291006982327,
+      0.06302720913663507,
+      0.06343324994668365,
+      0.06253870786167681
+    ]
+  },
+  "summary": {
+    "candidate": {
+      "median_seconds": 0.05884737498126924,
+      "range_seconds": [
+        0.05729866703040898,
+        0.06919579184614122
+      ]
+    },
+    "reference": {
+      "median_seconds": 0.06429062504321337,
+      "range_seconds": [
+        0.062343291006982327,
+        0.09916904103010893
+      ]
+    }
+  },
+  "timed_region": "fresh interpreter, all imports, all 30 matrices, every retained check and stdout",
+  "version": "3.12.15",
+  "work": {
+    "dense_products": 22050,
+    "entries": 4550,
+    "multiplications": 300,
+    "nonzero_products": 6650,
+    "skipped_products": 15400,
+    "stdout": "PASS_Q_TOEPLITZ_FACTORIZATION matrices=30 no_new_kappa\n",
+    "transcript_sha256": "65582bc026421c9031681b874e3067e456bc74c39e71a6b717e6a1c3a4f39e82"
+  }
+}

research/q-toeplitz-zero-products/README.md

@@ -0,0 +1,154 @@
+# Omit 15,400 exact zero products from the complete q-Toeplitz replay
+
+This is a finite certificate of an implementation improvement. The complete
+fixed replay from James Reed's [PR3865](https://github.com/CrocSwap/integer-mult-bounds/pull/3865)
+uses 300 rational matrix multiplications. This opt-in implementation reduces
+their scalar multiplications from **22,050 to 6,650**, while comparing every
+one of their **4,550 output entries** with the original implementation.
+All thirty matrices and every original factor, Gaussian, inverse, width,
+norm and final receipt check remain. There is no new factorization or kappa.
+
+The predecessor's files and default command remain unchanged. [replay.py](replay.py)
+loads the authenticated original code and replaces only its `multiply`
+function with [kernel.py](kernel.py). No parameter, order, check or expected
+result changes. Sparse matrix multiplication and exact zero annihilation are
+classical. The contribution is their application to this complete fixed replay,
+its exact operation reduction, conformance evidence and measured runtime.
+
+## Why the omission is exact
+
+For each entry, the original algorithm sums `left[i][k] * right[k][j]` in
+increasing k order. The replacement starts every entry at rational zero and
+visits k in the same order. It skips a term only when one rational operand
+equals zero. Every nonzero summand therefore has the same value and order.
+All entries, including the zeros, are materialized. No numerical threshold,
+floating arithmetic, probabilistic check, rank surrogate or result cache is used.
+
+The original command runs ten multiplications at each order n=1,...,6 for
+each of five q values. Its dense count is `5*10*sum(n^3)=22050`. The count of
+nonzero operand pairs per q and n is
+
+```text
+binom(n+2,3) + 5*binom(n+1,2) + 2*sum(k^2, k=1..n) + n^2 + n^3.
+```
+
+The terms come from the lower-by-lower product, five diagonal/triangular
+products, two products of opposite triangular matrices, one diagonal/dense
+product and the final dense product. Summing gives 6,650. The checker also
+enumerates every actual operand pair, rather than trusting this expression.
+The new matrix identities are implementation-equivalence identities; the
+predecessor's factorization theorem is a cited input, not a new audit result.
+
+## Evidence and timing
+
+[verify.py](verify.py) checks all 300 complete products and generates a receipt
+binding the nine package files by SHA256. The operand/output transcript hash is
+`65582bc026421c9031681b874e3067e456bc74c39e71a6b717e6a1c3a4f39e82`.
+It also checks all 6,561 ordered pairs of two-by-two matrices with entries
+in `{-1/3,0,2/5}`, dense fixtures at orders one through six and an exact
+nonzero `2^-2048` product. Four invalid matrix shapes reject. Three actual
+source-byte corruptions reject before execution. All-zero and reversed-column
+replacement kernels fail retained checks. Explicit exceptions keep acceptance
+and rejection checks active under optimized Python.
+
+The diagnostic baseline profile attributed 0.038 of 0.048 seconds to matrix
+multiplication; the candidate profile attributed 0.014 of 0.025 seconds to it.
+Profilers are outside the benchmark. Rational arithmetic is the changed cost;
+startup, imports, source authentication and all other checks remain.
+
+[BENCHMARK.json](BENCHMARK.json) contains fifteen alternating fresh-process
+pairs on macOS arm64, CPython 3.12.15. Complete-command medians are **64.291 ms
+and 58.847 ms**, an 8.47% median reduction. Ranges are 62.343 to 99.169 ms and
+57.299 to 69.196 ms. Fourteen of fifteen paired candidate runs are faster; the
+median paired reduction is 5.397 ms. All thirty processes finish successfully
+with byte-identical stdout and no stderr. This is a descriptive host measurement,
+not a portable speed guarantee or a CI timing threshold. The machine has eighteen
+cores; orientation load averages were 3.16, 2.01 and 1.73. Concurrent work and
+interpreter startup limit timing precision. The exact 15,400-product reduction
+does not depend on these timing samples.
+
+Each timed region includes interpreter startup, imports, all thirty matrices,
+every retained check and stdout. Both sides use a fresh shared private bytecode
+namespace with writes disabled, so both compile their standard-library imports.
+The candidate additionally authenticates the source bytes. No scientific result
+or matrix is reused across processes. Raw samples and implementation hashes are
+generated by [benchmark.py](benchmark.py).
+
+## Reproduce
+
+From the repository root with CPython 3.12 and the standard library:
+
+```sh
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -W error -B research/q-toeplitz-zero-products/replay.py
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -W error -B research/q-toeplitz-zero-products/verify.py
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/q-toeplitz-zero-products
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/q-toeplitz-zero-products --check-source-history
+```
+
+Any installed CPython 3.12 may replace that interpreter path. The registered
+manifest runs both the complete replay and conformance checks in normal and
+optimized modes. Expected markers include `PASS_Q_TOEPLITZ_FACTORIZATION`,
+`PASS_Q_TOEPLITZ_ZERO_PRODUCTS` and `PASS_REGISTERED_RESEARCH_PACKAGE`.
+The normal command writes no artifacts. Set `TMPDIR` to a scratch directory
+when directing temporary corruption-control copies to a particular location.
+
+To reproduce timings, choose a fresh output file outside the checkout:
+
+```sh
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -W error -B research/q-toeplitz-zero-products/benchmark.py --rounds 15 --output /tmp/q-toeplitz-benchmark.json
+```
+
+The benchmark refuses output reuse and propagates failures. It is optional,
+because elapsed time is not a mathematical acceptance condition.
+
+## Scope and engineering constraints
+
+Canonical base is `7c49fe32c55de02c2c873655c804b16e4c3fef0a`, priority revision 8.
+[SOURCES.json](SOURCES.json) pins the original code, receipt and README as live
+inputs; the selected record and catalogue are historical provenance. Source
+hashes must match before execution. The code compiles those authenticated bytes
+directly, so a predecessor bytecode cache cannot substitute different code.
+The ordinary import environment for this package and the Python standard library
+remains trusted; the registered runner separately isolates bytecode imports.
+
+This is the fixed PR3865 exact rational suite, not the production Gaussian core,
+a native multiplier or a new negative mathematical result. Orders beyond six,
+new q domains, physical time savings, all-size proofs, supplier admission and
+the five selected framework assumptions remain outside the claim. No selected
+cold replay, Lean, Docker, independent human review or proof-kernel review is
+claimed. The selected frontier remains unchanged.
+
+The autonomous local run playbook stopped after the first arithmetic improvement.
+Build the Lever kept the replacement and conformance checker; Prove It Works
+required complete intermediate matrices rather than just the final marker.
+The benchmark checklist required complete fresh-process timing and error counts.
+Power of Ten bounds matrix loops by six, fixture enumeration by 81 squared,
+source inventory by five rows, live source reads by 128 KiB each and benchmark
+rounds by 31, with thirty-second subprocess timeouts. New functions fit within
+sixty lines and use explicit failure paths.
+
+Python-managed rational and container allocation is a rule-3 deviation, bounded
+by the fixed suite and fixture sizes. Rule-8/9 deviations are loading one fixed
+authenticated Python module and replacing one explicit function; two deliberate
+fault kernels occur only in controls. No source rewriting or arbitrary module
+selection is exposed. A compiled implementation would be needed for a static
+allocation policy. Warning-strict normal and optimized registered commands
+provide recurring diagnostics. No production configuration changes are made.
+
+## Open next questions
+
+Does exact zero-product omission help a larger currently used rational replay
+with comparably sparse factors? That would give greater practical benefit than
+this short command. First profile one different complete replay and enumerate
+its zero operands. No larger workload or equivalence contract is supplied here.
+This package stops at its first improvement and reserves no follow-up.
+
+## Credit
+
+This contribution is produced by Nick Petalas. James Reed, jwr404, supplied
+PR3865's complete finite replay, rational fixtures and recorded result.
+Chafik Boukhalfa recorded the factorization in PR489, citing Alexey Kuznetsov;
+Jose Manuel, jmbarrancoml, retained its production context in PR3851. The
+underlying factorization and sparse multiplication methods retain their credit.
+No predecessor file or credit line is changed. New work is Apache-2.0.
+Independent human and formal proof-kernel review are unclaimed.

research/q-toeplitz-zero-products/SOURCES.json

@@ -0,0 +1,36 @@
+{
+  "schema_version": 1,
+  "base_commit": "7c49fe32c55de02c2c873655c804b16e4c3fef0a",
+  "files": [
+    {
+      "path": "certificates/selected-result.json",
+      "usage": "provenance",
+      "git_blob": "d59b5035e7b4cbaa9d688a1d32dd0d6eb79c466a",
+      "sha256": "9b8d0eea95b3bf0dfcb7938a732a6439b87a2685dec2a5ab9e525b3b34debbfa"
+    },
+    {
+      "path": "docs/research/agent-task-catalog.json",
+      "usage": "provenance",
+      "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec",
+      "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81"
+    },
+    {
+      "path": "research/q-toeplitz-factorization/README.md",
+      "usage": "input",
+      "git_blob": "7c33c3e220b0365cb77d107d3346a4bf057824c4",
+      "sha256": "17d085f10ceacc7c6f737e6982690e7f1e63088cab6080304650ea60a25ef22f"
+    },
+    {
+      "path": "research/q-toeplitz-factorization/RESULT.json",
+      "usage": "input",
+      "git_blob": "f542f21d58f9ed3b2001c821993b3e6dec1de525",
+      "sha256": "b9134a4b111bd0af30b5c6e429ca264ace9bbef17648a1f278d89d7e34949ada"
+    },
+    {
+      "path": "research/q-toeplitz-factorization/verify.py",
+      "usage": "input",
+      "git_blob": "d1b79772b35d47ab6ff0bd4006cace3d9e3aee49",
+      "sha256": "4c083eec7f5a87274ec1cc211375a27642696d82f5e228ccaf2791a05d857adc"
+    }
+  ]
+}

research/q-toeplitz-zero-products/TASK.json

@@ -0,0 +1,61 @@
+{
+  "schema_version": 1,
+  "task_id": "replay-runtime",
+  "priority_revision": 8,
+  "base_commit": "7c49fe32c55de02c2c873655c804b16e4c3fef0a",
+  "original_card": {
+    "id": "replay-runtime",
+    "question": "Can one measured replay bottleneck be sped up without dropping coverage?",
+    "starting_points": [
+      "scripts/run_isolated_tests.py",
+      "scripts/verify_replay_coverage.py",
+      "certificates/selected-result.json"
+    ],
+    "methods": [
+      "Profile one isolated replay and optimize the measured bottleneck.",
+      "Design source-bound reuse with explicit dependency invalidation and negative controls."
+    ],
+    "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.",
+    "first_step": {
+      "command": [
+        "python3",
+        "-B",
+        "scripts/research_selected_baseline.py"
+      ],
+      "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
+      "inspect": "Resolve the current selected review, separate complex/ordinary BIT/admitted BIT/outer budgets, and compare the card against live handoffs before starting. Historical ceilings require exact source/model equivalence.",
+      "scope": "Source-binding orientation only; not a scientific replay or all-size proof. Materialize selected inputs if this checkout is sparse."
+    },
+    "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.",
+    "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
+    "allowed_variations": [
+      "Profile one stage or design one source-bound cache boundary."
+    ],
+    "scope_change_triggers": [
+      "Skipping coverage or comparing different fixtures to claim speedup."
+    ],
+    "overlap_terms": [
+      "replay runtime",
+      "cache invalidation",
+      "isolated tests"
+    ]
+  },
+  "scope_status": "refined",
+  "actual_scope": "Preserve the complete fixed PR3865 q-Toeplitz replay while omitting exact zero scalar products in its rational matrix multiplication.",
+  "scope_change_reason": "The replay-runtime card is narrowed to the 30-matrix PR3865 command, including all 300 complete matrix multiplications and every original check.",
+  "bottleneck_link": "This is verification tooling. It reduces exact replay arithmetic without claiming a new mathematical factorization, native improvement or kappa.",
+  "budget_minutes": 60,
+  "stop_rule": "Stop at the first complete-output-preserving arithmetic reduction and measured complete-command improvement; package it without extending the matrix domain.",
+  "outcome": "completed",
+  "evidence": "The complete conformance stream compares all 4550 entries of 300 matrix products. Exact multiplication count falls from 22050 to 6650, eliminating 15400 zero products. All original checks remain. BENCHMARK.json records 15 alternating fresh-process pairs with zero errors.",
+  "author": "Nick Petalas",
+  "ai_assistance": "Independent human and formal proof-kernel review are unclaimed.",
+  "open_questions": [
+    {
+      "question": "Does this exact zero-product omission improve a larger currently used rational replay with comparably sparse factors?",
+      "why_it_matters": "The present standalone replay is short, so interpreter startup limits its practical gain. A larger existing replay may benefit more.",
+      "first_step": "Profile one different complete replay and enumerate its zero products before changing it.",
+      "blocked_by": "No larger workload or unmodified output-equivalence contract is established by this package."
+    }
+  ]
+}