Add correctness-gated UART parser profiles and an opt-in chunk scanner

.github/workflows/tests.yml

@@ -30,6 +30,8 @@ jobs:
       - name: Build source and wheel distributions
         run: python -m build
       - name: Install wheel instead of editable source
-        run: python -m pip install --force-reinstall --no-deps dist/uart_fixture_kit-0.1.0-py3-none-any.whl
+        run: python -m pip install --force-reinstall --no-deps dist/uart_fixture_kit-0.2.0-py3-none-any.whl
       - name: Replay installed fixtures
         run: python -m uart_fixture_kit demo --json
+      - name: Profile installed parsers with a small synthetic corpus
+        run: python -m uart_fixture_kit profile --records 8 --repeats 3 --warmups 1 --output profile-smoke.json --html profile-smoke.html

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 0.2.0
+
+- Chunk-scanning JSON alternative, preserving the byte-wise reference parser.
+- Correctness-gated parser profiles with fixed synthetic workloads, raw paired timing samples, source/fixture hashes and cProfile hotspots.
+- Standalone HTML profile reports and installed-wheel CLI verification in CI.
+
 ## 0.1.0
 
 - Protocol-independent JSON byte fixtures and virtual-time replay.

MANIFEST.in

@@ -1,2 +1,2 @@
 include CONTRIBUTING.md CHANGELOG.md
-recursive-include docs *.md
+recursive-include docs *.md *.json *.html *.png

README.md

@@ -11,7 +11,7 @@ A serial read can return half a message, several messages, or noise followed by
 Python 3.10 or newer. No runtime dependencies, API keys, GPU, or hardware.
 
 ```console
-python -m pip install "git+https://github.com/vanovri/uart-fixture-kit.git@v0.1.0"
+python -m pip install "git+https://github.com/vanovri/uart-fixture-kit.git@v0.2.0"
 uart-fixture-kit demo
 uart-fixture-kit demo --json
 ```
@@ -20,6 +20,29 @@ The demo runs bundled synthetic cases for newline JSON and a small binary protoc
 
 GitHub is the distribution source for this release. This package has not been published on PyPI.
 
+## UART Parse Profiler
+
+Version 0.2 adds a chunk-scanning JSON alternative without changing the byte-wise reference parser or the default demo. Measure both against the same synthetic input and virtual timestamps:
+
+```console
+uart-fixture-kit profile --records 2048 --repeats 9 --warmups 3 --output uart-profile.json --html uart-profile.html
+```
+
+Open the HTML file for the timing table and cProfile hotspots. JSON keeps every timing sample, environment details, source hashes and a manifest covering input bytes, timestamps and expected records. Seven saved JSON fixtures and four generated workloads must match their explicit record oracles on both parsers before timing. Each timed run is checked again outside the timer. No network connection or serial hardware is required.
+
+One Windows/CPython 3.14.7 run on 18 October 2026 measured the following medians over nine paired repeats:
+
+| Synthetic workload | Reference ms | Chunk scanning ms | Reference / candidate |
+| --- | ---: | ---: | ---: |
+| 76,739 bytes in 4,096-byte chunks | 18.368 | 8.669 | 2.12x |
+| The same stream, repeating chunks 1/2/7/3 | 27.723 | 26.099 | 1.06x |
+| The same stream, one byte per event | 44.280 | 56.507 | 0.78x |
+| Malformed, oversized, stale and partial input recovery | 0.064 | 0.046 | 1.38x |
+
+The alternative was **slower for byte-by-byte arrival**. The smaller difference on fragmented input may be sensitive to measurement noise. These are parser/event-dispatch measurements on one machine, not UART link speed or a general performance guarantee. The original parser remains the default. See [methodology and exact replay](docs/PROFILING.md), [recorded JSON](docs/profile-sample.json) and [standalone HTML](docs/profile-sample.html).
+
+![Actual local profile report, including the slower byte-wise case](docs/profile-preview.png)
+
 ## Catch a real class of bug
 
 This fixture delivers `OK\n` in two reads:
@@ -150,6 +173,8 @@ python -m build
 
 Tests include a failing broken-parser regression, every two-chunk split of representative streams, byte-by-byte corruption recovery, timer boundary cases, CLI exit codes, fixture validation, and installed-package demo checks. CI runs on Windows and Linux across Python 3.10, 3.12, and 3.14.
 
+Profiler tests also cover the 127/128/129-byte line boundary, mixed invalid input, empty-feed timeout behavior, bounded oversized partial input, rejection of an incorrect candidate before timing, raw-sample reports and the installed-wheel CLI.
+
 ## Related tools
 
 [pySerial](https://pyserial.readthedocs.io/en/latest/) provides serial transport and loopback facilities. [Serial Studio](https://github.com/Serial-Studio/Serial-Studio) provides telemetry visualization and capture/replay. [WireDAQ](https://github.com/wuisabel-gif/WireDAQ) includes its own wire protocol and fault simulation. This project focuses on a small fixture contract for adapters to existing parsers. Replay and fault injection are established techniques.

docs/PROFILING.md

@@ -0,0 +1,42 @@
+# UART Parse Profiler
+
+This extends the existing synthetic parser fixture kit. The reference `JsonLineParser` remains unchanged. `ChunkedJsonLineParser` scans for newlines using `bytes.find` and appends bounded slices instead of looping over every byte in Python. Both retain unfinished bytes, reject lines longer than 128 bytes, recover after newline or inactivity, validate fields and emit the same diagnostic records. The alternative inherits the reference's `advance` and `finish` methods.
+
+## Replay the comparison
+
+```console
+python -m uart_fixture_kit profile --records 2048 --repeats 9 --warmups 3 --output uart-profile.json --html uart-profile.html
+```
+
+This runs two parsers over seven bundled JSON fixtures and four generated workloads, giving 22 correctness comparisons. A mismatch prevents timing entirely. Subsequent warm-ups, measured runs and cProfile runs must also preserve canonical JSON output exactly. Boolean and numeric validation, diagnostic order and unfinished-buffer behavior belong to the contract, not just decoded values.
+
+The three valid-data workloads use the same 2,048 records and 76,739 bytes. Only chunk boundaries and the resulting virtual event schedule differ: chunks of up to 4,096 bytes, a repeating 1/2/7/3 schedule, and individual bytes. The fourth workload has an explicit oracle for an expired partial record, invalid UTF-8, an oversized line, a bad unit, a valid message, and an EOF fragment. Workload creation is deterministic and uses no random state. Additional seeded differential tests explore mixed-stream chunk boundaries, alongside independent saved oracles and line-boundary cases.
+
+`fixture_manifest` and each workload hash cover canonical JSON containing the fixture name, exact event bytes, timestamps, end time and expected records. The report hashes both parser class definitions. Because the alternative inherits timer/EOF methods, keep the reference source hash with the alternative's hash. Reports record Python, OS, CPU information, repeat count and warm-up count. Source and fixture hashes identify what ran, not physical-device validity.
+
+## What the timer measures
+
+The `perf_counter_ns` interval includes constructing a fresh parser, dispatching virtual `advance`/`feed`/`finish` calls and collecting emitted records. It excludes fixture generation, JSON equivalence checks, hashing, report creation and cProfile. Virtual timestamps are inputs to parser logic, not wall-clock sleeps.
+
+Both parsers receive warm-ups before measurement. Each repeat is paired. The reference runs first on even-numbered iterations and second on odd-numbered iterations to reduce a fixed ordering bias. Garbage collection is disabled only during each timed run and restored to its previous state afterward. Raw nanosecond samples are retained, with median, minimum, maximum, population standard deviation and nearest-rank observed p95. With nine samples, p95 is the maximum observed sample, not a latency-distribution estimate.
+
+`speed_ratio` is reference median divided by alternative median. Above 1 means the alternative took less median time. `candidate_time_change_percent` is (alternative/reference - 1) times 100, so a positive number means a slowdown. These ratios use measured parser times, not a device's baud rate. Small recovery timings and small differences are especially sensitive to noise.
+
+A separate cProfile run uses the bulk workload. Its cumulative timings overlap and include instrumentation overhead. Do not add cumulative times or compare them directly with uninstrumented wall times. It helps locate Python-level work; it does not prove the entire application sped up.
+
+## Observed example
+
+The checked-in `profile-sample.json` and HTML came from one run on 18 October 2026 WIB, CPython 3.14.7, Windows 11, Intel64 Family 6 Model 165, with eight logical CPUs. With nine measured repeats and three warm-ups:
+
+- Bulk chunk median: 18.368 ms reference, 8.669 ms alternative, ratio 2.12x.
+- Fragmented median: 27.723 ms reference, 26.099 ms alternative, ratio 1.06x.
+- Byte-wise median: 44.280 ms reference, 56.507 ms alternative, ratio 0.78x.
+- Fault-recovery median: 0.064 ms reference, 0.046 ms alternative, ratio 1.38x.
+
+The alternative was slower on one-byte events. It remains opt-in. Larger chunks allow the C-level search to avoid much of the Python byte loop, while tiny chunks still pay search and method-call overhead. On the bulk cProfile run, the reference made 74,691 `bytearray.append` calls. That observation supports the mechanism being explored; it does not establish the same ratio on another machine.
+
+There is no CPU-affinity setting, process isolation, thermal control or hardware I/O. Other applications may affect the results. The HTML and JSON deliberately preserve the slower case. Re-run on your target workload before choosing a parser. This synthetic comparison does not validate electrical UART behavior, sensor timing, production reliability or clinical use.
+
+## Provenance
+
+The source, workload definitions and record oracles are original synthetic additions for this project, developed with Codex assistance. They contain no shared thesis code, private captures or subject data. The broad theme of profiling before accepting an optimization was inspired by [Google's account of optimizing the ADK Python codebase](https://developers.googleblog.com/optimizing-a-200000-line-python-codebase-overnight-with-gemini-4-argon/). This project does not run or benchmark Gemini, and none of Google's performance figures are reused as local results.

docs/profile-sample.html

@@ -0,0 +1,26 @@
+<!doctype html><html lang="en"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
+<title>UART Parse Profiler</title><style>
+:root{font-family:system-ui,sans-serif;color:#142f40;background:#fafaf7}*{box-sizing:border-box}body{margin:0}main{max-width:1120px;margin:auto;padding:52px 24px}header{display:flex;justify-content:space-between;gap:20px;font-size:13px}h1{font-size:clamp(36px,5vw,60px);letter-spacing:-.04em;margin:52px 0 14px}p{line-height:1.7;max-width:850px}.label{color:#386452;font-weight:600}.gate{background:#142f40;color:#fafaf7;padding:24px 28px;margin:30px 0}.gate strong{font-size:28px}.table-wrap{overflow-x:auto}table{border-collapse:collapse;width:100%;margin:22px 0;text-align:left}th,td{border-bottom:1px solid #d4dcdd;padding:18px 12px;vertical-align:top}thead th{font-size:13px}td{font-variant-numeric:tabular-nums}small{display:block;font-size:12px;line-height:1.7;color:#526474;font-weight:400;margin-top:5px}.note{font-size:14px;color:#526474}h2{font-size:24px;margin-top:46px}code{font-size:12px;overflow-wrap:anywhere;display:block;margin:8px 0 20px}details{margin-top:28px}summary{cursor:pointer;padding:14px 0}ul{list-style:none;padding:0}footer{border-top:1px solid #d4dcdd;margin-top:45px;padding-top:20px;font-size:13px}a{color:inherit}@media(max-width:600px){main{padding:28px 16px}header{flex-direction:column;gap:6px}th,td{padding:14px 9px}table{min-width:640px}h1{margin-top:35px}}
+</style><main><header><strong>uart-fixture-kit / Parse Profiler</strong><span>2026-10-18T00:14:59.200252+00:00</span></header>
+<h1>Parser output first.<br>Timing second.</h1><p class="label">Synthetic byte streams. Virtual arrival times. Actual local measurements.</p>
+<p>The unchanged byte-wise JSON reference parser is compared with a chunk-scanning alternative. Every timed run must preserve the expected records, including diagnostics.</p>
+<section class="gate"><strong>22 / 22 correctness checks passed</strong><p>Bundled JSON fixtures and four generated workloads run through both parsers before measurement.</p></section>
+<h2>Paired timing results</h2><p class="note">Median milliseconds. The smaller line shows min–max over 9 measured runs after 3 warm-ups. Ratio = reference median / chunk-scanning median. Values below 1 mean the alternative was slower.</p>
+<div class="table-wrap"><table><thead><tr><th>Workload</th><th>Reference ms</th><th>Chunk scanning ms</th><th>Ratio</th></tr></thead><tbody><tr><th>bulk-4096<small>76,739 bytes / 19 events</small></th>
+<td>18.368<small>17.822–20.138</small></td>
+<td>8.669<small>8.493–9.218</small></td>
+<td>2.12x</td></tr><tr><th>fragmented-1-2-7-3<small>76,739 bytes / 23,612 events</small></th>
+<td>27.723<small>26.933–36.215</small></td>
+<td>26.099<small>24.787–28.874</small></td>
+<td>1.06x</td></tr><tr><th>bytewise<small>76,739 bytes / 76,739 events</small></th>
+<td>44.280<small>42.690–46.388</small></td>
+<td>56.507<small>54.889–59.571</small></td>
+<td>0.78x</td></tr><tr><th>fault-recovery<small>206 bytes / 32 events</small></th>
+<td>0.064<small>0.061–0.066</small></td>
+<td>0.046<small>0.041–0.048</small></td>
+<td>1.38x</td></tr></tbody></table></div>
+<h2>Measurement conditions</h2><p>CPython 3.14.7, Windows-11-10.0.26300-SP0, 8 logical CPUs.</p>
+<p class="note">Intel64 Family 6 Model 165 Stepping 2, GenuineIntel. Timer: perf_counter_ns. Parser construction, event dispatch and record creation are timed. Exact JSON comparison runs outside the timer. Paired order alternates. Garbage collection is disabled during each timed run and restored afterward. This is one machine, without process isolation or a CPU-affinity setting. No serial device, electrical timing or hardware speed is measured.</p>
+<details><summary>cProfile hotspots, bulk workload</summary><p class="note">A separate instrumented run. Cumulative times overlap and must not be added or compared directly with the uninstrumented timing table.</p><h3>bytewise_reference</h3><table><thead><tr><th>Function</th><th>Calls</th><th>Cumulative ms</th></tr></thead><tbody><tr><td>profiler.py:73 run_once</td><td>1</td><td>84.303</td></tr><tr><td>examples.py:25 feed</td><td>19</td><td>84.164</td></tr><tr><td>__init__.py:304 loads</td><td>2,048</td><td>18.016</td></tr><tr><td>~:0 &lt;method &#x27;append&#x27; of &#x27;bytearray&#x27; objects&gt;</td><td>74,691</td><td>12.599</td></tr><tr><td>~:0 &lt;built-in method builtins.len&gt;</td><td>78,787</td><td>9.845</td></tr><tr><td>decoder.py:340 decode</td><td>2,048</td><td>9.016</td></tr><tr><td>__init__.py:248 detect_encoding</td><td>2,048</td><td>4.146</td></tr></tbody></table><h3>chunk_scanning</h3><table><thead><tr><th>Function</th><th>Calls</th><th>Cumulative ms</th></tr></thead><tbody><tr><td>profiler.py:73 run_once</td><td>1</td><td>32.613</td></tr><tr><td>optimized.py:16 feed</td><td>19</td><td>32.504</td></tr><tr><td>__init__.py:304 loads</td><td>2,048</td><td>17.401</td></tr><tr><td>decoder.py:340 decode</td><td>2,048</td><td>8.779</td></tr><tr><td>__init__.py:248 detect_encoding</td><td>2,048</td><td>3.973</td></tr><tr><td>decoder.py:351 raw_decode</td><td>2,048</td><td>2.623</td></tr><tr><td>~:0 &lt;method &#x27;match&#x27; of &#x27;re.Pattern&#x27; objects&gt;</td><td>4,096</td><td>1.709</td></tr></tbody></table></details>
+<details><summary>Fixture and parser SHA256 evidence</summary><p class="note">Fixture digests cover canonical event bytes, virtual timestamps, expected records and name. Source digests cover the two class definitions; the alternative inherits timer and EOF handling from the reference.</p><h3>Parser sources</h3><code>c9562cf89b2049d0b7299cf7ab857b008e0efb6de0fc4fa1dfdecf4abfda73e7</code><code>da86bce5a1d40f33b62efb4b8c088be1122c49934bd3ff718d8bbb504113a7ac</code><ul><li><strong>bulk-4096</strong><code>cc202aae41aa6da835ca9f34f99acee02936d6e0afdaa376ecff663a94b890f8</code></li><li><strong>fragmented-1-2-7-3</strong><code>579d413e5b2fb5a10c22ca452acb358f6728251da0a368b4a4267de214513694</code></li><li><strong>bytewise</strong><code>33172f6e1f12446c27e1ff3e3e03a833fc3bf28c4aff577ffef7913f881d2d92</code></li><li><strong>fault-recovery</strong><code>03df6f9982da793b55f4d70c410c5afb95372f1b3fdd42d5857be84082f5d0e1</code></li></ul></details>
+<footer><a href="https://github.com/vanovri/uart-fixture-kit">Source and contracts</a>. Developed with Codex assistance. Results depend on workload and environment.</footer></main></html>