Benchmarking tooling for PolyXML's Rust core and its language bindings. The directory is organized as one self-contained directory per language plus a cross-language orchestrator, so new suites never crowd the root.
benchmarks/
├── README.md # This index: catalog, methodology, conventions
├── run_all.sh # Orchestrator: builds the extension, runs Rust + Python suites
├── __init__.py # Package marker for `python -m benchmarks.python`
├── cli/ # CLI startup and argument validation (hyperfine)
├── python/ # Python bindings vs the Python XML ecosystem
├── python-json/ # Python JSON data-binding vs xsdata (msgspec)
├── python-binary/ # Python binary serialization vs pickle and cloudpickle
├── python-aot/ # Python Ahead-of-Time (AOT) PyO3 C-extensions vs Dataclasses
├── java/ # Java bindings vs JAXB, Jackson, and Panama (JMH)
├── go/ # Go generated models vs handwritten structs
├── cpp/ # C++ generated-model XML adapter
├── csharp/ # C# generated models vs handwritten classes
├── rust-phf-e2e/ # Generated Rust match vs phf XML decoding
├── rust-phf-compile/ # Capped generated-consumer compile time and binary size
├── workloads/ # Shared XML input fixtures for cross-language runs
└── typescript-wasm/ # Node, Bun, and browser Wasm comparisons
Each suite directory owns its README.md, runner, dependencies, and generated
results; generated output stays in the suite's gitignored target/ (or
equivalent) and never at this root.
| Suite | Location | Tooling | How to run |
|---|---|---|---|
| Matched serializer comparison | serializer-comparison/ |
Fresh-process Rust/Python, native Go/C#, Java JMH | See suite README |
| Rust core engine | crates/polyxml-core/benches/ |
Criterion.rs | cargo bench --bench core_benchmarks |
| Rust same-host XML/Serde regression | rust-xml-regression/ |
Generated consumers + Criterion | See suite README |
| Rust runtime investigation | rust-runtime-investigation/ |
Dynamic core constraints, allocations and instruction profiles | See suite README |
| Rust tag dispatch | crates/polyxml-core/benches/ |
Criterion.rs + perf stat |
cargo bench --bench tag_dispatch (results) |
| Rust end-to-end dispatch | rust-phf-e2e/ |
Generated decoders | ./benchmarks/rust-phf-e2e/run.sh |
| Rust dispatch compile cost | rust-phf-compile/ |
Sequential capped release builds, GNU time, size | See suite README |
| Python XML comparative | python/ |
Custom CLI suite | python -m benchmarks.python or ./benchmarks/run_all.sh |
| Python JSON comparative | python-json/ |
Custom CLI suite | ./benchmarks/python-json/run.sh |
| Python binary serialization | python-binary/ |
Custom CLI suite | ./benchmarks/python-binary/run.sh |
| Python AOT native extensions | python-aot/ |
Custom CLI suite | ./benchmarks/python-aot/run.sh |
| CLI startup | cli/ |
hyperfine | ./benchmarks/cli/benchmark.sh |
| Java four-runtime | java/ |
JMH (Maven) | See java/README.md |
| Go models | go/ |
go test -bench |
./benchmarks/go/run.sh |
| C++ native binding + XML adapter | cpp/ |
C++20 chrono | ./benchmarks/cpp/run_runtime.sh and ./benchmarks/cpp/run.sh |
| C# models | csharp/ |
.NET Stopwatch | ./benchmarks/csharp/run.sh |
| TypeScript/Wasm | typescript-wasm/ |
Node, Bun, Chromium | See typescript-wasm/README.md and methodology |
Readers in all seven targets can use the same byte-for-byte sensor XML fixtures, and evaluate moderate nested/attribute workloads using the trade-order workload. Their different return values, runtimes, and serializers still require separate interpretation.
The Rust Criterion suite lives inside its crate because cargo bench requires
benches/ next to the crate manifest — it is the one deliberate exception to the
one-directory-per-suite rule. Filter a benchmark and open the HTML report:
cargo bench --bench core_benchmarks -- deserialization
# Report: target/criterion/report/index.html./benchmarks/run_all.shRequires Rust 1.80+ and Python 3.12+ (or uv). The script compiles
polyxml-python in --release, runs Criterion, then runs the Python suite and
writes benchmarks/python/results.md / results.json. The Java, Go, C++, C#,
TypeScript/Wasm, and CLI suites have separate entry points shown in the catalog;
their runtime dependencies and workloads differ substantially.
Suite READMEs document their own tooling; these rules apply to all of them:
- Smoke runs prove execution, not speed. Short iterations and single forks are execution checks only — never quote them as performance evidence.
- Measure on an otherwise idle host and record the toolchain, OS/CPU, and repository revision next to the numbers.
- Inspect confidence intervals and repeat before making any throughput claim; a single run on a shared machine is noise.
- Published figures live in
docs/benchmarks/index.md— update them from a full, idle-host run, not from a smoke. - Run under a memory cap on shared/small hosts. Heavy builds and bench
runs go through
scripts/memcap.sh, which caps the run at 60% of available RAM in an isolated cgroup (kernel OOM-kills only the runaway process instead of freezing the host;ulimitfallback where systemd is unavailable). The suite entry points (run_all.sh,cli/benchmark.sh,rust-phf-e2e/run.sh,scripts/perf_stat.sh) already re-exec through it; wrap any ad-hoccargo bench/--releasebuild the same way. Tune withPOLYXML_MEMCAP_PCT, opt out withPOLYXML_MEMCAP_DISABLE=1.
- Create
benchmarks/<language>/and copyjava/README.mdas a template: it covers build, run commands, workloads, interpretation caveats, and profiler usage. - Keep the suite self-contained — build files, generators, tests, and results all live under the suite directory, with generated output gitignored.
- Do not add loose files to this root; it holds only this index, the orchestrator, and the package marker.
- If
run_all.shshould drive the suite, gate it on toolchain detection so machines without that toolchain still succeed. - Add the suite to the catalog above and link it from
docs/benchmarks/index.md.
Published methodology, result tables, and reproduction commands:
docs/benchmarks/index.md.