Skip to content

Create a fresh parser per benchmark iteration - #318

Merged
Kludex merged 2 commits into
mainfrom
fix-benchmark-parser-lifecycle
Aug 9, 2026
Merged

Create a fresh parser per benchmark iteration#318
Kludex merged 2 commits into
mainfrom
fix-benchmark-parser-lifecycle

Conversation

@Kludex

@Kludex Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • replace parser fixtures with explicit parser factory helpers
  • create and finalize a fresh parser inside every measured benchmark invocation
  • ensure repeated CodSpeed iterations parse the complete input instead of writing to an already-finished parser

Why

CodSpeed invokes each benchmark function repeatedly while retaining its pytest fixtures. The multipart fixture previously yielded one parser for the entire benchmark run. After the first invocation that parser was in MultipartState.END, so later iterations only discarded input.

The querystring fixture had a similar lifecycle problem: repeated invocations continued from the previous parser state, and finalize() ran only once after all measured iterations.

The broken lifecycle was visible locally because the simple and 100-field multipart benchmarks had nearly identical simulated costs. With a fresh parser per invocation, the 100-field benchmark performs far fewer iterations and correctly measures the complete parse.

CodSpeed report

CodSpeed now reports five new benchmarks and no performance regression, establishing the corrected baseline successfully. The five old benchmark identifiers are skipped because their ended-parser measurements are intentionally replaced.

This PR deliberately changes the measured workload. The base branch parses the multipart input only on the first benchmark iteration; subsequent iterations write to a parser already in MultipartState.END and discard the input. The head branch parses the complete request on every iteration.

The benchmark functions are renamed from test_multipart_* to test_parse_multipart_* (and likewise for querystrings) because these are new benchmark semantics, not comparable performance measurements. This establishes a clean CodSpeed baseline for complete request parses and prevents the ended-parser measurements from being reported as package regressions.

Validation

  • 160 tests pass
  • 100% statement coverage
  • Ruff passes
  • mypy passes
  • all five CodSpeed benchmarks run successfully with a fresh parser per invocation

Review in cubic

@codspeed-hq

codspeed-hq Bot commented Aug 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

🆕 5 new benchmarks
⏩ 5 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation test_parse_multipart_file_upload N/A 6.2 ms N/A
🆕 Simulation test_parse_multipart_large_form N/A 3.7 ms N/A
🆕 Simulation test_parse_multipart_simple_form N/A 242.5 µs N/A
🆕 Simulation test_parse_multipart_worstcase_boundary_chars N/A 1 ms N/A
🆕 Simulation test_parse_querystring_large_form N/A 558.7 µs N/A

Comparing fix-benchmark-parser-lifecycle (c3186eb) with main (aa7048c)

Open in CodSpeed

Footnotes

  1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

The latest CodSpeed run now reports five new complete-parse benchmarks and no performance regression, so the corrected baseline is established successfully. The five old benchmarks are skipped because their ended-parser measurements were intentionally replaced. No further code change is needed.

@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

The failing CodSpeed comparison was between two different workloads, not a package regression. In c3186eb I renamed the benchmark functions to start a new CodSpeed series for complete request parses. This prevents comparison against the old ended-parser measurements and gives follow-up performance PRs a valid baseline.

@Kludex
Kludex merged commit d9cb4c6 into main Aug 9, 2026
15 checks passed
@Kludex
Kludex deleted the fix-benchmark-parser-lifecycle branch August 9, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant