Skip to content

Publish Linux WAF benchmark and performance report - #9

Draft
upgle wants to merge 15 commits into
perf/libmodsecurity-connector-hot-pathsfrom
codex/linux-waf-pgo-benchmark
Draft

Publish Linux WAF benchmark and performance report#9
upgle wants to merge 15 commits into
perf/libmodsecurity-connector-hot-pathsfrom
codex/linux-waf-pgo-benchmark

Conversation

@upgle

@upgle upgle commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a benchmark-only Envoy target that measures an Envoy-only reference, this repository's
    libmodsecurity filter, and the Coraza Dynamic Module with the same executable and OWASP CRS files
  • add a manually dispatched Linux x86_64 Bazel opt + ThinLTO + instrumentation-PGO workflow with
    order-rotated runs, CPU partitioning, runtime-opt-in phase-1 stage timers, and Linux perf
    collection
  • publish a README snapshot, detailed methodology and limitations, raw and normalized results,
    archived textual perf reports, and deterministic SVG chart generation

Stack

This benchmark and documentation PR is temporarily stacked on
#11, which contains the separately reviewed
libmodsecurity connector optimization bundle. After #11 merges, this PR can be rebased and
retargeted to main without changing the benchmark payload.

The measured source is preserved independently of this rewritten PR branch:

  • source archive: benchmark/waf-linux-2026-07-22
  • project commit: 6c396c920970bc4d09b7a2f1c8b14b4f7775eaba
  • project tree: 9900d4510fa8fdc1e81956fc3d7e82b41c6f2b0d
  • connector patch SHA-256:
    558d0e5c6e039472828c27c3fac8ded2987d2f026b13b8bd4a5be9e3ca41918e

Recorded snapshot

The archived run uses Envoy 1.39.0, libmodsecurity 3.0.16, Coraza 3.7.0 through Built On Envoy
0.6.2, and OWASP CRS 4.28.0.

Across the 12 measured workload and concurrency combinations, the libmodsecurity integration
recorded higher median request throughput. The primary figure reports actual requests per second;
paired throughput, Envoy CPU-per-request, and p99 ratios are retained as supporting detail.

The source result mislabeled the final binary as stripped. The archived build manifest records the
actual --strip=never flag; this PR corrects future workflow metadata while preserving the
original reported value in the normalized evidence.

Method

  • use one Envoy worker and either one or 16 persistent HTTP/1.1 client connections
  • warm each fresh process and rotate mode order over three repeats
  • pin Envoy separately from clients and the loopback upstream when runner topology permits
  • validate every response and record throughput, latency, Envoy CPU, RSS, startup time, and
    terminal filter statistics
  • profile native phase-1 blocking separately with stage counters disabled for Linux perf
    sampling

Validation

  • Python comparison and chart-renderer tests: 19 passed
  • six Bazel engine, filter, and performance-tool targets passed
  • raw-result SHA-256 and compressed perf artifact integrity verified
  • chart and normalized-data regeneration is byte-for-byte deterministic
  • workflow YAML, Bash syntax, JSON, SVG/XML, documentation links, and git diff --check validated

Interpretation boundaries

  • this is one three-repeat GitHub-hosted runner snapshot, not a confidence interval or universal
    engine ranking
  • the headers_c16 observed range crosses parity despite its median
  • results include complete Envoy integration overhead and do not isolate regex engines or language
    runtimes
  • only the native C++ artifacts consume the LLVM profile; the Go module retains its upstream
    optimized release settings
  • Envoy-only attack paths are not security-equivalent to inspected paths

@upgle upgle changed the title Optimize ModSecurity hot paths and add Linux PGO performance reporting Add Linux PGO WAF benchmark and performance report Jul 25, 2026
@upgle
upgle force-pushed the codex/linux-waf-pgo-benchmark branch from 4354a1c to a632057 Compare July 27, 2026 14:28
@upgle upgle changed the title Add Linux PGO WAF benchmark and performance report Publish Linux WAF benchmark and performance report Jul 27, 2026
@upgle
upgle changed the base branch from main to perf/libmodsecurity-connector-hot-paths July 27, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant