Add body-pressure CI coverage#7
Open
upgle wants to merge 4 commits into
Open
Conversation
upgle
force-pushed
the
codex/body-pressure-ci
branch
from
July 19, 2026 10:15
a3e3307 to
7ec21a6
Compare
There was a problem hiding this comment.
Pull request overview
This PR expands the CI/QA pipeline to add deterministic “body-pressure” coverage (large request/response inspection at 1 MiB and concurrent budget exhaustion), improves benchmark/report visibility in GitHub Actions summaries, and adds a single-retry mechanism for qualification runs that only fail performance thresholds.
Changes:
- Add a CI wrapper to retry qualification performance-threshold failures once while preserving attempt artifacts.
- Extend the qualification benchmark to support response inspection, large-body workloads, p99.9 reporting, and sampled peak RSS evidence.
- Render CRS/benchmark summaries and publish Markdown previews + annotations in the QA workflow, and run the body-pressure stress profile in the existing QA job.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/run-ci-qualification.sh | Adds one-retry orchestration for performance-threshold-only qualification failures and manages attempt artifact layout. |
| tools/qualification-benchmark.py | Adds body-pressure workload parameters, response inspection config, p99.9 reporting, and peak RSS sampling/recording. |
| tools/ci-report-summary.py | Generates GitHub Actions QA summary content, markdown previews, and threshold annotations from produced artifacts. |
| tools/ci-envoy-build.sh | Runs qualification via the retry wrapper and adds body-pressure stress execution to the QA mode. |
| tools/BUILD | Exports new CI helper scripts for Bazel data dependencies. |
| test/tools/ci_report_summary_test.py | Adds unit tests for summary rendering, preview generation, and annotation formatting. |
| test/tools/ci_qualification_retry_test.py | Adds unit tests validating retry/publish behavior for qualification runs. |
| test/tools/BUILD | Registers the new tool-focused py_tests and their script data dependencies. |
| test/README.md | Documents the new body-pressure target and updated integration coverage scope. |
| test/performance/qualification_bootstrap.yaml.tmpl | Enables response max-bytes limits and increases route timeout for the benchmark profile. |
| test/integration/filter_protocol_integration_test.cc | Adds integration coverage for chunked response overflow and HTTP/2 aggregate-budget exhaustion behavior. |
| test/integration/envoy_http_integration_test.py | Adds upstream endpoints + tests for exact-limit responses and chunked overflow above the inspection limit. |
| Makefile | Adds body-pressure-stress target and includes new tool tests in test/check. |
| docs/development.md | Updates developer docs to describe body-pressure CI evidence, summaries, retry behavior, and new benchmark outputs. |
| .github/workflows/ci.yml | Adds QA summary generation, preview artifact uploads, annotations, and body-pressure artifact publishing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+559
to
+562
| if rss_sampler_stop is not None: | ||
| rss_sampler_stop.set() | ||
| if rss_sampler_thread is not None: | ||
| rss_sampler_thread.join(timeout=1) |
Comment on lines
+148
to
+152
| body = ( | ||
| b"r" * ARGS.large_body_bytes | ||
| if self.path == "/large-response" | ||
| else b"ok" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
The existing CI covers boundary conditions, short body waves, and atomic budget contention, but it did not hold multiple real HTTP streams against the aggregate body budget or transfer large inspected responses. It also lacked a pull-request stress profile for concurrent 1 MiB request/response traffic and repeated 256 KiB body waves.
The generated benchmark and CRS reports were available only as archived workflow artifacts, so reviewers had to download and unpack them before inspecting the results. A single GitHub-hosted runner tail-latency excursion could also fail the qualification gate even when Envoy CPU per request was unchanged.
Impact
Pull requests gain deterministic aggregate-budget and actual-response-overflow coverage through the existing QA and sanitizer targets. The full stress profile reuses QA's already-built custom Envoy binary, adding the workload without another build job.
Reviewers can inspect CRS result counts and benchmark latency, throughput, CPU, RSS, and terminal-gauge evidence directly in the Actions run summary. A completed qualification measurement that only exceeds performance thresholds is retried once; both attempts must exceed before QA fails, and both reports are retained. Runtime errors, request failures, and incomplete reports fail immediately without a retry. Body-pressure stress still runs after repeated qualification performance failures so the PR retains its stress evidence.
The first qualification threshold failure is emitted as a warning annotation and a repeated final failure as an error. Standalone Markdown reports open in the browser, and complete JSON, Markdown, and diagnostic evidence remains available as 30-day workflow artifacts.
Unexpected responses, timeouts, and non-zero terminal gauges fail QA. Body-pressure performance and RSS thresholds remain diagnostic until stable Linux evidence is reviewed.
Validation
bazel test --macos_minimum_os=10.15 //test/integration:filter_protocol_integration_test //test/integration:envoy_http_integration_testpython3 -m unittest test.tools.ci_qualification_retry_test test.tools.ci_report_summary_test— 12 tests passedbazel test --macos_minimum_os=10.15 //test/tools:ci_qualification_retry_test //test/tools:ci_report_summary_testgit diff --check