Skip to content

feat: 16.5.3 ci code coverage & quality gates - #81

Merged
vieiralucas merged 8 commits into
mainfrom
feat/16.5.3-ci-code-coverage-quality-gates
Mar 22, 2026
Merged

vieiralucas merged 8 commits into
mainfrom
feat/16.5.3-ci-code-coverage-quality-gates

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Mar 21, 2026 •

Copy link
Copy Markdown
Member

Summary

  • New .github/workflows/coverage.yml with cargo-llvm-cov + nextest
  • Per-crate coverage breakdown displayed in GitHub step summary
  • Security-critical paths explicitly reported: auth module, auth middleware, cluster module, TLS config
  • Coverage baseline established (workflow verified on feature branch — run 23384145766 succeeded)
  • --no-fail-fast ensures coverage reports even if individual tests have issues

Test plan

  • Coverage workflow verified on feature branch (run 23384145766 succeeded)
  • Workflow trigger narrowed back to main + pull_request after verification
  • CI passes on PR (coverage workflow will run as part of PR checks)

🤖 Generated with Claude Code


Summary by cubic

Integrates code coverage into the main CI test job using cargo-llvm-cov + nextest, and posts overall, per-crate, and security-path coverage in the PR summary. Also adds a readable benchmark summary to the regression workflow.

  • New Features

    • Folds coverage into .github/workflows/ci.yml “Test + Coverage” job; removes the standalone coverage workflow.
    • Installs llvm-tools-preview and cargo-llvm-cov,cargo-nextest; runs: cargo llvm-cov nextest --workspace --exclude fila-bench --exclude fila-e2e --no-fail-fast --json --summary-only --output-path coverage.json.
    • Displays overall and per-crate coverage in the step summary; highlights crates/fila-core/src/broker/auth.rs, crates/fila-server/src/auth.rs, crates/fila-core/src/cluster/, and TLS config in crates/fila-core/src/broker/config.rs.
    • Adds a benchmark summary to bench-regression.yml with “vs main baseline” when available, or a fallback table from bench-aggregated.json.
  • Bug Fixes

    • Fixes per-crate/path coverage parsing for absolute paths using contains/endswith.
    • Formats overall coverage percent to one decimal place for readability.

Written for commit 4ab17bd. Summary will update on new commits.

Benchmark Results (vs main baseline)

Baseline commit: 64935c6 PR commit: 592fcc1 Threshold: 10%

Benchmark Baseline Current Change Unit
compaction_active_p99 0.39 0.47 +20.3% ms 🔴
compaction_idle_p99 0.39 0.47 +21.4% ms 🔴
compaction_p99_delta -0.02 -0.01 +62.3% ms 🔴
consumer_concurrency_100_throughput 2195.00 2043.00 -6.9% msg/s
consumer_concurrency_10_throughput 2197.00 2051.67 -6.6% msg/s
consumer_concurrency_1_throughput 360.33 349.33 -3.1% msg/s
e2e_latency_p50_light 0.31 0.41 +31.9% ms 🔴
e2e_latency_p95_light 0.36 0.47 +30.3% ms 🔴
e2e_latency_p99_light 0.53 0.55 +2.3% ms
enqueue_throughput_1kb 3495.88 2694.85 -22.9% msg/s 🔴
enqueue_throughput_1kb_mbps 3.41 2.63 -22.9% MB/s 🔴
fairness_accuracy_max_deviation 0.20 0.20 +0.0% % deviation
fairness_accuracy_tenant-1 0.20 0.20 +0.0% % deviation
fairness_accuracy_tenant-2 0.20 0.20 +0.0% % deviation
fairness_accuracy_tenant-3 0.10 0.10 +0.0% % deviation
fairness_accuracy_tenant-4 0.10 0.10 +0.0% % deviation
fairness_accuracy_tenant-5 0.10 0.10 +0.0% % deviation
fairness_overhead_fair_throughput 1408.72 1381.41 -1.9% msg/s
fairness_overhead_fifo_throughput 1458.04 1421.12 -2.5% msg/s
fairness_overhead_pct 3.39 2.79 -17.6% % 🟢
key_cardinality_10_throughput 3193.98 2427.66 -24.0% msg/s 🔴
key_cardinality_10k_throughput 597.54 582.38 -2.5% msg/s
key_cardinality_1k_throughput 1142.30 1090.79 -4.5% msg/s
lua_on_enqueue_overhead_us 17.52 25.32 +44.5% us 🔴
lua_throughput_with_hook 1172.63 1141.97 -2.6% msg/s
memory_per_message_overhead 97.89 2904.47 +2866.9% bytes/msg 🔴
memory_rss_idle 163.53 145.63 -10.9% MB 🟢
memory_rss_loaded_10k 165.18 173.80 +5.2% MB

Summary: 10 regressed, 2 improved, 16 unchanged

⚠️ Performance regression detected — 10 metric(s) exceeded the 10% threshold

vieiralucas added a commit that referenced this pull request Mar 21, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/coverage.yml">

<violation number="1" location=".github/workflows/coverage.yml:97">
P2: This workflow reports coverage but does not implement an actual gate/regression check, so coverage drops will not fail CI.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread .github/workflows/coverage.yml Outdated
- coverage.yml: runs cargo-llvm-cov on every PR and push to main
- per-crate coverage breakdown in github step summary
- security-critical path coverage (auth, cluster, tls config) highlighted
- temporarily includes feature branch trigger for verification
The Test job in ci.yml now runs tests with cargo-llvm-cov instead of
plain nextest, producing coverage as a side effect. This eliminates:
- a redundant full workspace build (coverage.yml did its own build)
- a redundant test run (same tests, just uninstrumented)

The standalone coverage.yml is deleted. One build, one test run, coverage
included.
- fix coverage report: use contains/endswith instead of startswith for
  jq file path matching (paths are absolute in llvm-cov JSON output)
- format overall percentage to 1 decimal place
- add benchmark results to github step summary in bench-regression.yml
@vieiralucas
vieiralucas force-pushed the feat/16.5.3-ci-code-coverage-quality-gates branch from eb1b4cc to 4ab17bd Compare March 22, 2026 01:05
@vieiralucas
vieiralucas merged commit ae97fbe into main Mar 22, 2026
7 checks passed
@vieiralucas
vieiralucas deleted the feat/16.5.3-ci-code-coverage-quality-gates branch March 22, 2026 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant