Skip to content

Add Datadog code coverage upload#1718

Draft
ManuelPalenzuelaDD wants to merge 1 commit intomainfrom
add-datadog-coverage
Draft

Add Datadog code coverage upload#1718
ManuelPalenzuelaDD wants to merge 1 commit intomainfrom
add-datadog-coverage

Conversation

@ManuelPalenzuelaDD
Copy link

@ManuelPalenzuelaDD ManuelPalenzuelaDD commented Mar 12, 2026 •
edited
Loading

What does this PR do?

We're migrating Datadog repositories from Codecov to Datadog Code Coverage for tracking test coverage. This PR is the first step: it adds a Datadog coverage upload alongside the existing Codecov upload so we can run both systems in parallel and verify parity before switching over.

Changes

  • Added a DataDog/coverage-upload-github-action@v1 step to the coverage workflow, immediately after the existing Codecov upload step.
  • The existing Codecov upload is unchanged — nothing is removed or modified.
  • The Datadog upload uses continue-on-error: true, so it will never block CI even if it fails.

Why are we doing this?

As part of a company-wide effort, we're consolidating code coverage reporting into Datadog's own Code Coverage product. This gives us:

  • Coverage data integrated directly into Datadog CI Visibility
  • PR gates and coverage checks natively in Datadog
  • No dependency on a third-party service (Codecov) for coverage reporting

Validation

CI has run on this PR and both uploads completed successfully. Coverage numbers match:

System Coverage
Codecov 71.24%
Datadog 71.25%

The 0.01% difference is within expected tolerance (rounding differences in line counting).

Next steps (not in this PR)

Once this PR is merged and we've confirmed Datadog coverage is stable over several commits:

  1. Remove the Codecov upload step and CODECOV_TOKEN secret
  2. Remove .codecov.yml
  3. Optionally configure PR gates in code-coverage.datadog.yml

No action needed from reviewers beyond normal review

This is a low-risk, additive change. The new step runs independently of the existing CI pipeline and cannot cause test failures.

@ManuelPalenzuelaDD
Add Datadog code coverage upload alongside Codecov
ffcdf9b 
Side-by-side coverage reporting: adds Datadog coverage upload step
after the existing Codecov upload in the coverage workflow. Uses the
DataDog/coverage-upload-github-action with LCOV format. Both systems
will run in parallel for comparison.
@codecov-commenter
Copy link

codecov-commenter commented Mar 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.24%. Comparing base (bc8f764) to head (ffcdf9b).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1718      +/-   ##
==========================================
+ Coverage   71.23%   71.24%   +0.01%     
==========================================
  Files         429      429              
  Lines       63547    63547              
==========================================
+ Hits        45268    45277       +9     
+ Misses      18279    18270       -9
Components Coverage Δ
libdd-crashtracker 62.39% <ø> (+0.04%) ⬆️
libdd-crashtracker-ffi 17.18% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 88.32% <ø> (ø)
libdd-data-pipeline-ffi 76.94% <ø> (ø)
libdd-common 79.73% <ø> (ø)
libdd-common-ffi 73.40% <ø> (ø)
libdd-telemetry 62.48% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 80.35% <ø> (ø)
libdd-profiling 81.60% <ø> (+0.01%) ⬆️
libdd-profiling-ffi 63.65% <ø> (ø)
datadog-sidecar 32.61% <ø> (ø)
datdog-sidecar-ffi 8.35% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 91.80% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.08% <ø> (ø)
datadog-tracer-flare 88.95% <ø> (ø)
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Mar 12, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 71.25%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ffcdf9b | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Mar 12, 2026 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2026-03-12 14:25:28

Comparing candidate commit ffcdf9b in PR branch add-datadog-coverage with baseline commit fc16a5e in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 2 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 10.639ms 10.674ms ± 0.015ms 10.673ms ± 0.009ms 10.681ms 10.700ms 10.733ms 10.746ms 0.68% 1.247 3.762 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [10.672ms; 10.676ms] or [-0.019%; +0.019%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 88.567µs 88.924µs ± 0.150µs 88.940µs ± 0.087µs 89.014µs 89.122µs 89.233µs 89.826µs 1.00% 0.895 5.589 0.17% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [88.903µs; 88.945µs] or [-0.023%; +0.023%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 164.060µs 164.678µs ± 0.387µs 164.612µs ± 0.117µs 164.739µs 165.095µs 165.962µs 168.769µs 2.53% 6.473 61.807 0.23% 0.027µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [164.624µs; 164.732µs] or [-0.033%; +0.033%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 145.170µs 147.005µs ± 1.610µs 146.775µs ± 0.471µs 147.223µs 148.473µs 153.049µs 162.513µs 10.72% 5.777 46.504 1.09% 0.114µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [146.782µs; 147.228µs] or [-0.152%; +0.152%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2598 execution_time 3.425ms 3.479ms ± 0.050ms 3.458ms ± 0.012ms 3.484ms 3.587ms 3.625ms 3.630ms 4.97% 1.552 1.142 1.44% 0.004ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2598 execution_time [3.472ms; 3.486ms] or [-0.200%; +0.200%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.979µs 5.070µs ± 0.055µs 5.076µs ± 0.064µs 5.108µs 5.146µs 5.149µs 5.151µs 1.47% 0.022 -1.429 1.07% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.062µs; 5.078µs] or [-0.149%; +0.149%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.672µs 33.469µs ± 0.986µs 32.870µs ± 0.083µs 34.691µs 35.264µs 35.719µs 36.505µs 11.06% 1.034 -0.596 2.94% 0.070µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.332µs; 33.605µs] or [-0.408%; +0.408%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 13.939ms 14.017ms ± 0.037ms 14.013ms ± 0.019ms 14.034ms 14.077ms 14.139ms 14.207ms 1.38% 1.326 4.406 0.26% 0.003ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [14.012ms; 14.022ms] or [-0.036%; +0.036%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample2_frames_x1000 execution_time 727.974µs 729.714µs ± 0.808µs 729.653µs ± 0.358µs 730.008µs 730.701µs 731.575µs 737.628µs 1.09% 4.872 44.672 0.11% 0.057µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample2_frames_x1000 execution_time [729.602µs; 729.826µs] or [-0.015%; +0.015%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 493.187µs 494.120µs ± 0.615µs 494.027µs ± 0.270µs 494.317µs 494.765µs 496.618µs 499.384µs 1.08% 4.111 28.424 0.12% 0.044µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2002466.371op/s 2023801.158op/s ± 2507.892op/s 2024180.751op/s ± 1107.675op/s 2025237.383op/s 2026240.650op/s 2027031.645op/s 2027628.922op/s 0.17% -4.061 27.859 0.12% 177.335op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 370.032µs 370.677µs ± 0.303µs 370.635µs ± 0.174µs 370.851µs 371.116µs 371.294µs 372.882µs 0.61% 2.105 12.969 0.08% 0.021µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2681811.048op/s 2697766.029op/s ± 2201.149op/s 2698074.760op/s ± 1263.493op/s 2699085.500op/s 2700589.832op/s 2702066.434op/s 2702471.833op/s 0.16% -2.079 12.745 0.08% 155.645op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 167.335µs 167.680µs ± 0.207µs 167.667µs ± 0.095µs 167.756µs 167.920µs 168.123µs 169.769µs 1.25% 5.249 49.979 0.12% 0.015µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5890373.035op/s 5963750.535op/s ± 7320.263op/s 5964208.645op/s ± 3378.899op/s 5967587.603op/s 5971292.194op/s 5974231.963op/s 5976045.231op/s 0.20% -5.160 48.786 0.12% 517.621op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.183µs 37.287µs ± 0.067µs 37.275µs ± 0.033µs 37.321µs 37.382µs 37.417µs 37.928µs 1.75% 4.546 39.683 0.18% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26365815.139op/s 26819038.048op/s ± 47914.998op/s 26827532.972op/s ± 23399.601op/s 26847020.741op/s 26870801.087op/s 26879017.231op/s 26893721.392op/s 0.25% -4.434 38.253 0.18% 3388.102op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 45.581µs 45.708µs ± 0.054µs 45.703µs ± 0.035µs 45.742µs 45.794µs 45.866µs 45.932µs 0.50% 0.747 1.879 0.12% 0.004µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21771120.445op/s 21878060.593op/s ± 25750.970op/s 21880273.028op/s ± 16902.976op/s 21895271.151op/s 21918282.422op/s 21928172.400op/s 21939105.536op/s 0.27% -0.736 1.841 0.12% 1820.869op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [494.035µs; 494.206µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2023453.589op/s; 2024148.728op/s] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [370.635µs; 370.719µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2697460.971op/s; 2698071.087op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [167.651µs; 167.709µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5962736.017op/s; 5964765.053op/s] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.278µs; 37.296µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26812397.490op/s; 26825678.606op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [45.700µs; 45.715µs] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21874491.756op/s; 21881629.430op/s] or [-0.016%; +0.016%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 48.913ms 49.410ms ± 0.836ms 49.328ms ± 0.122ms 49.413ms 49.637ms 52.000ms 58.797ms 19.20% 8.597 85.638 1.69% 0.059ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [49.294ms; 49.526ms] or [-0.235%; +0.235%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.903µs 3.915µs ± 0.003µs 3.915µs ± 0.002µs 3.917µs 3.920µs 3.921µs 3.922µs 0.16% -0.233 1.487 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255003813.580op/s 255410221.488op/s ± 171819.995op/s 255413580.818op/s ± 119888.238op/s 255533922.143op/s 255649984.741op/s 255707315.763op/s 256227121.325op/s 0.32% 0.240 1.512 0.07% 12149.508op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 79.575µs 80.171µs ± 0.137µs 80.177µs ± 0.054µs 80.229µs 80.297µs 80.402µs 81.252µs 1.34% 1.305 21.740 0.17% 0.010µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12307395.415op/s 12473318.378op/s ± 21270.747op/s 12472480.724op/s ± 8410.896op/s 12481283.383op/s 12496805.948op/s 12552140.996op/s 12566741.967op/s 0.76% -1.193 21.058 0.17% 1504.069op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 67.832µs 67.904µs ± 0.062µs 67.895µs ± 0.024µs 67.923µs 67.987µs 68.019µs 68.575µs 1.00% 6.501 66.244 0.09% 0.004µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 14582582.445op/s 14726669.641op/s ± 13395.429op/s 14728564.987op/s ± 5168.222op/s 14733290.607op/s 14739521.679op/s 14741527.275op/s 14742351.509op/s 0.09% -6.430 65.203 0.09% 947.200op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.895µs 3.915µs ± 0.003µs 3.914µs ± 0.001µs 3.917µs 3.920µs 3.922µs 3.925µs 0.28% -0.819 10.224 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254760760.701op/s 255437888.299op/s ± 191968.720op/s 255471246.718op/s ± 96316.590op/s 255553928.273op/s 255641249.460op/s 255731737.983op/s 256747484.679op/s 0.50% 0.845 10.394 0.07% 13574.238op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 64.609µs 64.705µs ± 0.047µs 64.702µs ± 0.031µs 64.733µs 64.789µs 64.821µs 64.861µs 0.25% 0.389 0.052 0.07% 0.003µs 1 200
credit_card/is_card_number/378282246310005 throughput 15417553.093op/s 15454821.651op/s ± 11141.860op/s 15455354.889op/s ± 7365.950op/s 15462498.418op/s 15471080.142op/s 15476183.747op/s 15477755.370op/s 0.14% -0.385 0.046 0.07% 787.848op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 45.401µs 45.681µs ± 0.088µs 45.691µs ± 0.058µs 45.745µs 45.815µs 45.850µs 45.867µs 0.39% -0.466 0.077 0.19% 0.006µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21802112.874op/s 21890932.899op/s ± 42446.930op/s 21886270.743op/s ± 27882.557op/s 21918565.577op/s 21969545.585op/s 21997743.091op/s 22025822.511op/s 0.64% 0.477 0.091 0.19% 3001.451op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.590µs 6.630µs ± 0.010µs 6.633µs ± 0.004µs 6.636µs 6.641µs 6.644µs 6.648µs 0.22% -1.409 2.109 0.15% 0.001µs 1 200
credit_card/is_card_number/x371413321323331 throughput 150427308.195op/s 150825283.539op/s ± 220515.084op/s 150757122.754op/s ± 82519.625op/s 150912542.427op/s 151223989.194op/s 151515055.151op/s 151754676.004op/s 0.66% 1.419 2.148 0.15% 15592.771op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.896µs 3.914µs ± 0.003µs 3.914µs ± 0.001µs 3.915µs 3.919µs 3.921µs 3.923µs 0.23% -0.845 10.907 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254902203.353op/s 255475355.106op/s ± 173594.804op/s 255490878.976op/s ± 86126.376op/s 255576877.525op/s 255685264.431op/s 255742902.071op/s 256669875.626op/s 0.46% 0.870 11.065 0.07% 12275.006op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.562µs 65.713µs ± 0.052µs 65.712µs ± 0.038µs 65.751µs 65.792µs 65.816µs 65.841µs 0.20% -0.123 -0.360 0.08% 0.004µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15188048.892op/s 15217733.992op/s ± 11973.834op/s 15217848.726op/s ± 8763.856op/s 15226428.183op/s 15237595.571op/s 15243938.323op/s 15252637.510op/s 0.23% 0.127 -0.357 0.08% 846.678op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 53.371µs 53.433µs ± 0.037µs 53.426µs ± 0.022µs 53.450µs 53.513µs 53.547µs 53.557µs 0.25% 1.016 0.988 0.07% 0.003µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18671523.117op/s 18714872.717op/s ± 12958.752op/s 18717545.469op/s ± 7582.109op/s 18723745.207op/s 18731877.780op/s 18735948.182op/s 18736886.671op/s 0.10% -1.012 0.978 0.07% 916.322op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.916µs ± 0.003µs 3.916µs ± 0.002µs 3.918µs 3.920µs 3.921µs 3.923µs 0.18% -1.604 10.272 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254891898.490op/s 255372828.819op/s ± 193971.795op/s 255347028.265op/s ± 134667.052op/s 255510859.779op/s 255631103.299op/s 255726802.060op/s 256730996.177op/s 0.54% 1.626 10.456 0.08% 13715.877op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 50.150µs 50.214µs ± 0.030µs 50.211µs ± 0.021µs 50.235µs 50.271µs 50.284µs 50.296µs 0.17% 0.328 -0.364 0.06% 0.002µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 19882350.169op/s 19914839.320op/s ± 11812.511op/s 19916012.186op/s ± 8508.052op/s 19923985.016op/s 19932666.757op/s 19936354.007op/s 19940245.068op/s 0.12% -0.325 -0.367 0.06% 835.271op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 45.382µs 45.677µs ± 0.092µs 45.672µs ± 0.061µs 45.735µs 45.830µs 45.902µs 45.907µs 0.52% 0.028 0.151 0.20% 0.006µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21782991.306op/s 21892755.702op/s ± 43979.523op/s 21895357.710op/s ± 29222.191op/s 21920740.088op/s 21964477.621op/s 21985839.790op/s 22035130.609op/s 0.64% -0.015 0.156 0.20% 3109.822op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.578µs 6.628µs ± 0.012µs 6.633µs ± 0.005µs 6.636µs 6.643µs 6.649µs 6.651µs 0.28% -1.164 1.656 0.18% 0.001µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 150346613.122op/s 150872113.213op/s ± 278930.401op/s 150767595.153op/s ± 118623.101op/s 151054581.687op/s 151447679.195op/s 151789837.539op/s 152027642.268op/s 0.84% 1.177 1.702 0.18% 19723.358op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.915µs; 3.916µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ throughput [255386408.889op/s; 255434034.087op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [80.152µs; 80.190µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12470370.457op/s; 12476266.299op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [67.895µs; 67.913µs] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/ 378282246310005 throughput [14724813.164op/s; 14728526.119op/s] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/37828224631 execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631 throughput [255411283.281op/s; 255464493.317op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 execution_time [64.698µs; 64.711µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 throughput [15453277.496op/s; 15456365.805op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [45.669µs; 45.693µs] or [-0.027%; +0.027%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [21885050.163op/s; 21896815.635op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.629µs; 6.632µs] or [-0.020%; +0.020%] None None None
credit_card/is_card_number/x371413321323331 throughput [150794722.270op/s; 150855844.809op/s] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.914µs; 3.915µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255451296.536op/s; 255499413.676op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [65.706µs; 65.720µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15216074.534op/s; 15219393.450op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [53.428µs; 53.439µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18713076.759op/s; 18716668.675op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.915µs; 3.916µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255345946.194op/s; 255399711.444op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [50.210µs; 50.218µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [19913202.220op/s; 19916476.421op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [45.665µs; 45.690µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [21886660.563op/s; 21898850.841op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.626µs; 6.630µs] or [-0.026%; +0.026%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [150833456.142op/s; 150910770.284op/s] or [-0.026%; +0.026%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.946µs 25.932µs ± 9.890µs 18.501µs ± 0.409µs 34.888µs 44.871µs 46.043µs 73.198µs 295.65% 1.115 1.301 38.04% 0.699µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.561µs; 27.302µs] or [-5.286%; +5.286%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.189µs 3.185µs ± 1.428µs 2.983µs ± 0.027µs 3.009µs 3.674µs 13.836µs 14.868µs 398.39% 7.397 55.746 44.74% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.987µs; 3.383µs] or [-6.216%; +6.216%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.322µs 2.398µs ± 0.022µs 2.395µs ± 0.007µs 2.407µs 2.439µs 2.448µs 2.451µs 2.34% -0.559 2.276 0.90% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.395µs; 2.401µs] or [-0.125%; +0.125%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 190.907ns 193.229ns ± 2.025ns 192.984ns ± 1.326ns 194.177ns 196.448ns 200.413ns 202.400ns 4.88% 1.512 3.552 1.05% 0.143ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [192.948ns; 193.510ns] or [-0.145%; +0.145%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_frames_x1000 execution_time 4.159ms 4.163ms ± 0.007ms 4.162ms ± 0.001ms 4.164ms 4.166ms 4.169ms 4.250ms 2.11% 11.684 151.590 0.16% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_frames_x1000 execution_time [4.162ms; 4.164ms] or [-0.022%; +0.022%] None None None

Group 18

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 244.933ns 254.636ns ± 11.567ns 249.433ns ± 2.815ns 262.209ns 282.700ns 285.620ns 286.292ns 14.78% 1.554 1.165 4.53% 0.818ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [253.033ns; 256.239ns] or [-0.630%; +0.630%] None None None

Group 19

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ffcdf9b 1773324474 add-datadog-coverage
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 205.936µs 206.707µs ± 0.395µs 206.701µs ± 0.285µs 206.969µs 207.355µs 207.734µs 207.781µs 0.52% 0.419 -0.276 0.19% 0.028µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4812755.206op/s 4837772.838op/s ± 9231.070op/s 4837904.064op/s ± 6678.775op/s 4844741.742op/s 4851555.323op/s 4854056.666op/s 4855866.720op/s 0.37% -0.410 -0.287 0.19% 652.735op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.540µs 18.711µs ± 0.123µs 18.677µs ± 0.072µs 18.791µs 18.949µs 19.059µs 19.216µs 2.88% 1.100 1.585 0.66% 0.009µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 52039806.041op/s 53447794.585op/s ± 348697.717op/s 53541036.280op/s ± 207122.277op/s 53693278.628op/s 53929782.099op/s 53937975.232op/s 53938172.227op/s 0.74% -1.054 1.411 0.65% 24656.652op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.588µs 10.686µs ± 0.053µs 10.676µs ± 0.030µs 10.713µs 10.798µs 10.852µs 10.912µs 2.21% 1.333 2.481 0.50% 0.004µs 1 200
normalization/normalize_name/normalize_name/good throughput 91642421.067op/s 93579629.203op/s ± 465427.187op/s 93671523.285op/s ± 262221.401op/s 93902148.588op/s 94149317.081op/s 94254376.258op/s 94443479.182op/s 0.82% -1.293 2.320 0.50% 32910.672op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [206.653µs; 206.762µs] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4836493.501op/s; 4839052.176op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.694µs; 18.728µs] or [-0.091%; +0.091%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [53399468.435op/s; 53496120.735op/s] or [-0.090%; +0.090%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.679µs; 10.694µs] or [-0.069%; +0.069%] None None None
normalization/normalize_name/normalize_name/good throughput [93515125.471op/s; 93644132.935op/s] or [-0.069%; +0.069%] None None None

Baseline

Omitted due to size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ManuelPalenzuelaDD @codecov-commenter