Skip to content

Benchmark every pull request against its base branch - #1326

Merged
uNetworkingAB merged 4 commits into
uNetworking:masterfrom
nigrosimone:benchmark-ci
Sep 30, 2026
Merged

uNetworkingAB merged 4 commits into
uNetworking:masterfrom
nigrosimone:benchmark-ci

Conversation

@nigrosimone

@nigrosimone nigrosimone commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

A performance PR here comes with numbers measured by its author, on his machine, with his method, and the review can only take them or leave them. This adds a workflow that measures the PR itself, on the runner, against its base branch.

It builds both, starts servers from each build and alternates the load between them scenario by scenario, so both arms see the same machine in the same minute and only the ratio is read. Each arm runs twice, the second time from a copy of its build: base against base and head against head is the noise of that run, and a row is marked only when it moved more than that. Every server reports its own cpu time too, so the table shows how busy it was and what a request cost it. http_load_test from uSockets for the http rows and load_test for the ws rows, both built from the submodule, the server pinned to a core whose hyperthread stays idle.

Scenarios: hello world, headers in and out, the JSON POST of http_load_test through onData, a microcached route, ws echo at 20 B and 4 KB. Same design as brianc/node-postgres#3775.

The table is posted as a comment on the PR by a second workflow on workflow_run, so also on a PR from a fork, where the token of the run cannot comment. workflow_run uses the file on master, so the comment starts after the merge. About 15 minutes per run. Same code on both arms on a hosted runner (nigrosimone#4):

Scenario Base req/s Head req/s Head / base Rounds Noise Busy CPU per request
http/hello-world 107869 108234 1.004x (+0.4%) 1.00 to 1.01 ±1.0% 100% / 100% 9.26 / 9.23 us (0.996x)
http/headers 92553 92745 1.001x (+0.1%) 1.00 to 1.00 ±0.7% 100% / 100% 10.79 / 10.77 us (0.998x)
http/json-post 84469 85234 1.009x (+0.9%) 1.00 to 1.01 ±1.0% 100% / 100% 11.83 / 11.72 us (0.991x)
http/cached 117905 117662 0.999x (-0.1%) 0.99 to 1.00 ±2.0% 100% / 100% 8.48 / 8.50 us (1.000x)
ws/echo-20b 111423 111237 0.997x (-0.3%) 0.99 to 1.00 ±1.4% 100% / 100% 8.97 / 8.98 us (1.003x)
ws/echo-4kb 96290 98132 1.012x (+1.2%) 1.00 to 1.02 ±2.6% 100% / 100% 10.37 / 10.18 us (0.988x)

Node v26.10.0, AMD EPYC 7763, 4 cores (server on cpu 0, load on 2,3), 100 connections.

see https://github.com/uNetworking/uWebSockets.js/actions/runs/36669803181/jobs/89233603871/summary_raw

@uNetworkingAB

Copy link
Copy Markdown
Contributor

I would love it more if it used uSockets http_load_test instead of wrk

@nigrosimone

Copy link
Copy Markdown
Contributor Author

Done: the http rows use http_load_test from uSockets now, built from the submodule without ssl. It always sends GET / (or its POST), so each scenario gets its own servers with the route on "/". The cached route runs no JS, there http_load_test counts the responses. The table in the description is a new same-code run.

# built with it
- name: Build http_load_test
working-directory: head/uWebSockets/uSockets
run: clang-18 -O3 -DLIBUS_NO_SSL -Isrc src/*.c src/eventing/*.c src/crypto/*.c examples/http_load_test.c -o http_load_test

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay either way but this is just "make examples" in uSockets folder otherwise

@uNetworkingAB
uNetworkingAB merged commit 43ae9f7 into uNetworking:master Sep 30, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants