Skip to content

chore: prepare v0.13.0#2549

Merged
larseggert merged 1 commit intomozilla:mainfrom
mxinden:prepare-v0.13.0
Apr 7, 2025
Merged

chore: prepare v0.13.0#2549
larseggert merged 1 commit intomozilla:mainfrom
mxinden:prepare-v0.13.0

Conversation

@mxinden
Copy link
Copy Markdown
Member

@mxinden mxinden commented Apr 1, 2025

Bugfixes I would like to wait for:

Any objections? Anything else to include?

@larseggert
Copy link
Copy Markdown
Collaborator

Maybe #2470 and/or #2505?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 1, 2025

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 1, 2025

Benchmark results

Performance differences relative to d947c94.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: 💚 Performance has improved.
       time:   [700.55 ms 705.48 ms 710.33 ms]
       thrpt:  [140.78 MiB/s 141.75 MiB/s 142.75 MiB/s]
change:
       time:   [-3.6796% -2.7335% -1.7463%] (p = 0.00 < 0.05)
       thrpt:  [+1.7773% +2.8103% +3.8202%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [348.48 ms 350.23 ms 351.97 ms]
       thrpt:  [28.412 Kelem/s 28.552 Kelem/s 28.696 Kelem/s]
change:
       time:   [-0.8573% -0.1160% +0.6344%] (p = 0.76 > 0.05)
       thrpt:  [-0.6304% +0.1161% +0.8647%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [25.686 ms 25.843 ms 25.998 ms]
       thrpt:  [38.465  elem/s 38.696  elem/s 38.932  elem/s]
change:
       time:   [-1.7789% -0.8794% +0.0043%] (p = 0.05 > 0.05)
       thrpt:  [-0.0043% +0.8872% +1.8111%]
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: 💚 Performance has improved.
       time:   [2.2949 s 2.3162 s 2.3376 s]
       thrpt:  [42.780 MiB/s 43.175 MiB/s 43.575 MiB/s]
change:
       time:   [-5.2546% -4.0278% -2.8255%] (p = 0.00 < 0.05)
       thrpt:  [+2.9076% +4.1968% +5.5461%]
decode 4096 bytes, mask ff: Change within noise threshold.
       time:   [12.069 µs 12.099 µs 12.137 µs]
       change: [+0.0380% +0.5587% +1.0865%] (p = 0.03 < 0.05)

Found 18 outliers among 100 measurements (18.00%)
6 (6.00%) low severe
4 (4.00%) low mild
1 (1.00%) high mild
7 (7.00%) high severe

decode 1048576 bytes, mask ff: 💔 Performance has regressed.
       time:   [3.0747 ms 3.0836 ms 3.0935 ms]
       change: [+3.8959% +4.3469% +4.8098%] (p = 0.00 < 0.05)

Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) high mild
7 (7.00%) high severe

decode 4096 bytes, mask 7f: Change within noise threshold.
       time:   [20.207 µs 20.270 µs 20.335 µs]
       change: [+0.6421% +1.0114% +1.3731%] (p = 0.00 < 0.05)

Found 25 outliers among 100 measurements (25.00%)
3 (3.00%) low severe
2 (2.00%) low mild
1 (1.00%) high mild
19 (19.00%) high severe

decode 1048576 bytes, mask 7f: 💔 Performance has regressed.
       time:   [5.2538 ms 5.2669 ms 5.2805 ms]
       change: [+9.1750% +9.6070% +10.049%] (p = 0.00 < 0.05)

Found 15 outliers among 100 measurements (15.00%)
15 (15.00%) high severe

decode 4096 bytes, mask 3f: 💔 Performance has regressed.
       time:   [7.0161 µs 7.0455 µs 7.0823 µs]
       change: [+10.386% +11.250% +12.392%] (p = 0.00 < 0.05)

Found 13 outliers among 100 measurements (13.00%)
2 (2.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe

decode 1048576 bytes, mask 3f: 💚 Performance has improved.
       time:   [1.7908 ms 1.7964 ms 1.8034 ms]
       change: [-17.179% -16.767% -16.352%] (p = 0.00 < 0.05)

Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) high mild
5 (5.00%) high severe

1 streams of 1 bytes/multistream: No change in performance detected.
       time:   [71.474 µs 71.678 µs 71.886 µs]
       change: [-2.3706% -0.3158% +1.1986%] (p = 0.78 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1000 streams of 1 bytes/multistream: Change within noise threshold.
       time:   [25.359 ms 25.403 ms 25.448 ms]
       change: [-0.4806% -0.2471% -0.0080%] (p = 0.04 < 0.05)
10000 streams of 1 bytes/multistream: No change in performance detected.
       time:   [1.7010 s 1.7030 s 1.7050 s]
       change: [-0.3082% -0.1366% +0.0223%] (p = 0.11 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
6 (6.00%) high mild

1 streams of 1000 bytes/multistream: No change in performance detected.
       time:   [73.075 µs 74.125 µs 75.617 µs]
       change: [-2.6924% +0.1566% +3.0273%] (p = 0.92 > 0.05)

Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe

100 streams of 1000 bytes/multistream: Change within noise threshold.
       time:   [3.4123 ms 3.4195 ms 3.4276 ms]
       change: [+0.5495% +0.8292% +1.1091%] (p = 0.00 < 0.05)

Found 21 outliers among 100 measurements (21.00%)
21 (21.00%) high severe

1000 streams of 1000 bytes/multistream: Change within noise threshold.
       time:   [145.92 ms 146.00 ms 146.09 ms]
       change: [+0.0069% +0.0861% +0.1648%] (p = 0.03 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [94.731 ns 95.019 ns 95.310 ns]
       change: [-0.7920% -0.1285% +0.5555%] (p = 0.73 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
11 (11.00%) high mild
2 (2.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [112.52 ns 112.86 ns 113.22 ns]
       change: [-0.6928% -0.2842% +0.1482%] (p = 0.21 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
2 (2.00%) low mild
1 (1.00%) high mild
12 (12.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [111.74 ns 112.00 ns 112.34 ns]
       change: [-0.8697% -0.2636% +0.4776%] (p = 0.44 > 0.05)

Found 21 outliers among 100 measurements (21.00%)
3 (3.00%) low severe
5 (5.00%) low mild
3 (3.00%) high mild
10 (10.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [93.970 ns 94.380 ns 94.822 ns]
       change: [-2.0716% -0.8335% +0.3671%] (p = 0.19 > 0.05)

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

RxStreamOrderer::inbound_frame(): 💚 Performance has improved.
       time:   [114.61 ms 114.67 ms 114.73 ms]
       change: [-2.2983% -2.2201% -2.1507%] (p = 0.00 < 0.05)

Found 20 outliers among 100 measurements (20.00%)
4 (4.00%) low severe
5 (5.00%) low mild
11 (11.00%) high mild

SentPackets::take_ranges: Change within noise threshold.
       time:   [8.3622 µs 8.5685 µs 8.7607 µs]
       change: [+0.9506% +4.3454% +7.9817%] (p = 0.01 < 0.05)

Found 16 outliers among 100 measurements (16.00%)
3 (3.00%) low severe
11 (11.00%) low mild
2 (2.00%) high severe

transfer/pacing-false/varying-seeds: Change within noise threshold.
       time:   [34.509 ms 34.570 ms 34.630 ms]
       change: [-2.8093% -2.5508% -2.3016%] (p = 0.00 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [34.612 ms 34.667 ms 34.722 ms]
       change: [-2.1063% -1.8915% -1.6701%] (p = 0.00 < 0.05)
transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [34.330 ms 34.386 ms 34.443 ms]
       change: [-2.9999% -2.7929% -2.5835%] (p = 0.00 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [35.039 ms 35.086 ms 35.132 ms]
       change: [-3.1684% -2.9814% -2.7807%] (p = 0.00 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild

Client/server transfer results

Performance differences relative to d947c94.

Transfer of 33554432 bytes over loopback, 30 runs. All unit-less numbers are in milliseconds.

Client Server CC Pacing Mean ± σ Min Max MiB/s ± σ Δ main Δ main
neqo neqo reno on 413.4 ± 37.6 374.4 588.8 77.4 ± 0.9 0.3 0.1%
neqo neqo reno 436.9 ± 137.4 366.5 1121.2 73.2 ± 0.2 -18.5 -4.1%
neqo neqo cubic on 413.9 ± 35.3 388.2 585.2 77.3 ± 0.9 -2.3 -0.5%
neqo neqo cubic 408.7 ± 19.8 375.7 484.0 78.3 ± 1.6 -1.4 -0.3%
google neqo reno on 758.0 ± 77.4 560.4 839.0 42.2 ± 0.4 -1.3 -0.2%
google neqo reno 756.3 ± 82.1 558.7 834.9 42.3 ± 0.4 -4.9 -0.6%
google neqo cubic on 754.9 ± 89.0 545.4 952.1 42.4 ± 0.4 -0.9 -0.1%
google neqo cubic 751.3 ± 82.7 567.9 857.8 42.6 ± 0.4 -4.0 -0.5%
google google 574.8 ± 38.7 545.3 756.3 55.7 ± 0.8 8.9 1.6%
neqo msquic reno on 265.9 ± 20.6 246.1 347.0 120.3 ± 1.6 -7.3 -2.7%
neqo msquic reno 264.0 ± 13.0 245.9 292.4 121.2 ± 2.5 -4.1 -1.5%
neqo msquic cubic on 265.6 ± 15.3 246.2 316.6 120.5 ± 2.1 -0.3 -0.1%
neqo msquic cubic 264.3 ± 12.8 244.0 293.8 121.1 ± 2.5 -6.6 -2.4%
msquic msquic 171.7 ± 21.5 149.7 225.6 186.4 ± 1.5 -9.2 -5.1%

⬇️ Download logs

@mxinden
Copy link
Copy Markdown
Member Author

mxinden commented Apr 7, 2025

Maybe #2470 and/or #2505?

Both merged. @larseggert fine if I proceed here?

@larseggert
Copy link
Copy Markdown
Collaborator

Go for it.

@mxinden mxinden marked this pull request as ready for review April 7, 2025 14:43
@larseggert larseggert added this pull request to the merge queue Apr 7, 2025
Merged via the queue into mozilla:main with commit 00646f0 Apr 7, 2025
119 of 121 checks passed
@larseggert
Copy link
Copy Markdown
Collaborator

larseggert commented Apr 7, 2025

@mxinden we should also include #2568!

@mxinden
Copy link
Copy Markdown
Member Author

mxinden commented Apr 8, 2025

For the record, downstream tracking issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1959128

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants