Skip to content

switch compression in zip archives to deflate#3315

Merged
syphar merged 1 commit intorust-lang:mainfrom
syphar:deflate
Apr 27, 2026
Merged

switch compression in zip archives to deflate#3315
syphar merged 1 commit intorust-lang:mainfrom
syphar:deflate

Conversation

@syphar
Copy link
Copy Markdown
Member

@syphar syphar commented Apr 26, 2026

Up to discussion of course.

I realized:

  • creating the archives can be quite slow, and the main issue is bzip compression speed
  • Right now the webserver creates quite some CPU load, I guess partially because of bzip2 decompression.

Storage size is either nearly the same, at most deflate archives is 5-10% bigger.

After fixing the decompression benchmark we also were able to see the actual throughputs.

Then looking into benches:

regex html/compress bzip2
                        time:   [1.6310 s 1.6630 s 1.7063 s]
                        thrpt:  [6.0118 MiB/s 6.1685 MiB/s 6.2893 MiB/s]
                 change:
                        time:   [+0.4105% +2.4544% +5.4159%] (p = 0.06 > 0.05)
                        thrpt:  [−5.1376% −2.3956% −0.4088%]
regex html/decompress bzip2
                        time:   [103.30 ms 105.30 ms 107.35 ms]
                        thrpt:  [95.560 MiB/s 97.417 MiB/s 99.305 MiB/s]
                 change:
                        time:   [+828204% +841346% +854357%] (p = 0.00 < 0.05)
                        thrpt:  [−99.988% −99.988% −99.988%]
                        Performance has regressed.
regex html/compress deflate
                        time:   [66.063 ms 66.796 ms 67.226 ms]
                        thrpt:  [152.59 MiB/s 153.57 MiB/s 155.28 MiB/s]
                 change:
                        time:   [−2.8354% −1.1810% +0.2685%] (p = 0.19 > 0.05)
                        thrpt:  [−0.2677% +1.1951% +2.9181%]
regex html/decompress deflate
                        time:   [11.182 ms 11.482 ms 11.840 ms]
                        thrpt:  [866.41 MiB/s 893.38 MiB/s 917.35 MiB/s]
                 change:
                        time:   [+73719% +75869% +78203%] (p = 0.00 < 0.05)
                        thrpt:  [−99.872% −99.868% −99.865%]

I probably should have some more thinking / research / measurement on compression/decompression speed, and what the impact onto our webserver could be.

@syphar syphar self-assigned this Apr 26, 2026
@syphar syphar requested a review from a team as a code owner April 26, 2026 23:19
@github-actions github-actions Bot added the S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed label Apr 26, 2026
@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented Apr 26, 2026

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@syphar syphar merged commit 30cf20c into rust-lang:main Apr 27, 2026
11 checks passed
@syphar syphar deleted the deflate branch April 27, 2026 00:11
@github-actions github-actions Bot added S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants