Skip to content

feat: add KPAR compression options#186

Merged
Jonas-Puksta-Sensmetry merged 20 commits intomainfrom
jp/kpar-compression
Mar 13, 2026
Merged

feat: add KPAR compression options#186
Jonas-Puksta-Sensmetry merged 20 commits intomainfrom
jp/kpar-compression

Conversation

@Jonas-Puksta-Sensmetry
Copy link
Collaborator

@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry commented Feb 27, 2026

Addresses #174.
Add bzip2, zstd, xz, and ppmd KPAR compression methods gated behind feature flags kpar-bzip2, kpar-zstd, kpar-xz, and kpar-ppmd respectively.

@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry changed the title Add Kpar compression options Feat: add Kpar compression options Feb 27, 2026
@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry changed the title Feat: add Kpar compression options Feat: add KPAR compression options Feb 27, 2026
@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry changed the title Feat: add KPAR compression options feat: add KPAR compression options Feb 27, 2026
Jonas-Puksta-Sensmetry and others added 10 commits February 27, 2026 18:18
…ndings are incomplete so far

Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Signed-off-by: Jonas Pukšta <146448971+Jonas-Puksta-Sensmetry@users.noreply.github.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
@consideRatio consideRatio force-pushed the jp/kpar-compression branch from 7a6ff9f to 1ce5ab8 Compare March 2, 2026 11:34
@consideRatio
Copy link
Collaborator

@Jonas-Puksta-Sensmetry I haven't really looked at this PR besides the CI failure, but with @andrius-puksta-sensmetry approving review and the commit I pushed to fix a real failure, I think you should merge if you found 1ce5ab8 to be OK!

Signed-off-by: Erik Sundell <erik.i.sundell@gmail.com>
@consideRatio
Copy link
Collaborator

@victor-linroth-sensmetry does 1ce5ab8 look okay to you? Should we go for a merge? Eager to reduce the amount of open PRs ;D

@victor-linroth-sensmetry
Copy link
Collaborator

I'll see if I have time to look at it later.

Signed-off-by: Erik Sundell <erik.i.sundell@gmail.com>
@consideRatio
Copy link
Collaborator

I lack the background to this PR and wasn't part of the discussion leading up to it, sorry for the late input! I'm not looking for a discussion to onboard me, but for a quick self-check that you still think it's good to do this.

  1. Flexibility for one user packaging something imposes more requirements on the user unpackaging it. Is supporting more flexibility actually positive?
  2. The .zip archives uploaded to PyPI, etc., for wheels and source code are constrained for security reasons, I recall. I recall security fixes to protect PyPI users from unsafe uploads, where during extraction there could be mischief like unzipping to relative paths outside the folder you meant to extract to, potentially overriding .bashrc or something like that. Are we feeling confident that this isn't causing issues like that?

@andrius-puksta-sensmetry
Copy link
Collaborator

Flexibility for one user packaging something imposes more requirements on the user unpackaging it. Is supporting more flexibility actually positive?

This is why we by default will support only no compression and deflate (which is used ~everywhere by default). Other compression methods are exposed to enable benchmarking and possibly choosing a different default in the future, depending on how well they compress SysML2 JSON representation, as that representation tends to be huge.

The .zip archives uploaded to PyPI, etc., for wheels and source code are constrained for security reasons, I recall. I recall security fixes to protect PyPI users from unsafe uploads, where during extraction there could be mischief like unzipping to relative paths outside the folder you meant to extract to, potentially overriding .bashrc or something like that. Are we feeling confident that this isn't causing issues like that?

I don't see how this would affect security. It only changes the compression algorithm, metadata AFAIK remains exposed in exactly the same way.

For more details/rationale ask Vytautas.

@andrius-puksta-sensmetry andrius-puksta-sensmetry marked this pull request as draft March 10, 2026 09:30
…odule is conditional on filesystem feature

Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
Signed-off-by: jonas.puksta.sensmetry <jonas.puksta@sensmetry.com>
@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry marked this pull request as ready for review March 13, 2026 07:31
@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry merged commit 1aeb7f4 into main Mar 13, 2026
56 checks passed
@Jonas-Puksta-Sensmetry Jonas-Puksta-Sensmetry deleted the jp/kpar-compression branch March 13, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants