Skip to content

Client throughput: compress and decompress at least as fast as xz #105

Closed

Description

@afflom

Current state

On 1 MB samples, the client from #104 takes 1.7 s to compress and 1.9 s to decompress Rust source, and 5.4 s / 5.3 s for an ELF binary, which is about 0.2–0.6 MB/s. xz -9 compresses at a few MB/s and decompresses at tens of MB/s; gzip is faster still.

Work

  1. Profile the exported Rust (generated/rust/uorc_{compress,decompress}) on the benchmark corpus.
  2. Known hot spots: the SHA-256 chain, the canonical re-encode check on decode, and whole-buffer Bytes copies in the exported code.
  3. Where the fix belongs in LexLean's Rust output, PR it upstream and pin to the patch until it is released.

Done when

On the release benchmark (#51), compress throughput is at least xz -9's and decompress throughput is at least xz -d's.

Metadata

Metadata

Assignees

No one assigned

    Labels

    decoderUORC backlog label: decoderencoderUORC backlog label: encoder

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions