For both StreamingEncoder and StreamingDecoder structs, the config set is the standard one, for example here: let mut decoder = DecoderImpl::new(reader, config::standard());.
From my understanding the standard config uses an algorithm to compress integers in order to use less bytes, making the amount of bytes it uses not constant.
Currently i am working on a project where i need to read the last n elements of a file written with the StreamingEncoder, and to do so the more efficient way would be to get the amount of bytes used to store 1 instance of the element and then using a BufReader seek from the end of the file n * element_bytes_num bytes, if the lenght of the elements variates then it wouldn't be possible to do so.
I feel like adding a field for the config on the StreamingEncoder and StreamingDecoder should solve all of these problems while still mantaining mostly the same code. Basically for the user you will simply end up with access to a new method like new_with_config(reader, config) or something like that.
For example:
fn main() {
let n = u64::MAX;
let p = 0;
let buf = Vec::new();
let mut encoded = StreamingEncoder::new(buf);
encoded.write_item(&n).unwrap();
let size = encoded.finish().unwrap().len();
println!("Encoded usize {n} into {size} bytes");
let buf2 = Vec::new();
let mut encoded = StreamingEncoder::new(buf2);
encoded.write_item(&p).unwrap();
let size = encoded.finish().unwrap().len();
println!("Encoded usize {p} into {size} bytes");
}Outputs this:
Encoded usize 18446744073709551615 into 35 bytes
Encoded usize 0 into 27 bytes
For both
StreamingEncoderandStreamingDecoderstructs, the config set is the standard one, for example here:let mut decoder = DecoderImpl::new(reader, config::standard());.From my understanding the standard config uses an algorithm to compress integers in order to use less bytes, making the amount of bytes it uses not constant.
Currently i am working on a project where i need to read the last n elements of a file written with the
StreamingEncoder, and to do so the more efficient way would be to get the amount of bytes used to store 1 instance of the element and then using aBufReaderseek from the end of the file n * element_bytes_num bytes, if the lenght of the elements variates then it wouldn't be possible to do so.I feel like adding a field for the config on the
StreamingEncoderandStreamingDecodershould solve all of these problems while still mantaining mostly the same code. Basically for the user you will simply end up with access to a new method likenew_with_config(reader, config)or something like that.For example:
Outputs this: