Regression in decompressing bad data using miniz_oxide>=0.8.5 w/ small input

[milesgranger]

Copying the issue from miniz_oxide: Frommi/miniz_oxide#174

xref: milesgranger/cramjam#211

Seems a decompression error only arrives if the bad input data is over 4 characters long:

This does not raise an error:

    fn decompress_bad_data() {
        let mut decoder = DeflateDecoder::new(b"1".as_slice());
        let mut out = vec![];
        assert!(std::io::copy(&mut decoder, &mut out).is_err());
    }

But this will, 5 bytes at least for me, seems to spark the error, up to 4 bytes will not raise an error.

    fn decompress_bad_data() {
        let mut decoder = DeflateDecoder::new(b"12345".as_slice());
        let mut out = vec![];
        assert!(std::io::copy(&mut decoder, &mut out).is_err());
    }

---

And as pointed out in the linked issue, it actually depends on what the first byte is on whether it fails as expected or not - further adding to the awkwardness.

[Byron]

Thanks a lot for reporting!

I think the key here is this comment from the linked miniz_oxide issue which I partially quote:

Your issue is with how flate2 treats deflate streams that are decoding fine but are incomplete - it treats those as valid and discards the error for those streams returned by the various backends. As stated earlier, flate2 would need some additions if you want to be able to distinguish incomplete streams from valid ones.

It looks like that tests in that direction are missing or could be improved in order to figure out how be able to differentiate legitimate broken small streams properly.

There should also be a differentiation between short sequences of bytes which legitimately could be valid, and those which never can.

Thus, it sounded like "1" is actually valid, but of course, trying to copy everything like in the example should result in an error.

More resources:

• playground for first example that should work
• playground for second example that actually works.