Repository navigation
fix: bzip2 identify requires the stream block magic, not just "BZh<n>" - #61
Merged
Merged
Conversation
The bzip2 signature validated only four bytes: the "BZh" magic plus an ASCII block-size digit (1-9). In a multi-megabyte opaque or compressed blob a chance "BZh<n>" sequence satisfies that and was reported as a bzip2 finding, even though the bytes are not a bzip2 stream and fail to decompress. Require the 48-bit stream block magic that follows the 4-byte header: the first compressed block's 0x314159265359, or the end-of-stream 0x177245385090 for a stream with no blocks. That lifts the match from 4 bytes to 10 and removes the false-positive class while still identifying real streams (empty streams included). Still structural tier; no decode step added. Adds tests/test_bzip2.py: real and empty streams identified at offset 0 and at a nonzero offset; "BZh<n>" + non-magic and random post-header blobs rejected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
bzip2signature matched on four bytes only:BZhplus an ASCII block-sizedigit
1-9. A chanceBZh<n>sequence in opaque or compressed data (forexample a modem flash image) satisfied that structural test and was reported as
a bzip2 finding, even though the bytes are not a bzip2 stream and fail to
decompress.
Fix
A real bzip2 stream carries a 48-bit block magic immediately after the 4-byte
header:
0x314159265359for the first compressed block, or the end-of-streammagic
0x177245385090when the stream has no blocks. The signature now readsthose six bytes and requires one of the two, lifting the match from 4 bytes to
10. bzip2 stays at
structuraltier (no decode step); the change only removesthe false-positive class.
Result
BZh7-plus-non-magic sequence in an opaque flash image is no longeridentified as bzip2 (the spurious finding drops from the summary).
offset 0 and at nonzero offsets.
Tests
tests/test_bzip2.py(wired intorun.sh): real and empty streams match;BZh<n>-plus-non-magic and 64 random post-header blobs are rejected.synthetic bzip2 sample already carries the block magic).