Repository navigation
gif/png: size findings to their real terminator, not to EOF (#58) - #59
Merged
Merged
Conversation
GIF and PNG were magic-only signatures with no size expression and no validator, so a finding's size stayed unknown and resolve() back-filled it to the next finding or EOF. A small image embedded in a larger file was therefore reported as spanning everything after it (GIF over-reported ~2860x, PNG ~1450x on the issue's repro), mislabeling the trailing bytes as image in the map, letting a few-byte asset dominate the whole-file assessment, and making -c/-e carve unrelated data. jpeg and bmp already compute their real extent; gif and png were the outliers. Add a validator for each, mirroring jpeg: - png: walk length|type|data|crc chunks from the IHDR to IEND and set the size. Verify each chunk's CRC-32; a full walk with every CRC valid reports verified, otherwise consistent. - gif: walk image-descriptor / extension blocks and their sub-block chains to the 0x3B trailer and set the size (consistent; GIF has no checksum). Both only size, they never drop a real image: the PNG signature+IHDR magic and the GIF magic are decisive, so a truncated-but-valid image keeps its finding and falls back to size inference. The gif validator additionally rejects clear false positives of the weak 6-byte magic: after the header and global color table a real GIF must begin with a block introducer (0x2C/0x21/0x3B), and anything else is not a GIF. A truncated real GIF still has a valid first introducer, so it survives the gate. Verification: the issue repro is fixed (GIF 100035 -> 28, PNG 100069 -> 69). Across a firmware corpus, 103 GIF and 295 PNG findings now size exactly to their terminator (carved bytes end at 0x3B / IEND), and the gif gate drops 12 bogus magic-only matches. The handful that still fall back are genuinely unresolvable (a corrupt PNG whose IDAT CRC fails with no reachable IEND, and GIF false positives whose LZW image data does not decode) and were mis-sized before this change too. Full test suite green; ASan/UBSan clean on the new test, the fixtures, and every corpus file containing a gif/png, for both identify and extract. New tests/test_image_size.py is a hard gate: it reproduces the over-sizing (fails without the validators) and also covers embedded-at-offset sizing, a false-positive GIF rejection, and that a truncated real image is still identified rather than dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #58. GIF and PNG were magic-only signatures with no size expression and no validator, so a finding's size stayed unknown and
resolve()back-filled it to the next finding or EOF. A small image embedded in a larger file was reported as spanning everything after it (GIF over-reported ~2860x, PNG ~1450x on the issue's repro). That mislabels trailing bytes as image in the map, lets a few-byte asset dominate the whole-file assessment, and makes-c/-ecarve unrelated data.jpegandbmpalready compute their real extent;gifandpngwere the outliers.Fix
Two validators, mirroring the
jpegone:length | type | data | crcchunks from the IHDR toIENDand set the size. Each chunk's CRC-32 is checked; a full walk with every CRC valid reportsverified, otherwiseconsistent.0x3Btrailer and set the size (consistent— GIF has no checksum to reachverified).Both only size, they never drop a real image. The PNG signature+IHDR magic (16 bytes) and the GIF magic are decisive, so a truncated-but-valid image keeps its finding and falls back to size inference.
The
gifvalidator additionally rejects clear false positives of the weak 6-byte magic: after the header and global color table, a real GIF must begin with a block introducer (0x2C/0x21/0x3B), and anything else is not a GIF. A truncated real GIF still has a valid first introducer, so it survives the gate. This kills the same "a strayGIF89ain binary data swallows the rest of the file" symptom from the false-positive direction.Verification
0x3B/IEND), and the gif gate drops 12 bogus magic-only matches.Regression test
tests/test_image_size.pyis a hard gate (self-contained, stdlibzlib/struct, no external tool). It reproduces the over-sizing (fails without the validators), and covers embedded-at-offset sizing, a false-positive GIF rejection, and that a truncated real image is still identified rather than dropped.