Read deep sample-count table little-endian in testReadDeep#2509
Read deep sample-count table little-endian in testReadDeep#2509Scottcjn wants to merge 2 commits into
Conversation
testReadDeep reinterpreted the raw deep sample-count table as host-order uint32. exr_read_deep_chunk returns the table as stored on disk, and the OpenEXR format stores it little-endian, so on big-endian hosts the count came out byte-reversed and the packed.size () == sampcount * bps check aborted (AcademySoftwareFoundation#1175). Read the last sample count as explicit little-endian bytes instead. This is byte-order independent: the test passes on big-endian and is unchanged on little-endian. Verified on POWER8: OpenEXRCore.testReadDeep passes on native big-endian ppc64 and on ppc64le. Signed-off-by: Scott Boudreaux <121303252+Scottcjn@users.noreply.github.com>
|
|
|
I think the library should be returning you data in your native format (which should be true for normal pixel data once you've run unpack), so I would say the more appropriate fix is to put a priv_to_native32 call in the read_deep_chunk function after it has successfully read, so at line 1642 of chunk.c (after the success check): if (sample_data) priv_to_native32 (sample_data, cinfo->sample_count_data_size / sizeof(uint32_t)); or so, allowing immediate use of the sample count table values. Supporting the two read modes may get confusing to check that this is the correct action... There is likely to be the inverse issues when writing, but we can get there later |
The on-disk deep sample-count table is little-endian. exr_read_deep_chunk returned it raw, so callers on big-endian hosts received byte-reversed counts (AcademySoftwareFoundation#1175). Convert the table with priv_to_native32 after a successful read -- matching how the scanline/tile read paths already normalise data -- so callers receive host-order counts consistent with unpacked pixel data. With the library returning native data, revert the testReadDeep workaround to the straightforward reinterpret_cast read: it now passes on big-endian for the right reason and is unchanged on little-endian. Addresses review feedback on AcademySoftwareFoundation#2509 (move the fix into the library rather than the test). Signed-off-by: Scott Boudreaux <121303252+Scottcjn@users.noreply.github.com>
|
Thanks @kdt3rd — agreed, the library should hand back native-order counts. Moved the fix into One small note — the field is With the library returning native data, I reverted the |
Addresses one of the big-endian test failures in #1175.
testReadDeepreinterprets the raw deep sample-count table as a host-orderuint32_t.exr_read_deep_chunkreturns that table as stored on disk, and the OpenEXR format stores it little-endian, so on big-endian hosts the count is byte-reversed and thepacked.size () == sampcount[N-1] * bpscheck aborts.This reads the last sample count as explicit little-endian bytes instead, which is byte-order independent (the same value on both endiannesses). It follows the little-endian convention already used by
Imf::Xdrand the Core's*_to_native32helpers, and it matches the "little endian assumptions exist in the tests" point raised in the issue thread.Testing
Built and tested on real POWER8 hardware:
OpenEXRCore.testReadDeepfails before this change (raw little-endian table read as host order) and passes after.OpenEXRCore.testReadDeeppasses, unchanged.This is a test-only change; no library or file-format behavior is affected. The remaining big-endian failures in #1175 (the
testAttributessegfault and the DWA-compression tests) are separate and not touched here.