Skip to content

Read deep sample-count table little-endian in testReadDeep#2509

Open
Scottcjn wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
Scottcjn:bugfix/be-deep-sample-count-test
Open

Read deep sample-count table little-endian in testReadDeep#2509
Scottcjn wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
Scottcjn:bugfix/be-deep-sample-count-test

Conversation

@Scottcjn

Copy link
Copy Markdown
Contributor

Addresses one of the big-endian test failures in #1175.

testReadDeep reinterprets the raw deep sample-count table as a host-order uint32_t. exr_read_deep_chunk returns that table as stored on disk, and the OpenEXR format stores it little-endian, so on big-endian hosts the count is byte-reversed and the packed.size () == sampcount[N-1] * bps check aborts.

This reads the last sample count as explicit little-endian bytes instead, which is byte-order independent (the same value on both endiannesses). It follows the little-endian convention already used by Imf::Xdr and the Core's *_to_native32 helpers, and it matches the "little endian assumptions exist in the tests" point raised in the issue thread.

Testing

Built and tested on real POWER8 hardware:

  • ppc64 big-endian (gcc 13.3): OpenEXRCore.testReadDeep fails before this change (raw little-endian table read as host order) and passes after.
  • ppc64le (gcc 10): OpenEXRCore.testReadDeep passes, unchanged.

This is a test-only change; no library or file-format behavior is affected. The remaining big-endian failures in #1175 (the testAttributes segfault and the DWA-compression tests) are separate and not touched here.

testReadDeep reinterpreted the raw deep sample-count table as
host-order uint32. exr_read_deep_chunk returns the table as stored on
disk, and the OpenEXR format stores it little-endian, so on big-endian
hosts the count came out byte-reversed and the
packed.size () == sampcount * bps check aborted (AcademySoftwareFoundation#1175).

Read the last sample count as explicit little-endian bytes instead.
This is byte-order independent: the test passes on big-endian and is
unchanged on little-endian.

Verified on POWER8: OpenEXRCore.testReadDeep passes on native
big-endian ppc64 and on ppc64le.

Signed-off-by: Scott Boudreaux <121303252+Scottcjn@users.noreply.github.com>
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 26, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: Scottcjn / name: Scott Boudreaux (3bb6870)

@kdt3rd

kdt3rd commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

I think the library should be returning you data in your native format (which should be true for normal pixel data once you've run unpack), so I would say the more appropriate fix is to put a priv_to_native32 call in the read_deep_chunk function after it has successfully read, so at line 1642 of chunk.c (after the success check):

if (sample_data) priv_to_native32 (sample_data, cinfo->sample_count_data_size / sizeof(uint32_t));

or so, allowing immediate use of the sample count table values. Supporting the two read modes may get confusing to check that this is the correct action...

There is likely to be the inverse issues when writing, but we can get there later

The on-disk deep sample-count table is little-endian. exr_read_deep_chunk
returned it raw, so callers on big-endian hosts received byte-reversed
counts (AcademySoftwareFoundation#1175). Convert the table with priv_to_native32 after a successful
read -- matching how the scanline/tile read paths already normalise data --
so callers receive host-order counts consistent with unpacked pixel data.

With the library returning native data, revert the testReadDeep workaround
to the straightforward reinterpret_cast read: it now passes on big-endian
for the right reason and is unchanged on little-endian.

Addresses review feedback on AcademySoftwareFoundation#2509 (move the fix into the library rather
than the test).

Signed-off-by: Scott Boudreaux <121303252+Scottcjn@users.noreply.github.com>
@Scottcjn

Copy link
Copy Markdown
Contributor Author

Thanks @kdt3rd — agreed, the library should hand back native-order counts. Moved the fix into read_deep_chunk: after the sample-count table read succeeds it now runs priv_to_native32 over the table (no-op on little-endian, le32toh on big-endian), matching how the scanline/tile read paths already normalise data, so callers receive host-order counts consistent with unpacked pixel data.

One small note — the field is cinfo->sample_count_table_size (your line referenced sample_count_data_size), so the count is sample_count_table_size / sizeof(uint32_t).

With the library returning native data, I reverted the testReadDeep workaround back to the straightforward reinterpret_cast read — it now passes on big-endian for the right reason and is unchanged on little-endian. Agreed there is likely the inverse issue on write; happy to follow that up separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants