Skip to content

Fix lines left unconverted for NLT type 0 and unsigned type 3 - #421

Merged
aous72 merged 1 commit into
aous72:masterfrom
mardgez:fix-nlt-type0-unconverted
Nov 14, 2026
Merged

aous72 merged 1 commit into
aous72:masterfrom
mardgez:fix-nlt-type0-unconverted

Conversation

@mardgez

@mardgez mardgez commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Since #356, tile::push() and tile::pull() choose a component's sample converter by the type of its NLT record:

  • no record: the plain converter;
  • type 3: the type 3 converter (on the reversible path, for signed samples only);
  • types 2 and 4: the LUT converter.

Before #356, the plain converter was the fallback for everything else. Two cases now match no branch, so no converter runs and the line keeps whatever its buffer held before:

  • A type 0 (no nonlinearity) record. param_nlt::read() accepts one, and set_nonlinear_transform(c, OJPH_NLT_NO_NLT) writes one for a component that should not use the other components' nonlinearity.
  • Type 3 on an unsigned component that uses the reversible path.

When decoding, the output lines of such a component are never written. The application gets the uninitialized contents of the line buffers allocated in codestream::finalize_alloc() instead of the decoded samples, and MemorySanitizer reports their use. When encoding, the codestream is built from the uninitialized contents of the tile's lines.

Change

tile::finalize_alloc() now sets nlt_ptr to NULL in both cases. Every dispatch in push() and pull() then uses the plain converter for them, with and without the colour transform, as before #356. I put the fix in one place instead of adding an else to each of the eight dispatches, because the reversible ones would each need the sign-dependent shift repeated. Irreversible type 3 is left unchanged.

Tests (tests/test_nlt.cpp)

  • TypeZeroRecordOfOneComponentKeepsSamples: all components get type 3 except one, which gets type 0. That component decodes to the same samples as without a nonlinearity, and the reversible round trip is lossless.
  • TypeZeroMarkerSegmentKeepsSamples: a codestream with an inserted type 0 NLT marker segment decodes to the same samples as without it, with both wavelets.
  • BinaryComplementOfUnsignedSamplesIsLossless: a reversible round trip of 8-bit unsigned samples with type 3 is lossless. It is checked with and without the colour transform, and with and without one component on type 0.

When the colour transform is employed, the test's encode and decode helpers now exchange lines interleaved, as the library requires.

All three tests fail without the change; for example, more than 12230 of 12288 samples are wrong in every case of the last test. With the change they pass, and the whole ctest suite passes (138/138). test_nlt also passes under MemorySanitizer.

This is the change of #408, which is no longer reachable, rebased on the current master.

Since aous72#356, tile::push() and tile::pull() pick the sample converter of a
component by the type of its NLT record: the plain converter when there
is no record, the type 3 converter for type 3 (for signed samples only
on the reversible path), and the LUT converter for types 2 and 4. Before
cases now match no branch, so no converter runs and the line keeps
whatever its buffer held before:

- a type 0 (no nonlinearity) record, which param_nlt::read() accepts and
  set_nonlinear_transform(c, OJPH_NLT_NO_NLT) writes for a component
  that should not use the nonlinearity of the other components;
- type 3 on an unsigned component that uses the reversible path.

On decoding, the output lines of such a component are never written, so
the application receives the uninitialized contents of the line buffers
allocated in codestream::finalize_alloc() instead of the decoded
samples; MemorySanitizer reports their use. On encoding, the codestream
is built from the uninitialized contents of the tile's lines.

Set nlt_ptr to NULL for both cases in tile::finalize_alloc(), so that
every dispatch in push() and pull(), with and without the colour
transform, uses the plain converter for them, as before aous72#356.

Add three tests to tests/test_nlt.cpp:
- TypeZeroRecordOfOneComponentKeepsSamples: type 3 for all components
  and type 0 for one of them; that component decodes to the same samples
  as without a nonlinearity, and the reversible round trip is lossless.
- TypeZeroMarkerSegmentKeepsSamples: a codestream with an inserted type
  0 NLT marker segment decodes to the same samples as without it.
- BinaryComplementOfUnsignedSamplesIsLossless: a reversible round trip
  of 8-bit unsigned samples with type 3 is lossless, with and without
  the colour transform and with and without a type 0 component.
The encoding and decoding helpers of the test use interleaved lines when
the colour transform is employed, as the library requires.

All three tests fail without this change and pass with it; the rest of
the test suite passes.
@aous72

aous72 commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Hi Mar,

Thank you for this PR. It was an oversight.
The ultimate plan is to move all NLT processing to one function in the respective file, ojph_colour.cpp, ojph_colour_ss2.cpp, ... etc.
The proposed solution is the best for the time being.

Kind regards,
Aous.

@aous72
aous72 merged commit 97a9579 into aous72:master Nov 14, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants