Skip to content

Converter map validation: check every axis, also after an empty one - #87

Merged
phantoslayer21 merged 1 commit into
mainfrom
fix/54-converter-map-axes
Nov 13, 2026
Merged

phantoslayer21 merged 1 commit into
mainfrom
fix/54-converter-map-axes

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Fixes #54

What was wrong

check_group_reads_v5 (clean/converter/src/map.cpp) stopped checking a tensor's axes as soon as one axis had an effective length of 0 (limit: 0). The axes after it never reached the checked_mul / checked_add overflow checks, so a map with, for example, axes [{stride: 1, limit: 0}, {stride: 9223372036854775807}] loaded, and Decoder::indexed (clean/converter/src/convert5.cpp) would then compute stride * index in signed 64-bit and overflow.

There is no visible effect today: the zero-limit axis already makes the tensor all zeros and every source read stays bounds-checked, and it needs a hand-made map. But the map is meant to be validated completely before any input is read, and this was a gap in that.

The fix

The loop over the tensor's axes no longer breaks at an empty axis; it records that the tensor reads nothing (any = false) and continues, so every non-empty axis goes through the checked multiplication and addition. An empty axis still adds no span, and a tensor that reads nothing still skips the source-range check afterwards, as before. No other change.

The test

A new mutation in corrupt_map_cases (clean/converter/tests/run_tests5.py, next to the existing ones): on the tensor G1/b of the synthetic feature map, axis 0 gets limit: 0 and axis 1 gets stride: 9223372036854775807. It must be refused as a damaged map (exit code 20, "converter data is damaged") before any DLL is opened, like every other mutation of that table.

Before / after (MSVC, Release; cmake -S clean/converter -B <out> -G Ninja ..., cmake --build <out>, ctest --test-dir <out> -R synthetic5):

  • main plus the mutation only: 1069 checks passed, 3 failed (exit code 10 "This file cannot be read": the map loaded and the converter went on to read the missing DLL; "message" and "no DLL read" fail too).
  • with the fix: 1072 checks passed, 0 failed.

Tested

  • ctest of the converter: unit, decompression-vectors, synthetic, synthetic5: all passed with the fix.
  • synthetic5 before the fix (with the new mutation) fails as above.
  • Repository gates (secrets, personal data, licence, view layout): pass.

Not tested

  • The checks on the spec team's synthetic files are skipped by run_tests5.py here (their stand-ins are not in the repository), as in CI.
  • No GPU and no model data; Decoder::indexed itself is unchanged and was not exercised with an overflowing map (the map is now refused before it).

check_group_reads_v5 stopped checking a tensor's axes at the first axis with an
effective length of 0, so a later axis never reached the overflow checks and a
map with a huge stride after an empty axis loaded (and Decoder::indexed could
then overflow in signed 64-bit). The loop now continues past an empty axis: it
still means the tensor reads nothing, but every axis gets its checked
multiplication and addition.

Fixes #54

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 13, 2026 21:17
@phantoslayer21
phantoslayer21 merged commit 04f3c28 into main Nov 13, 2026
2 checks passed
@phantoslayer21
phantoslayer21 deleted the fix/54-converter-map-axes branch November 13, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converter map validation: check every axis, also after an empty one

1 participant