Skip to content

fix(core): split flat indices with std.intdiv instead of f32 division - #120

Open
MiniX16 wants to merge 2 commits into
software-mansion:mainfrom
MiniX16:fix/intdiv-indices
Open

MiniX16 wants to merge 2 commits into
software-mansion:mainfrom
MiniX16:fix/intdiv-indices

Conversation

@MiniX16

@MiniX16 MiniX16 commented Nov 14, 2026 •

Copy link
Copy Markdown

Description

Inside 'use gpu', / compiles to f32 division, so d.u32(a / b) is u32(f32(a) / f32(b)), not integer division. Every d.u32(a / b) left in src/core/kernels (116 sites in 15 files) is now std.intdiv(a, b), as rows.ts, transpose.ts and attnFlash.ts already do. The six 2^24 cases that were it.fails pass now and are plain it.

The bug is not limited to large tensors. On an AMD Radeon 610M (RDNA2, Mesa 26.2.1 RADV, Chromium 156), u32(f32(a) / f32(b)) comes out one too low for every exact multiple of a divisor that is not a power of two: 3 / 3 = 0, 96 / 96 = 0. On that GPU main fails 97 GPU tests, small shapes included, plus the MobileNetV4 and Higgs encoder zoo tests.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Performance (kernel tuning, graph building, anything related to performance)
  • Other (chores, tests, code style improvements etc.)

Testing instructions

  1. pnpm --filter runntime test:gpu (local GPU). New hwc4 cases with c = 10 (c4 = 3) for the toHwc4/toChw round-trip and channelAffine: they fail on main on the AMD GPU and pass with the fix. On that GPU the suite goes from 97 failures to 1, the existing tanh it.fails for a Metal bug.
  2. pnpm --filter runntime bench:ab kernels

A/B against main (Radeon 610M): 201 cases, 10 faster, 2 tiledVec4 rows slower in the first run that came out same in two reruns. Non-zero output differences are main computing wrong on this GPU.

kernel case A (µs) B (µs) Δ max abs(A−B)
conv tiled (8 ch/block), YOLO26n layer1 16→32 k3 s2 @320² 1026 769 -25.0% 0
conv tiled (16 ch/block), YOLO26n layer7 128→256 k3 s2 @40² 885 638 -27.9% 0
conv tiled + fused residual, 64→64 k3 @80² 1321 1003 -24.1% 0
elementwise swigluChunk f16, prefill [200,2304] 63.3 55.9 -11.8% 0
conv depthwise (1 col), MobileNetV4 576ch k5 @14² 117 116 same 2.8e+0
ssm ssmScanProject, DepthART-B c 96 rank 12 205 207 same 4.2e+1

Screenshots

N/A

Related issues

Closes #57

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

  • colWord in moeQuantW.ts is called with literals, so its argument goes through d.u32 to keep the result u32.

Inside 'use gpu', d.u32(a / b) compiles to u32(f32(a) / f32(b)). Past 2^24,
or on drivers whose f32 division is not correctly rounded, the quotient lands
on the wrong row. Every kernel now uses std.intdiv, and the 2^24 cases that
were it.fails pass.
…lock counts that are not a power of two

colWord is called with literals, so std.intdiv on its i32 argument returned
i32 where d.u32 returned u32. The toHwc4/toChw round-trip and channelAffine
cases use c = 10 (c4 = 3), which fails on main on GPUs whose f32 division is
not exact for such divisors.
@MiniX16
MiniX16 marked this pull request as ready for review November 14, 2026 14:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

d.u32(a / b) divides in f32 and gives wrong indices on large tensors

1 participant