Overview
NumPy explicitly limits arrays to 64 dimensions via NPY_MAXDIMS = 64 (bumped from 32 in NumPy 2.0). NumSharp has no explicit limit, but investigation reveals implicit limits from stackalloc usage that cap practical ndim at ~385,000 dimensions.
This issue tracks whether NumSharp should:
- Keep the implicit ~385K limit (6,000x more than NumPy)
- Add an explicit
MAXDIMS constant matching NumPy's 64
- Remove stackalloc bottlenecks to support higher ndim
Why NumPy Limits to 64 Dimensions
Short answer: Historical/practical reasons, not fundamental necessity. NumPy wants to eventually remove the limit entirely.
The Real Reasons
-
Stack allocation for scratch space - NumPy's C code uses fixed-size arrays on the stack:
npy_intp shape[NPY_MAXDIMS]; // Static allocation
This is fast but requires knowing max size at compile time.
-
ABI stability - Changing NPY_MAXDIMS breaks binary compatibility with compiled extensions. The iterator macros are particularly problematic.
-
Historical accident - Original limit was 32 (speculation: 2*32 for 32-bit systems). Bumped to 64 in NumPy 2.0 for 64-bit era.
-
Sentinel value abuse - axis=MAXDIMS was used internally to mean axis=None. NumPy 2.0 introduced NPY_AXIS_RAVEL to fix this.
NumPy's Long-Term Goal
From the PR #25149 discussion:
"The long-term goal is to remove the limit completely (i.e. remove NPY_MAXDIMS as a public constant)"
They're being pragmatic - some iterator paths are still limited to 32 dims due to legacy code.
Implication for NumSharp
NumSharp's ~385K limit from stackalloc is actually the same pattern as NumPy's NPY_MAXDIMS - both are stack allocation limits. The difference:
- NumPy: Explicit compile-time constant (64)
- NumSharp: Implicit runtime stack size (~385K)
If NumPy's goal is to remove limits entirely, NumSharp could do the same by replacing stackalloc with heap/pooled allocations.
Current Implicit Limits in NumSharp
Stackalloc Bottlenecks (cause StackOverflowException)
| Location |
Allocation |
Tested Limit |
Default.Broadcasting.cs:411,463 |
stackalloc int[nd] |
~385,600 |
ILKernelGenerator.Scan.cs:1499-1500 |
stackalloc int[ndim] x2 |
~900,000 |
ILKernelGenerator.Reduction.Axis*.cs |
stackalloc int[ndim-1] |
~385,000 |
Default.NonZero.cs:225 |
stackalloc int[ndim-1] |
~380,000 |
NDArray.Indexing.Selection.Getter.cs:582 |
stackalloc int[srcDims] |
~385,000 |
Bottleneck: AreBroadcastable() at ~385,600 dimensions. Any broadcasting operation hits this limit.
Other Limits (not blocking int.MaxValue for ndim specifically)
| Limit |
Type |
Notes |
Array.MaxLength |
2,147,483,591 |
.NET limit for int[] dimensions/strides arrays |
int size field |
Silent overflow |
Uses unchecked, wraps at 2^31 total elements |
int offset field |
2^31 addressable |
Limits element addressing, not ndim |
| Stride overflow |
Silent corruption |
strides[0]=0 when dims overflow (tested: 35 dims of size 2) |
Evidence
Tested empirically:
ndim=385,600: AreBroadcastable OK
ndim=386,000: StackOverflowException at DefaultEngine.AreBroadcastable
Stack usage: 385,600 × 4 bytes = 1.54 MB (main thread stack ~1.5-2 MB on Windows)
Comparison
| Library |
Max ndim |
Type |
| NumPy |
64 |
Explicit (NPY_MAXDIMS) |
| NumSharp |
~385,000 |
Implicit (stackalloc) |
NumSharp supports 6,000x more dimensions than NumPy.
Options
Option A: Keep Current Behavior
- Pro: Already supports vastly more dims than NumPy (385K vs 64)
- Con: Implicit limit, StackOverflowException is unfriendly error
Option B: Add Explicit MAXDIMS (like NumPy)
- Add
public const int MAXDIMS = 64; or higher
- Validate in Shape constructor, throw
ArgumentException
- Pro: Clear error, matches NumPy semantics
- Con: Breaking change if anyone uses >64 dims
Option C: Remove Stackalloc Bottlenecks
- Replace
stackalloc int[nd] with heap allocation or pooled arrays
- Pro: Supports Array.MaxLength (~2.1 billion) dimensions theoretically
- Con: Performance regression for common cases, still limited by memory
Option D: Hybrid (Recommended)
- Add
MAXDIMS constant but set it high (e.g., 1024 or 65536)
- Replace stackalloc with conditional:
stackalloc for small ndim, heap for large
- Pro: Best of both worlds - fast for common cases, no arbitrary limit
- Con: More complex
- Aligns with NumPy's long-term goal of removing limits
Recommendation
Option D (Hybrid) - Aligns with NumPy's stated goal to eventually remove NPY_MAXDIMS entirely:
- Replace
stackalloc int[nd] with:
Span<int> buffer = nd <= 64 ? stackalloc int[64] : new int[nd];
- This gives stack performance for typical cases (≤64 dims) while supporting arbitrary ndim
- No artificial limit, no StackOverflowException
References
Overview
NumPy explicitly limits arrays to 64 dimensions via
NPY_MAXDIMS = 64(bumped from 32 in NumPy 2.0). NumSharp has no explicit limit, but investigation reveals implicit limits fromstackallocusage that cap practical ndim at ~385,000 dimensions.This issue tracks whether NumSharp should:
MAXDIMSconstant matching NumPy's 64Why NumPy Limits to 64 Dimensions
Short answer: Historical/practical reasons, not fundamental necessity. NumPy wants to eventually remove the limit entirely.
The Real Reasons
Stack allocation for scratch space - NumPy's C code uses fixed-size arrays on the stack:
This is fast but requires knowing max size at compile time.
ABI stability - Changing
NPY_MAXDIMSbreaks binary compatibility with compiled extensions. The iterator macros are particularly problematic.Historical accident - Original limit was 32 (speculation:
2*32for 32-bit systems). Bumped to 64 in NumPy 2.0 for 64-bit era.Sentinel value abuse -
axis=MAXDIMSwas used internally to meanaxis=None. NumPy 2.0 introducedNPY_AXIS_RAVELto fix this.NumPy's Long-Term Goal
From the PR #25149 discussion:
They're being pragmatic - some iterator paths are still limited to 32 dims due to legacy code.
Implication for NumSharp
NumSharp's ~385K limit from
stackallocis actually the same pattern as NumPy'sNPY_MAXDIMS- both are stack allocation limits. The difference:If NumPy's goal is to remove limits entirely, NumSharp could do the same by replacing
stackallocwith heap/pooled allocations.Current Implicit Limits in NumSharp
Stackalloc Bottlenecks (cause StackOverflowException)
Default.Broadcasting.cs:411,463stackalloc int[nd]ILKernelGenerator.Scan.cs:1499-1500stackalloc int[ndim]x2ILKernelGenerator.Reduction.Axis*.csstackalloc int[ndim-1]Default.NonZero.cs:225stackalloc int[ndim-1]NDArray.Indexing.Selection.Getter.cs:582stackalloc int[srcDims]Bottleneck:
AreBroadcastable()at ~385,600 dimensions. Any broadcasting operation hits this limit.Other Limits (not blocking int.MaxValue for ndim specifically)
Array.MaxLengthint[]dimensions/strides arraysint sizefieldunchecked, wraps at 2^31 total elementsint offsetfieldstrides[0]=0when dims overflow (tested: 35 dims of size 2)Evidence
Tested empirically:
Stack usage: 385,600 × 4 bytes = 1.54 MB (main thread stack ~1.5-2 MB on Windows)
Comparison
NPY_MAXDIMS)NumSharp supports 6,000x more dimensions than NumPy.
Options
Option A: Keep Current Behavior
Option B: Add Explicit MAXDIMS (like NumPy)
public const int MAXDIMS = 64;or higherArgumentExceptionOption C: Remove Stackalloc Bottlenecks
stackalloc int[nd]with heap allocation or pooled arraysOption D: Hybrid (Recommended)
MAXDIMSconstant but set it high (e.g., 1024 or 65536)stackallocfor small ndim, heap for largeRecommendation
Option D (Hybrid) - Aligns with NumPy's stated goal to eventually remove
NPY_MAXDIMSentirely:stackalloc int[nd]with:References