Concatenation / forced indexing

[MilesCranmer]

Hi @mcabbott,
Really enjoying this package, thanks for making it.

I was thinking about a more intuitive way of doing concatenation in higher dimensions (with differently-shaped arrays), and I wondered if the following trick with TensorCast.jl would work:

X = randn(5, 100)
y = randn(100)

@cast data[i, j] := (1 <= i <= 5) ? X[i, j] : y[j] (i in 1:6)

Essentially what I am attempting to do here is creating a new array of shape (6, 100), where the first 5 rows are from X, and the last row is from y. I see the following error:

ERROR: DimensionMismatch: range of index i must agree
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/TensorCast/mQB8h/src/macro.jl:209

I know that the range of indices is usually inferred from arrays, but I thought: perhaps if I pass the range explicitly like this (i in 1:6), it would ignore the inferred range.

Is this syntax for concatenation possible in any way, or does it break key assumptions in the macro?

Thanks!
Miles

[mcabbott]

Hi Miles, glad if it's useful!

This certainly isn't supported right now, all indices must run the full range, and explicit ranges like (i in 1:6) have no special priority, all must agree.

This concatenation idea is roughly the inverse of #56.

I would have to think a bit, but at first glance adding this to @cast sounds very complicated. The example you write is a simple one, how would something like vcat(X, inv.(X)) be written? It could be this:

@cast out[i, j] := (i <= 5) ? X[i, j] : inv(X[i, j])

then i has 3 different ranges which seems hard to keep track of.

Maybe, in addition to A[(i,j)] which means ⊗ and can be written A[i⊗j], there should be some other combined-index for ⊕? With some unknown operator... or perhaps ⊕ inside & outside of indexing?

@cast out[cat(i, i2), j] := X[i, j] ?? inv(X[i2, j])

@cast data[i ⊕ i2, j] := X[i, j] ⊕ y[j] (i2 in 1:1)

Can I then write sqrt(X[i, j] ⊕ y[j]) / z[j], broadcast things inside & outside of the cat on the RHS?