Plan to support OCP Microscaling INT8 element data type?

[wonjeon]

Wonder if you guys have a plan to support the element data type of INT8 in the OCP Microscaling specification, which is described in the chapter 5.3.4, https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf.

[seberg]

TL;DR: I doubt it fits here, there is a lot of complexity compared to the other dtypes.

Since I had briefly thought about it, maybe some information:

• This is only possible with the new NumPy API because it is a parametric dtype. So there is a hurdle as it likely only makes sense after using the new API also for other dtypes. It also means that such a dtype is quite different and could not share much of the implementation.
• Unless the scaling factor is identical for the whole NumPy array (and not some blocks or so), things get vastly more complicated (unless maybe each element of the array is a whole block). So that would add another pretty big thing to figure out.
  (Yeah, there are potential approaches similar to how the new StringDType works now but...)

So my suspicion is that unless this was a critical feature, ml_dtypes isn't the right place to experiment with this. (numpy-user-dtypes may have pointers, I even have a minute "scaled float" dtype in NumPy for testing that has a single factor for the whole array -- .)
If it was clear that it was just a single scale factor and the rest of the repo was converted, maybe things would start looking a bit different.

(Of course I can't judge the need from e.g. the JAX side.)