The possiblity of support Tensorflow in current Wasi-NN inferfaces

[gusye1234]

Hi, I'm trying to support TensorFlow backend for Wasi-NN, but encountered several problems. To specify, the C API's version is 2.6.0

1. Tensorflow input and output can't use numbers, but string to index.

https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/c/c_api.h#L688-L689

2. To run a Tensorflow model, one must prepare the input and output nodes before the actual computation.

https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/c/c_api.h#L1351-L1357

I noticed that you guys may have plans to support Tensorflow. Have you encountered those issues? And if the above two points about Tensorflow are correct, do you think Wasi-NN should adapt its interfaces?

[abrown]

Actually, yes, that has been a bit of a challenge for @brianjjones who has been working on a TensorFlow implementation of wasi-nn in Wasmtime. There have been some hoops to jump through, e.g., here, and perhaps some examination of other parts of that tf.rs file should show how he resolved the problem.

I would like to point out one additional difficulty with wasi-nn and TF: wasi-nn expects to be able to receive the model, weights, etc. as raw byte buffers but, AFAICT, TF expects to have access to the file system for reading a set of files. This is a bit tricky in a WASI environment, since it relies on the user having made that directory a preopened directory, which may not be immediately apparent. Furthermore, it is then odd that wasi-nn models using OpenVINO, e.g., can be initialized with byte buffers but in the TF case these byte buffers are file paths — inconsistent usage of the API. I see two ways around this: a) TF adds methods for initializing models from byte buffers, or b) we add a new wasi-nn mechanism to pre-initialize models prior to the Wasm module's instantiation; pre-initialized models would then be referred to by name. This second method might be useful elsewhere so I have been leaning that direction — any thoughts?

[gusye1234]

Thanks for your guide, will look through this PR

TF expects to have access to the file system for reading a set of files

I decided to only load .pb file😂 so that it can still pass the raw bytes to wasi-nn and create a graph. Not familiar with tensorflow-rs, I guess that could also be a workaround for wasmtime. But yeah, that's another difficulty.

[tonibofarull]

I'm currently mantaining a wasi-nn implementation in WAMR (written in C) where we support TFLite;

• https://github.com/bytecodealliance/wasm-micro-runtime/tree/main/core/iwasm/libraries/wasi-nn

[abrown]

I am going to close this. At the time this was written, users can use TensorFlow as their wasi-nn backend by:

• retrieving and using this open PR in Wasmtime: Adding the TensorFlow backend to wasi-nn bytecodealliance/wasmtime#3977
• using the implementation in WAMR: see @tonibofarull's link above
• using a tf-lite implementation in WasmEdge: https://wasmedge.org/book/en/contribute/build_from_src/plugin_wasi_nn.html

[geekbeast]

Noting this here as well so people are aware of security implications of using tensorflow with wasi-nn
bytecodealliance/wasmtime#3977 (comment)