Do something interesting with a Wasm-hosted, Wasm-targeting Clang/LLD?

[whitequark]

I now provide exactly that at @yowasp/clang. The JavaScript package has a little bit of cursed glue code that uses -### to work around the inability to spawn subprocesses, but other than that the build is pure-WASI, and isn't dependent on JavaScript. If you are fine with invoking the linker manually then you do not need this glue code.

I plan to upstream the remaining changes to LLVM but haven't done so yet.

I recall there being some interest among the general BA folks in such a compiler, but unfortunately I cannot recall from whom, where, and for what purpose. The obvious option would be to bootstrap the entire WASI SDK with it, which I haven't done because the last 14 continuous hours of watching LLVM build make me sick if I think about it. But maybe there are some accessibility / outreach type things possible now too, like a Wasm playground?

Anyway, please have fun, and feel free to close the issue whenever you feel it's been taken into consideration.

[alexcrichton]

This is awesome! I'm also no fan of babysitting long builds myself...

In terms of longevity one rough idea is that this repository would be a somewhat-ideal location to house the build scripts and final wasm artifact. Publication of the npm package I'd prefer to relegate to a different repository (such as the one you already have) but shifting the responsibility of building for wasm to this repo seems reasonable to me.

Would you like some help upstreaming the LLVM changes? Might be able to poke a few folks and see if they'd be able to help review and/or do logistics there. We could also carry a local patch in this repo but there's no current precedent for that so it might be a bit much to take that on infrastructure-wise for this.

[whitequark]

Would you like some help upstreaming the LLVM changes?

Sure. The patchset is pretty small now that the saga of enabling <mutex> et al on single-threaded Wasm is complete, and I have a pretty long-running Discourse thread on it. Practically, what's needed now is creating a new Wasm platform in LLVMSupport (rather than adding more defines to the Unix platform), which I haven't got time to do.

I think the patches can be fully upstreamed (based on previous discussion), and if BA promises to run the buildbot that will be much more convincing than if I do by virtue of it being an organization. I'm fine continuing to distribute the NPM packages.

[alexcrichton]

Is YoWASP/llvm-project@c37f6c3 effectively the only part that needs to be upstreamed to LLVM?

Also, do you have an already-pending PR on the LLVM side? Or any known concerns/blockers people might have raised? It looks pretty uncontroversial to me but I'm also no LLVM reviewer.

[whitequark]

Is YoWASP/llvm-project@c37f6c3 effectively the only part that needs to be upstreamed to LLVM?

Yes! This isn't a very big or controversial patch.

Also, do you have an already-pending PR on the LLVM side? Or any known concerns/blockers people might have raised? It looks pretty uncontroversial to me but I'm also no LLVM reviewer.

Here is my existing pending PR, and it has review comments that should be addressed which are still valid against the patch above. Mainly, they want me to stop adding ifdefs to Unix support.

[alexcrichton]

One thing that might help, which I've found useful in Rust at least, is to not set LLVM_ON_UNIX for example and leave that diasbled. That'll, in theory, fall back to generic/"other" implementations by default and then on an opt-in basis there can be || defined(__wasi__) in various places for functionality that wasi-libc provides. While wasi-libc is unix-like I've found that "this platform is unix" tends to pull in a lot more than wasi-libc needs. That might invert some of the #define in a different dierction where instead of excluding things it's including things, but it might help cut down on the conceptual #ifdef perhaps?

@sbc100 or @sunfishcode do y'all have thoughts/opinions about how best this might land though? I'm not too familiar with the LLVM review process myself.

[sbc100]

I know its already likely be split up a lot, but continuing to split it up and land the completely non-controversial parts separately is always a good idea in my book.

[geraintluff]

What's the status on this? Just checking whether a WASM-hosted WASI-SDK is likely to be stable for medium-term use, and keep up to date with any WASI-SDK updates.

[whitequark]

I've continued to maintain the patchset as-is. I likely won't have time to reimplement it in the way the LLVM reviewers want to see it in the next 1-2 months at least.