Description
TinyGo's current behavior is to handle module -> component conversion for -target wasip2.
Where is shell exec today?
wit-bindgen-go
https://github.com/bytecodealliance/wasm-tools-go/blob/main/wit/load.go#L54
wit: parsing / world resolver
Uses wasm-tools to convert wit -> resolved world json.
The resulting json is converted into structs for codegen.
TinyGo
https://github.com/tinygo-org/tinygo/blob/release/builder/build.go#L864
wit: parsing / world resolver
wasm metadata: attach wit to wasm (wasm-tools component embed)
wasm component encoder: convert wasm module to component ( wasm-tools component new )
Uses wasm-tools to adapt a wasm module to component.
Discussion
Should TinyGo / Go generate core wasm modules and leave metadata/component encoding to 3rd party tools; or should they be part of a wasi target ( or goos/goarch combination )?
Assuming we implement wit parsing / wasm patching in Go:
- How do we keep up with wit spec?
- In what sequence we'd create these packages?
Activity
ydnar commentedon Oct 8, 2024
TinyGo, by design and convention, has depended on external tools to handle parts of compilation (LLVM, asyncify, wasm-ld, among others). Based on statements made by the Go team and by historical precedent, I (and others, I believe) assume that this strategy would not apply to mainline ("big") Go. The
gocommand (and associated toolchain) contains the necessary capability to compile (and cross-compile) binaries for a variety of platforms, including wasm.wasm-tools componentsteps, and generated awasm32-wasip2-moduleform suitable for consumption bywasm-toolsor conforming runtimes?lxfontes commentedon Oct 8, 2024
we chatted about it during BA's meeting and no decisive actions other than researching what wasm-tools is actually doing under the hood and follow up next meeting.
We identified a few options to pass down wit:
witdirectory so wasm-tools (or something else) can perform embeddingwitdirectory and rely on information from generated code in downstream tooling. Options:go embedper package: Attaching each interface wit to its corresponding go packagego embedper project: Attach the whole wit to the entrypoint package under compilationgo:wasm{import,export}and markers to carry type information, so the wit can be discarded after codegenThis is a 🔑 point. We touched briefly on the point wrt making TinyGo / BigGo emit wasm32 module and leave
wasm-toolssteps to other tools. It would allow us to defer implementations based on upstream stability ( and yes, I'm looking at you WIT spec! 😂 )The interesting bit here is
wasm-tools-gois already a requirement, so something like this is not far from reality:wasm-tools-goas pre & post compilation steps, havingwit-bindgen-goandassemble-componentdefining their own api / data passing mechanism.The first incarnation of
assemble-componentcould be a full shell exec intowasm-tools, and feels this is a pattern acceptable in Big Go.again, no decisions made but wanted to relay the discussion.
ydnar commentedon Oct 9, 2024
I have WIT generation working with tree shaking in a branch, so we can emit minimal WIT files containing a single world to a world's Go package directory.
wasm-tools-go is only a prerequisite for code generation, so it's not really required to compile and build a Go program. This is similar to other tools used by big Go projects like protoc or avo.
ydnar commentedon Dec 8, 2024
Work is progressing here:
wasm-toolsas a WASI 0.1 module inside ofwit-bindgen-go: internal/wasmtools: run wasm-tools.wasm in embedded Wazero runtime #252ydnar commentedon Dec 24, 2024
Merged #272.
wasm-toolsis now vendored as a WebAssembly module into packageinternal/wasmtools. Thanks @Mossaka!