Skip to content

fix: enable SQL UDFs implemented in WebAssembly (Fixes #1653) - #2544

Open
nghoang1288 wants to merge 1 commit into
getdozer:mainfrom
nghoang1288:fix-issue-1653
Open

nghoang1288 wants to merge 1 commit into
getdozer:mainfrom
nghoang1288:fix-issue-1653

Conversation

@nghoang1288

Copy link
Copy Markdown

/claim #1653

Status

Draft implementation scope. The title describes the intended completed PR; the supplied patch only introduces the WASM configuration model and does not yet resolve the issue.

Summary of changes

The supplied patch adds UdfType::Wasm and a WasmConfig module path, preserving the existing ONNX and JavaScript configuration definitions.

The completed implementation must additionally:

  • Wire an optional wasm feature through the relevant crates and CLI.
  • Load and validate configured modules before processing records.
  • Register UDFs with SQL function resolution and expression evaluation.
  • Define the export convention and supported numeric ABI, including argument validation, checked conversions, return-type inference, and SQL NULL handling.
  • Support AssemblyScript’s env.abort import as a controlled runtime error.
  • Bound execution with fuel, memory, and table limits, including module initialization.
  • Isolate per-call mutable state.
  • Regenerate the JSON schema and document a reproducible AssemblyScript example.

The issue’s udfs.wasm mapping is incompatible with the currently documented array representation. Supporting that exact syntax requires a backward-compatible configuration deserializer and corresponding schema updates; silently replacing the existing array format would break existing configurations.

Root cause

WASM is absent from the supplied UDF configuration enum. Executing WASM from SQL additionally requires integration across configuration loading, SQL planning, and runtime evaluation.

Verification tested

Not run. No repository checkout, build environment, or test runner was available. Compilation and end-to-end behavior are not verified.

Before marking the implementation complete, validation must cover:

  • Existing ONNX/JavaScript configurations and feature-disabled builds.
  • WASM configuration parsing and path resolution.
  • SQL invocation of a real AssemblyScript module.
  • Invalid modules, missing exports, unsupported signatures, and incorrect arity.
  • NULL propagation and numeric conversion boundaries.
  • Traps, env.abort, fuel exhaustion, and memory/table limits.
  • Repeatable results across rows and streaming changes.

Required context to finish the patch

The workspace and crate Cargo manifests, top-level configuration loader, full expression implementation and error definitions, SQL function builder, CLI/startup UDF wiring, and existing UDF tests are needed to produce the remaining precise changes.

Fixes #1653 once the integration and verification above are complete; the supplied partial patch does not close the issue.

The supplied configuration model accepts only ONNX and JavaScript UDFs, so it cannot represent a WASM UDF; the published schema also requires `udfs` to be an array rather than the mapping proposed in the issue. Resolving this requires configuration support plus module loading, SQL function resolution, return-type inference, and expression evaluation—not merely another evaluator alongside `python_udf.rs`. The expression enum, SQL builder, startup wiring, and Cargo manifests were not supplied, so a complete, compile-valid integration patch cannot be established from these excerpts alone.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WASM UDF support

2 participants