Skip to content

BEAM-native JS engine and compiler#5

Open
dannote wants to merge 2614 commits into
masterfrom
beam-vm-interpreter
Open

BEAM-native JS engine and compiler#5
dannote wants to merge 2614 commits into
masterfrom
beam-vm-interpreter

Conversation

@dannote
Copy link
Copy Markdown
Member

@dannote dannote commented Apr 15, 2026

Introduces a BEAM-native JavaScript execution engine from the ground up. Before this PR, QuickBEAM executed JavaScript through the native QuickJS runtime only; there was no BEAM bytecode decoder, interpreter, or compiler backend.

This PR adds the full BEAM execution path: an Elixir QuickJS bytecode decoder, an interpreter that runs decoded QuickJS bytecode directly on the BEAM, and a compiler that lowers QuickJS bytecode into BEAM modules. The rest of the branch builds the runtime semantics, object model, compiler ABI, opcode metadata, audits, and parity fixes needed to make that new engine usable and maintainable.

Public API and runtime modes

  • mode: :beam support for running JavaScript through the BEAM VM backend
  • mode: :auto compiler-backed execution with fallback
  • strict compiler audit mode via mode: :beam_compiler
  • QUICKBEAM_MODE=beam_compiler support for audit runs
  • raw BEAM disassembly for the :beam backend via QuickBEAM.disasm/2
  • runtime/context integration, handler dispatch, module loading, dynamic import, globals, and interop for the VM path
  • stack traces, source positions, and Error.captureStackTrace

Bytecode VM

  • QuickJS bytecode decoder in Elixir
  • BEAM interpreter for QuickJS bytecode
  • control-flow, stack-depth, type, and diagnostics analysis passes
  • local/argument/capture slot handling, including large-frame tuple mode
  • generator and async-generator execution support
  • promise scheduling and continuation state
  • runtime invocation helpers for compiled and interpreted functions
  • direct-eval integration with caller context, globals, refs, arguments, and strictness handling

Compiler backend

  • hybrid compiler from QuickJS bytecode to BEAM modules
  • block lowering, CFG analysis, stack-depth inference, type inference, and optimizer passes
  • compiled runtime helpers for arithmetic, coercion, calls, construction, refs, globals, with-scope, iterators, objects, classes, promises, generators, and errors
  • strict compiler-vs-interpreter audit harnesses and semantic corpus benches
  • compiler opcode coverage reporting
  • compiler resource guardrails with interpreter fallback
  • compiler perf/ROI benchmark

Compiler/interpreter architecture hardening

  • centralized JavaScript object semantics behind ObjectModel.InternalMethods
  • decomposed ObjectModel.Get into focused helpers for own lookup, ordinary get, prototype traversal, callback factories, symbol/length behavior, and exotic gets
  • moved proxy behavior into focused proxy helper modules with shared ProxyDispatch
  • routed runtime/interpreter/semantics callers away from low-level OwnProperty, Prototype, Define, Delete, and HasProperty APIs
  • added runtime-facing architecture tests to prevent direct low-level object-model calls from returning
  • introduced RuntimeABI as the generated-code boundary and added disassembly coverage to prevent generated BEAM from calling lower-level VM internals directly
  • centralized opcode facts in OpcodeSpec, including format/operand metadata, stack effects, lowering family/module/handler metadata, control-flow metadata, call arity, slot metadata, and branch metadata

JavaScript semantics covered

  • Object, Array, Function, String, Number, Boolean
  • Math, JSON, Date, RegExp
  • Map, Set, WeakMap, WeakSet, collection iterators, and self-iterability
  • Symbol, well-known symbols, property keys, and Symbol.toStringTag
  • Promise, async / await, generators, delegated yield*, and async generators
  • Proxy and Reflect, including descriptor/prototype/extensibility invariants
  • TypedArray, ArrayBuffer, BigInt
  • classes, inheritance, super, private fields, private methods, private accessors, static private members, brand checks, and constructor return semantics
  • descriptors, Object.create, Object.assign, Object.fromEntries, freeze/seal/preventExtensions/isExtensible/isFrozen/isSealed
  • array expando properties, sparse arrays, callbacks, iterators, destructuring, spread, rest/default parameters, and arguments basics
  • eval, with, refs, globals, callback/global write freshness, delete semantics, in, instanceof, new, update operators, arithmetic/coercion edge cases, and BigInt operations
  • try/catch/finally, QuickJS gosub/ret finally control flow, nested catches inside finally, break/continue through finally, and catch-region slot preservation
  • for-in enumeration mutation and nullish object coercion for with/destructuring
  • Web API and Node-compat runtime integration used by the VM path

Semantic parity fixes discovered while bringing the backend online

  • direct eval strict-mode [[Set]] failure handling
  • direct eval caller arguments object reuse
  • global object fallback reads and transactional writes around observable setters/proxies
  • strict eval context installation before dispatch
  • proxy descriptor reflection normalization and prototype invariants
  • reference-backed process-state ownership for iterator and worker state
  • let loop closure snapshots for captured loop variables
  • iterator helper validation close timing
  • clearInterval cancellation during timer drain
  • BEAM conversion hiding internal VM slots such as :__internal_proto__

Static analysis and cleanup

  • updated ex_dna, ex_slop, and credo
  • mix lint now runs ExDNA with a zero-clone budget
  • clone debt removed across VM runtime helpers, parser helpers, Web API modules, and collection/object helpers
  • Bandit clock application startup fixed for --no-start test runs
  • shared runtime/context message handling fixed for context state without :mode

Audit and benchmark tooling

  • bench/vm_compiler_compat.exs
  • bench/vm_compiler_corpus.exs
  • bench/vm_compiler_opcode_coverage.exs
  • bench/vm_compiler_perf.exs
  • bench/vm_compiler_semantic_gaps.exs
  • bench/vm_compiler_test262.exs
  • test/support/vm_compiler_audit.ex
  • compiler acceptance and auto-mode tests
  • compiler ABI boundary disassembly tests
  • runtime-facing architecture boundary tests

Current compiler audit status

  • QuickJS bytecode opcode coverage: 245 / 246 unique opcodes
    • remaining missing opcode: invalid, intentionally unsupported
  • strict compiler Test262 statement window: 1582 / 1582 passing
  • semantic compiler corpus: 1000 / 1000 compiled with zero mismatches, fallbacks, or crashes
  • compiler invoke perf improved from 45.239µs to 42.845µs
  • current compiler perf audit average speedup: 1.113x

Recent validation

  • mix reach.check
  • mix test test/vm/architecture_boundary_test.exs test/vm/opcode_spec_test.exs test/vm/compiler/lowering_registry_test.exs test/vm/compiler/abi_boundary_test.exs test/vm/compiler test/vm/runtime --max-failures 5
  • mix test test/vm/modes/dual_mode_test.exs test/vm/modes/beam_compat_test.exs test/vm/semantics test/web_apis/beam_web_apis_test.exs test/vm/compiler test/vm/runtime --max-failures 5
  • mix test --max-failures 5

Previous broad validation:

  • mix compile --warnings-as-errors
  • mix lint
  • mix dialyzer
  • mix test test/js/parser test/vm/auto_mode_test.exs test/vm/compiler_differential_test.exs test/web_apis/beam_fetch_test.exs test/web_apis/beam_text_encoding_test.exs test/web_apis/beam_buffer_test.exs test/web_apis/beam_streams_test.exs test/web_apis/event_source_test.exs
  • mix test --no-start --exclude napi_addon --exclude napi_sqlite test/web_apis/event_source_test.exs test/web_apis/beam_event_source_test.exs
  • PARSER_BENCH=vm_compiler_semantics ./autoresearch.sh
  • TEST262_LIMIT=1500 TEST262_CASE_TIMEOUT=5000 PARSER_BENCH=vm_compiler_test262 ./autoresearch.sh
  • QUICKBEAM_BUILD=1 MIX_ENV=test mix test
  • MIX_ENV=test QUICKBEAM_BUILD=1 mix test test/vm/js_engine_test.exs --include js_engine --seed 0
  • mix format --check-formatted
  • mix credo --strict
  • mix dialyzer
  • mix ex_dna
  • zlint lib/quickbeam/*.zig lib/quickbeam/napi/*.zig
  • bunx oxlint -c oxlint.json --type-aware --type-check priv/ts/
  • bunx jscpd lib/quickbeam/*.zig priv/ts/*.ts --min-tokens 50 --threshold 0

@dannote dannote force-pushed the beam-vm-interpreter branch from 0eb3475 to 7c1c574 Compare April 15, 2026 14:06
@dannote dannote changed the title BEAM-native JS interpreter (Phase 0-1) BEAM-native JS interpreter Apr 16, 2026
@dannote dannote marked this pull request as ready for review April 16, 2026 08:41
@dannote dannote force-pushed the beam-vm-interpreter branch 2 times, most recently from 75fdba5 to 527d5b9 Compare April 20, 2026 08:45
@dannote dannote changed the title BEAM-native JS interpreter BEAM-native JS engine and compiler Apr 21, 2026
dannote added 30 commits May 24, 2026 12:25
# Conflicts:
#	lib/quickbeam/context.ex
#	lib/quickbeam/js/bundler.ex
#	lib/quickbeam/runtime.ex
#	mix.exs
#	mix.lock
#	npm.lock
#	package.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant