Skip to content

Publish test262 conformance for the vendored Boa engine in CI #35

Closed

Description

@hassidan

Summary

tests/test262/ exists in the repo but contains only __pycache__; there is no committed runner output, CI job, or published conformance number. The vendored Boa in vendor/boa_engine has been modified (inline caches, SLICEBC CodeBlock precompilation, fast paths), so upstream Boa's published ~90 % figure no longer describes this engine.

Why it matters

Users migrating from Node/Bun/Deno (all ≥ 99 % test262) need a number, and the project needs a regression guard for its engine patches. A conformance score is also the single most credible counter to "it's an interpreter, how much of JS does it actually run?".

Proposal

  • Add tests/test262/run.py (or reuse what generated the __pycache__) that runs the official tc39/test262 suite against slice run with the --deterministic flag, honouring includes, flags: [onlyStrict|noStrict|module|async], and negative expectations.
  • Commit an excludelist with reasons (e.g. Intl disabled, Temporal not implemented) rather than silently skipping.
  • Publish the result in CI as a percentage per directory (built-ins/, language/, intl402/, annexB/) and fail the job on regression.
  • Surface the headline number in the README next to the benchmark table.
  • Track the vendored Boa delta: a vendor/boa_engine/SLICE_PATCHES.md listing each modification and whether it affects semantics, so the test262 diff against upstream Boa 0.22 is explainable.

Activity

  1. siktec-lab commented on Nov 9, 2026

    @siktec-lab
    Collaborator

    Addressed in #41 (�0.10.6).

    • By design, slice pack and slice compile operate as an offline deterministic bundler over an already-populated
      ode_modules directory (allowing users to bring npm, pnpm, yarn, or bun).
    • Added a clear diagnostic hint when a bare specifier fails to resolve and
      ode_modules/ is missing, guiding users to run
      pm install (or pnpm i / yarn / �un install) first.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions