Skip to content

A grammar-constrained completion can abort the Wasm core #115

Description

@leehack

A grammar-constrained createCompletion can abort the Wasm core instead of returning text or an error:

Aborted(undefined). Build with -sASSERTIONS for more info.

In worker mode the abort also moves the bridge to the main thread for the rest of the session (__llamadartBridgeWorkerFallbackReason holds the abort text).

Repro (same on v0.1.44 and v0.1.47), Qwen3.5-0.8B Q4_K_M on the CPU:

await bridge.loadModelFromUrl(modelUrl, { nCtx: 1024, nThreads: 2, nGpuLayers: 0 });
await bridge.createCompletion(
  '<|im_start|>user\nIs the sky blue? Answer yes or no.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n',
  { nPredict: 8, temp: 0, topK: 1, seed: 1, grammar: 'root ::= "yes" | "no"' },
);
  • The same grammar also aborts with topK: 40, temp: 0.8.
  • A small JSON grammar aborts with topK: 1 and succeeds with topK: 40.
  • stories15M with the yes/no grammar and topK: 1 aborts too.

Likely cause: create_sampler (src/llama_webgpu_core.cpp:807-822 at 64ba825) adds the grammar sampler after top-k and top-p. When truncation keeps only tokens the grammar rejects, no candidate is left. llama.cpp's common_sampler avoids this: it checks the sampled token against the grammar and, if rejected, resamples with the grammar applied first.

Expected: a grammar-constrained completion returns text the grammar accepts, or rejects with an error the caller can handle.

Found while validating the bridge pin bump in leehack/llamadart#620. llamadart passes GenerationParams.grammar and topK through unchanged.

Activity

added
bugSomething isn't working
priority:P1Highest priority; correctness, security, or release blocker
on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:P1Highest priority; correctness, security, or release blocker

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions