Repository navigation
hrx-system 4ba76c18eafe: Qwen3-0.6B returns end-of-text at once on a 4.7K-token prompt (98d05d94 answers) #369
Description
Found 2026-10-09 while verifying the loom-jit port (fork #97) on strixhalo. Both builds below use the SAME llama.cpp (e44c9d01a4d5 + the #97 port, commit 30d6d3b44511) and differ only in third_party/hrx-system.
Repro: llama-server -m Qwen3-0.6B-Q4_K_M.gguf -dev HRX0 -ngl 99 -c 8192 -np 1 --jinja with IREE_HAL_AMDGPU_LIBHSA_PATH set, then /v1/chat/completions with temperature 0, cache_prompt:false, enable_thinking:false: a 213-token warm-up, then a 320-line distractor text with "the secret vault code is 7341" at line 203 and the question "What is the secret vault code? Answer with the number only." (4,732 prompt tokens). Script: strixhalo ~/wt/loom-cog/probe.py; log ~/wt/loom-cog/chain2.log.
| hrx-system | warm-up (213 tok) | needle (4,732 tok) |
|---|---|---|
98d05d94 (our fork 1bit/main: 51b1739 + lane-mask backport + knobs) |
"Hello! I'm here to help...", 16 tokens | "7341", 5 completion tokens, twice |
4ba76c18 (1bit/main merged with AMD c0b135a, 2026-10-07) |
same text, 16 tokens | empty, finish_reason=stop, 1 completion token, twice |
No NaN, no error in either server log. test-backend-ops -b HRX0 passes on the 4ba76c18 build (Backend HRX0: OK, 2/2), and tests/serve_e2e.sh passes on both, so neither gate catches this: only a prompt longer than one ubatch does (the same lesson as #319). The earlier 2026-10-05 attempt to pin upstream hrx-system (ee77c8f7f + 336037562) also gave wrong Qwen3 output; this is the same class.
Consequences:
- Fork serve --device hrx: default the context to 32768 so HRX0 keeps flash attention (#95) #97 (the loom-jit port) is fine as code: it compiles against both revisions and the 98d05d94 pair answers correctly with it. It stays a draft until an hrx-system revision past the LoomC API change also decodes correctly.
third_party/hrx-systemstays at 98d05d94. The ci(bump-hrx): hold hrx-system when our ggml-hrx uses LoomC symbols the new revision dropped #365 guard holds it while the LoomC symbols are missing; once serve --device hrx: default the context to 32768 so HRX0 keeps flash attention (#95) #97 lands those symbols exist again and the guard would let 4ba76c18 through, so the bump must additionally run a long-prompt check before moving hrx-system, or hrx-system stays a hand-moved pin.- Bisect target: AMD commits between 51b1739 and c0b135a (Loom passes: compact pass registry, fixed-edge copies, grouped activation vectorisation, 32d0c76a runtime globals) with the needle above as the oracle.
🤖 Generated with Claude Code
Activity
HRX hardware correctness gate on Strix Halo (hw-gate) (#403)