Starting vllm service failed with qwen3.6 35b a3b sym_int4 on B70

[HoppeDeng]

(EngineCore_DP0 pid=629674)�[0;0m ERROR 05-11 06:53:08 [core.py:938] RuntimeError: gdn_conv_fused_seq: HV (32) exceeds max v-thread slots even with WG_SIZE=64 (0). H=16

serving script:
python3 -m vllm.entrypoints.openai.api_server \

    --model "$MODEL_PATH" \

    --served-model-name "$MODEL_NAME" \

    --dtype=float16 \

    --enforce-eager \

    --port 8001 \

    --host 0.0.0.0 \

    --trust-remote-code \

    --gpu-memory-util=0.9 \

    --enable-prefix-caching \

    --max-num-batched-tokens 16384 \

    --disable-log-requests \

    --max-model-len  66560 \

    --block-size 64 \

    --quantization sym_int4 \

    -tp=1

[edisonchan]

I don’t think a single GPU is enough to fit the model.
And, I can not load the model too even I have 2 GPU (32g+32g = 64g).

[jonm0bius]

I got it running on sym int4 on a single gpu with similar settings 190000 context length kv cache fp8 but it needs more then 71 gb of system ram for qwen3.6-35b to not crash while online quant