Use-case report + two docs questions: Qwen3.8-27B uncensored at 262K on a single RTX 5090 via NInfer

[PcMaKeR]

First, thank you for NInfer. On an RTX 5090, for the hybrid Gated-DeltaNet Qwen3.8-27B, this was the only engine we found that runs the full 262,144 context with a real 4-bit KV cache and DFlash2 at the same time — SGLang's NVFP4 KV path hit KeyError: 'float4_e2m1fn_x2' on the triton backend, and trtllm_mha is SM100-only. NInfer just worked.

This is a use-case report (not a bug): we ran an uncensored Qwen3.8-27B through the existing qwen3_8_27b/convert_nvfp4 path and served it unmodified. I'd also like your guidance on two small documentation/contract points before I consider any docs PR.

What we did (supported product, unmodified engine)

• Built master (ninfer-serve) as documented; no code changes.
• Quantized huihui-ai/Huihui-Qwen3.8-27B-abliterated (uncensored, Apache-2.0) to the same mixed FP8+NVFP4 layout as unsloth/Qwen3.8-27B-NVFP4, using llm-compressor.
• python3 -m tools.convert.qwen3_8_27b.convert_nvfp4 --model <bf16> --quantized-model <our-nvfp4> --dflash2-model z-lab/Qwen3.8-27B-DFlash2 --out huihui.ninfer → preflight complete: 1190 objects, 233 FP8 and 168 NVFP4 source matrices, 81 DFlash2 source tensors.
• Served with --max-context 262144 --kv-dtype k8v4 --spec dflash2 --draft-tokens 3 --default-thinking-budget 600.

Measured (RTX 5090, thinking off)

• ~120–180 tok/s decode
• ~3,470 tok/s prefill on a 163,909-token prompt
• DFlash2 accept ~49%
• Needle-in-haystack at 164k passed
• Model load ~5 s

Environment

• NInfer master (commit 47bb19a)
• RTX 5090 (sm_120), Ubuntu 24.04, CUDA 13.3
• Registered identity: qwen3.8-27b / nvfp4

Two questions (docs / contract — no code expected from you)

1. The base tokenizer is hash-pinned to official Qwen/Qwen3.8-27B. Most uncensored fine-tunes ship a modified tokenizer.json and are rejected with tokenizer.json hash mismatch. huihui-ai and orcarouter pass because they only changed weights. Is this pin intentional for DFlash2 draft/target alignment (a correctness guarantee), and would relaxing it for a chat-template-only difference ever be in scope — or is the recommendation simply "use a base with the official tokenizer"? I'm happy to document whichever answer is correct.

2. The exact --quantized-model layout the converter validates (mixed-precision: FP8 W8A8 targeting self_attn.*, linear_attn.(in_proj_qkv|in_proj_z|out_proj), lm_head, layers.56-63.mlp; NVFP4 W4A4 group-16 on the other MLP; vision ignored) is currently only discoverable from the convert source or by matching unsloth/Qwen3.8-27B-NVFP4. Would a short note in the convert docs help people quantize their own base? I'd gladly draft it as a docs PR if you'd like.

Links (open source, with an authorized-use disclaimer)

• Step-by-step guide + scripts (build → quantize → convert → serve, EN + TR): https://huggingface.co/pcmaker/Qwen3.8-27B-Uncensored-NInfer-RTX5090-Guide
• Ready-to-run .ninfer artifact: https://huggingface.co/pcmaker/Huihui-Qwen3.8-27B-Uncensored-NInfer

Both credit NInfer prominently and link back to this repo. If a write-up like this is unwelcome or you'd prefer it framed differently, just let me know and I'll adjust. Thanks again for the engine.