Skip to content

Default Flash Next to Red Hat NVFP4 and advertise recipe capabilities - #15

Merged
dmytrogajewski merged 3 commits into
mainfrom
feat/redhat-qwen38-default
Oct 30, 2026
Merged

dmytrogajewski merged 3 commits into
mainfrom
feat/redhat-qwen38-default

Conversation

@dmytrogajewski

@dmytrogajewski dmytrogajewski commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

The default non-abliterated qwen3.8:flash-next / qwen3.8:flash-next-nvfp4 aliases now select RedHatAI's immutable NVFP4 checkpoint. Recipes also declare client capabilities explicitly, so a healthy qualified generation advertises its reasoning levels, default effort, summaries, tools, parallel tools and verbosity through the control API and public /v1/models. Codex consumes that contract instead of blanket feature defaults: the Red Hat model exposes efforts through xhigh, defaults to xhigh, and streams reasoning summaries without workaround flags. Explicit user overrides still win.

Capability declarations travel with the exact generation's verified gateway profile through executor IPC and reconciliation. Warming, unhealthy, mismatched and legacy generations remain conservative. Recipe validation rejects inconsistent declarations; a bounded startup reasoning probe requires both reasoning and a completed answer before publication. Legacy profile serialization preserves protected identities and historical evidence.

The Red Hat engine uses compressed-tensors W4A4 experts and disk-backed BF16 PLE with the owned, digest-pinned B12X runtime. It preserves 262,144-token context, FP8 KV, MTP4 and vision. The image normalizes the checkpoint's serialized attention-layer label for the pinned runtime while retaining QSA/indexer settings. Hash-guarded patches and real checkpoint/PLE reader tests cover compatibility, reads, strides, BF16 expansion, concurrency and failure cleanup. Existing RadixArk and separately selected abliterated recipes remain available.

Interrupted downloads credit only allocated bytes in safe managed partial files when planning additional storage. Transfer verification, temporary budgets and the 100 GiB disk reserve remain enforced. This PR also records the previously deployed abliterated recipe, updates zstd for the dependency audit and installs CPU reader test dependencies in CI.

Long coding sessions also exposed a transport failure during automatic compaction: the adapter's 1,800-second HTTP deadline was silently shortened by reqwest's 30-second Linux TCP unacknowledged-data timeout. Set that TCP deadline explicitly to the same finite inference budget. Translated Responses, Chat and Anthropic streams emit ten-second SSE comment heartbeats through the adapter during quiet prefill, without creating output events or token usage. Redacted diagnostics distinguish request/body failures and timeout classification. Engine deadlines and cancellation remain enforced.

Validation:

  • make lint, make test (1,101 passed), make test-client (186 passed), make audit
  • Real IPC and pinned HTTPS API-to-Codex catalog coverage, stale-generation rejection and native HTTP reasoning-probe failure cases
  • make web-test-browser, strict Rust documentation and native Codex launcher regression
  • Signed appliance and client updates applied; Red Hat generation 3 retained healthy with the exact same running container and runtime specification
  • Native Codex metadata-only check: actual generated catalog accepted; effort picker and default xhigh; effective xhigh / summary auto despite a stale none preference; no coding thread created
  • Live pinned HTTPS Responses with xhigh and reasoning summaries: tool call and stateless continuation completed; zero tools executed
  • New real TCP backpressure regression fails at 30 seconds before the fix and completes after a 35-second stall with the fix; pinned HTTPS plus private adapter regression proves quiet-prefill heartbeats and unchanged terminal usage
  • Installed launcher/private adapter: captured compaction retry completed in 158 seconds (125,613 input / 6,502 output tokens), with zero tools executed
  • Installed launcher/private adapter: 249,997-token request completed in 114 seconds, forwarded 11 heartbeats and retrieved its beginning-of-document marker
  • Runtime specification hash identical to generation 2, including image, context, resource limits, mounts, command and confinement; no kernel OOM or GPU no-memory events
  • Earlier generation-2 qualification also covered saved failing request replay, images, cancellation and 2/4/8 concurrent clients

The Red Hat snapshot is approximately 186 GB. Disk-backed PLE avoids allocating its full BF16 embedding table in unified memory. Memory/disk admission reserves and the bounded startup deadline are unchanged. Deployment preserved existing local launcher permission behavior through a private build overlay; unrelated workspace edits are excluded from this PR.

@dmytrogajewski dmytrogajewski changed the title Default Flash Next to Red Hat AI NVFP4 Default Flash Next to Red Hat NVFP4 and advertise recipe capabilities Oct 29, 2026
@dmytrogajewski
dmytrogajewski merged commit 55a32fa into main Oct 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants