Serve fails on M2 Max with apple_gpu_family_9_required

Running slipstream serve fails at runtime bootstrap with:
error: runtime bootstrap failed [resource_assembly]: apple_gpu_family_9_required

on an Apple M2 Max, which reports apple_gpu_family: 8. The check appears to
require Metal GPU family 9 (M3/ or newer), but this is not documented
in the release notes or enforced before model preparation.

Environment

Slipstream: v26.10.4 (slipstream-26.10.4-macos26-arm-64bit)
Device: Apple M2 Max, 30 GPU cores, 64 GB unified memory
macOS: 27.2.0
Model: swift-qwen38-flash-next-v3 (GGUF, prepared from local directory with MTP sidecar)

Steps to reproduce

/Users/..../slipstream-26.10.4-macos26-arm-64bit/bin/slipstream serve \
  --model /Users/...../models/swift-qwen38-flash-next-v3

Model preparation runs for ~5 minutes (304.5s in my case) and completes successfully
Loading phase starts, then fails immediately at bootstrap
Native protocol reaches EOF and the engine stops

Full log

[Slipstream] Preparing GGUF model from /Users/...../models/swift-qwen38-flash-next-v3...
=== Fast GGUF Ingestion for Qwen3.8-Flash-Next ===
Source: /Users/...../models/swift-qwen38-flash-next-v3
Output: /Users/...../models/swift-qwen38-flash-next-v3/prepared
Sidecar: /Users/...../models/swift-qwen38-flash-next-v3/MTP/mtp-shared-Q4_K_M.gguf
Workers: 5
... (all 48 layers + MTP Layer + Head + Embedding + N-gram table finish successfully) ...
Generated manifest.json
=== Successfully prepared Slipstream model in 304.5s ===
11:28:55 Loading · local/swift-qwen38-flash-next-v3
error: runtime bootstrap failed [resource_assembly]: apple_gpu_family_9_required
memory_plan_json: {"device_name":"Apple M2 Max","macos_version":"27.2.0","apple_gpu_family":8,"gpu_core_count":30,"physical_memory_bytes":68719476736,"recommended_max_working_set_bytes":64424509440,"max_buffer_length_bytes":41747087360,"max_threadgroup_memory_bytes":32768,"max_threadgroup_width":1024,"has_unified_memory":true,"supports_placement_sparse":true}
11:28:55 Error · native protocol reached EOF
11:28:55 Stopping · releasing engine resources

Questions

Is family 8 (M1/M2) support planned, or is family 9 a hard requirement going forward?
If it is a hard requirement, could the installer/binary validate it upfront?

Happy to provide more system details or test a build if needed.

Seeing same issue on 2023 Mac 64 gig. Apple M2 Ultra

Generated manifest.json
=== Successfully prepared Slipstream model in 321.5s ===
19:43:06 Loading · local/swift-qwen38-flash-next-v3
error: runtime bootstrap failed [resource_assembly]: apple_gpu_family_9_required
memory_plan_json: {"device_name":"Apple M2 Ultra","macos_version":"26.6.2","apple_gpu_family":8,"gpu_core_count":60,"physical_memory_bytes":68719476736,"recommended_max_wor
king_set_bytes":62277025792,"max_buffer_length_bytes":41747087360,"max_threadgroup_memory_bytes":32768,"max_threadgroup_width":1024,"has_unified_memory":true,"supports_plac
ement_sparse":true}
19:43:07 Error · native protocol reached EOF
19:43:07 Stopping · releasing engine resources

I checked my M2 Ultra chip. The issue seems to be around bf16 tensors and some hardware not supporting them.
model runtime requires Apple tensor BF16.
I am currently confirming if my device can handle BF16 tensors and if so I will change kMinimumAppleGpuFamily from 9 to 8 in DeviceCapabilities.hpp
If it works and runs I will confirm.

Can confirm, was able to get this running on 2023 Mac M2 Ultra 64gig.
Required a change in kMinimumAppleGpuFamily from 9 to 8 in DeviceCapabilities.hpp - Then rebuilt and tried again, it worked.

☺ slipstream serve --model ~/models/swift-qwen38-flash-next-v3 --port 8090 ruby-3.2.2 main ✗
15:21:25 Loading · local/swift-qwen38-flash-next-v3
Waiting for sufficient available memory to start; the macOS reserve remains protected...
[Qwen4Exp] resident layers: 0 / 48
^[[Qwen4Exp] MTP draft head loaded
[Qwen4Exp] expert cache: 288 experts per layer on average, pins 36.7055 GiB; free now 55.0646 GiB
15:21:42 Kernel policy for GPU family 8 with 60 cores.
15:22:01 Ready · local/swift-qwen38-flash-next-v3 · context 256K · http://127.0.0.1:8090
15:22:46 Error · not_found · GET /favicon.ico
15:25:22 Done · input 62 · cached 0 · output 3,367 · think xhigh · TTFT 3.9s · prompt 16 tok/s · decode 23.6 tok/s

With occasional - engine is recovering; retry shortly when using opencode.

I managed to get it running on M2 Max with basically the same change ref.: #17

I also had to change this though /shrug (was refusing to start nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF)

4. I managed to get it running on M2 Max with basically the same change ref.: #17
I also had to change this though /shrug (was refusing to start nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF)

Can you please tell me your numbers? (pp + tg)

Yeah, not amazing tbh- but I dunno if that's to be expected or what:

For this prompt with Pi, no extensions:

can you generate me a rust cli hello world with colours and flashing text

It generates more or less as expected:

Output from slipstream:

icarus-2:slipstream edwardbeech$ git checkout feature/add-hacky-m2-max-support
Switched to branch 'feature/add-hacky-m2-max-support'
Your branch is up to date with 'origin/feature/add-hacky-m2-max-support'.
icarus-2:slipstream edwardbeech$ make -j 12 && sudo sysctl iogpu.wired_limit_mb=60416 && ./slipstream serve --model nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF --max-memory 59G --max-context 131072 --no-webui --host 0.0.0.0 --port 8088
iogpu.wired_limit_mb: 60416 -> 60416
GGUF model nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF is already downloaded in /Users/edwardbeech/.slipstream/models/nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF
18:35:47 Loading · nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF
[Qwen4Exp] resident layers: 0 / 48
[Qwen4Exp] MTP draft head loaded
[Qwen4Exp] expert cache: 288 experts per layer on average, pins 36.7055 GiB; free now 52.0581 GiB
18:35:54 Kernel policy for GPU family 8 with 38 cores.
18:36:10 Ready · nitinpanj/Swift-Qwen3.8-Flash-Next-Q4_0-Q8out-v3-GGUF · context 128K · http://0.0.0.0:8088
18:37:08 Error · invalid_request_error · POST /v1/responses
18:37:54 Done · input 3,463 · cached 0 · output 87 · think low · tools 4·f088f416 · TTFT 33.3s · prompt 104 tok/s · decode 21.6 tok/s
18:38:01 Done · input 3,620 · cached 3,456 · output 89 · think low · tools 4·f088f416 · TTFT 3.0s · prompt 55 tok/s · decode 20.3 tok/s
18:38:07 Done · input 3,762 · cached 3,616 · output 96 · think low · tools 4·f088f416 · TTFT 2.7s · prompt 54 tok/s · decode 29.8 tok/s
18:38:36 Done · input 3,893 · cached 3,744 · output 529 · think low · tools 4·f088f416 · TTFT 2.8s · prompt 54 tok/s · decode 20.1 tok/s
18:38:48 Done · input 4,456 · cached 3,872 · output 57 · think low · tools 4·f088f416 · TTFT 9.3s · prompt 63 tok/s · decode 28.1 tok/s
18:39:04 Done · input 4,758 · cached 4,448 · output 163 · think low · tools 4·f088f416 · TTFT 6.7s · prompt 46 tok/s · decode 24.8 tok/s
18:39:12 Done · input 4,961 · cached 4,736 · output 86 · think low · tools 4·f088f416 · TTFT 5.1s · prompt 44 tok/s · decode 24.0 tok/s
18:39:45 Done · input 5,252 · cached 4,960 · output 410 · think low · tools 4·f088f416 · TTFT 5.9s · prompt 49 tok/s · decode 18.2 tok/s

Whereas the same prompt for Sushi (standard Qwen3.8 Flash Next, not Swift), roughly similar result:

Truncated output from Sushi:

[mtp] regime gate: two-chunk 23.70 ms/tok (42.19 tok/s) vs single 26.51 ms/tok (37.72 tok/s) -> two-chunk every round (period 24)
[mtp] regime gate: two-chunk 31.83 ms/tok (31.42 tok/s) vs single 29.91 ms/tok (33.43 tok/s) -> two-chunk throttled (period 13)
[mtp] regime gate: two-chunk 31.04 ms/tok (32.22 tok/s) vs single 32.78 ms/tok (30.51 tok/s) -> two-chunk every round (period 12)
[mtp] regime gate: two-chunk 34.99 ms/tok (28.58 tok/s) vs single 32.58 ms/tok (30.69 tok/s) -> two-chunk throttled (period 15)
[mtp] regime gate: two-chunk 34.85 ms/tok (28.69 tok/s) vs single 35.38 ms/tok (28.26 tok/s) -> two-chunk every round (period 8)
[mtp] regime gate: two-chunk 23.06 ms/tok (43.37 tok/s) vs single 21.34 ms/tok (46.86 tok/s) -> two-chunk throttled (period 17)
[mtp] regime gate: two-chunk 23.06 ms/tok (43.37 tok/s) vs single 23.38 ms/tok (42.77 tok/s) -> two-chunk every round (period 8)
[mtp] regime gate: two-chunk 30.87 ms/tok (32.39 tok/s) vs single 21.36 ms/tok (46.82 tok/s) -> two-chunk throttled (period 90)
[mtp] regime gate: two-chunk 30.87 ms/tok (32.39 tok/s) vs single 35.14 ms/tok (28.46 tok/s) -> two-chunk every round (period 28)
[mtp] regime gate: two-chunk 32.51 ms/tok (30.76 tok/s) vs single 20.50 ms/tok (48.78 tok/s) -> two-chunk throttled (period 118)
[mtp] regime gate: two-chunk 29.73 ms/tok (33.64 tok/s) vs single 19.58 ms/tok (51.07 tok/s) -> two-chunk throttled (period 104)

Now I dunno if those numbers are prefill or generation, but the Sushi output "feels" like it streams in a bit faster, so I assume generation?

Not very scientific, sorry!

EDIT: Couple more numbers I noticed in the Sushi logs:

  <- 14+9 tokens streamed [prefill: 25.1 tok/s, decode: 60.2 tok/s] [stop]
  <- 42+140 tokens streamed [prefill: 49.6 tok/s, decode: 34.6 tok/s] [stop]
  <- 201+47 tokens streamed [prefill: 185.3 tok/s, decode: 40.8 tok/s] [stop]

Navigation Menu

Repository navigation

Serve fails on M2 Max with apple_gpu_family_9_required #6

Environment

Steps to reproduce

Full log

Questions

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Serve fails on M2 Max with apple_gpu_family_9_required #6

Description

Environment

Steps to reproduce

Full log

Questions

Activity

WSINTRA commented on Oct 3, 2026

WSINTRA commented on Oct 4, 2026

WSINTRA commented on Oct 4, 2026

initialed85 commented on Oct 10, 2026

1RAFAL commented on Oct 10, 2026

initialed85 commented on Oct 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions