Skip to content

Add Ascend910B4 result of deepseek-v4.1-flash-1017 and glm-5.3-flash #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mario-LuX merged 1 commit into from
Nov 6, 2026
Merged

Add Ascend910B4 result of deepseek-v4.1-flash-1017 and glm-5.3-flash #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ English | [简体中文](README.zh-CN.md)
> `CONTINUATION_ENGINE`. Within this project, each engine + model combination has its own
> continuation strategy. The combinations verified so far:
>
> Status: ✅ verified, ❌ not supported.
> Status: ✅ verified, ⚠️ supported with issues (continuation works, but with the problems described in
> the notes), ❌ not supported.
>
> **SGLang (NVIDIA GPU, tested on v0.5.19+)**
>
Expand Down Expand Up @@ -35,8 +36,10 @@ English | [简体中文](README.zh-CN.md)
> | Qwen3-30B-A3B | ✅ Verified | Tested with the W8A8 weights |
> | Qwen3-14B | ✅ Verified | |
> | Qwen3-8B | ✅ Verified | |
> | GLM-5.3-Flash | ✅ Verified | Tested with the W8A8 weights |
> | Gemma-4-31B-it | ✅ Verified | The model itself is occasionally unstable: it may emit its control tokens as plain text in `content` (for example `</think>`, `<\|tool_call>`) so that the tool_call is not parsed and is lost. This comes from the model and its parser, not from how the continuation prefix is built |
> | Gemma-4-26B-A4B-it | ✅ Verified | Same as Gemma-4-31B-it. Its reasoning may also occasionally end up in `content` |
> | DeepSeek-V4.1-Flash-1017 | ⚠️ Supported with issues | Tested with the W4A8 weights. Requests are continued and tool_calls are produced, but the model's internal special tokens may leak into the continued output as plain text (for example `<ds_safety>…</ds_safety>`, `<DS_STREAMING>`, `@@INLINE_ICON@@`), in which case the answer may be cut short or end up in `reasoning` instead of `content`. A continuation from a short prefix may also drift off topic or switch language. The seam can occasionally lose or gain a character |
> | Qwen3-Omni-30B-A3B-Instruct | ❌ Not supported | |
>
> The model names in the tables are exactly the strings `CONTINUATION_MODEL` should be set to
Expand Down
4 changes: 3 additions & 1 deletion README.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
> 网关支持 **SGLang** 和 **vLLM** 两种推理引擎，通过 `CONTINUATION_ENGINE` 指定。本项目针对
> 不同的「引擎 + 模型」组合各有对应的续写策略，目前已验证的组合如下：
>
> 状态：✅ 已验证，❌ 不支持。
> 状态：✅ 已验证，⚠️ 支持但有问题（续写可用，但存在备注里描述的问题），❌ 不支持。
>
> **SGLang（NVIDIA GPU，在 v0.5.19+ 版本上验证）**
>
Expand Down Expand Up @@ -34,8 +34,10 @@
> | Qwen3-30B-A3B | ✅ 已验证 | 使用 W8A8 权重测试 |
> | Qwen3-14B | ✅ 已验证 | |
> | Qwen3-8B | ✅ 已验证 | |
> | GLM-5.3-Flash | ✅ 已验证 | 使用 W8A8 权重测试 |
> | Gemma-4-31B-it | ✅ 已验证 | 模型自身偶发不稳定：可能把控制 token 当作普通文本输出到 `content`（例如 `</think>`、`<\|tool_call>`），导致 tool_call 没有被解析出来而丢失。这来自模型及其解析器，不是续写前缀的构造问题 |
> | Gemma-4-26B-A4B-it | ✅ 已验证 | 同 Gemma-4-31B-it；另外它的思考内容偶发会落进 `content` |
> | DeepSeek-V4.1-Flash-1017 | ⚠️ 支持但有问题 | 使用 W4A8 权重测试。请求能够续写、tool_call 也能产出，但模型内部的特殊 token 可能以普通文本泄漏到续写内容里（例如 `<ds_safety>…</ds_safety>`、`<DS_STREAMING>`、`@@INLINE_ICON@@`），此时回答可能被截断，或落进 `reasoning` 而不是 `content`；前缀较短时续写还可能跑题、切换语言；断开处偶尔会丢字或多出字符 |
> | Qwen3-Omni-30B-A3B-Instruct | ❌ 不支持 | |
>
> 表中的模型名就是 `CONTINUATION_MODEL` 应设置的字符串（大小写不敏感，精确匹配）。表中模型
Expand Down
Loading