From 991c2a45f66ffe0c95c2b59099ee6113c1d00f65 Mon Sep 17 00:00:00 2001 From: Lu Xinlong Date: Fri, 6 Nov 2026 11:54:38 +0800 Subject: [PATCH] Add Ascend910B4 result of deepseek-v4.1-flash-1017 and glm-5.3-flash --- README.md | 5 ++++- README.zh-CN.md | 4 +++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f5d7542..252272e 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,8 @@ English | [简体中文](README.zh-CN.md) > `CONTINUATION_ENGINE`. Within this project, each engine + model combination has its own > continuation strategy. The combinations verified so far: > -> Status: ✅ verified, ❌ not supported. +> Status: ✅ verified, ⚠️ supported with issues (continuation works, but with the problems described in +> the notes), ❌ not supported. > > **SGLang (NVIDIA GPU, tested on v0.5.19+)** > @@ -35,8 +36,10 @@ English | [简体中文](README.zh-CN.md) > | Qwen3-30B-A3B | ✅ Verified | Tested with the W8A8 weights | > | Qwen3-14B | ✅ Verified | | > | Qwen3-8B | ✅ Verified | | +> | GLM-5.3-Flash | ✅ Verified | Tested with the W8A8 weights | > | Gemma-4-31B-it | ✅ Verified | The model itself is occasionally unstable: it may emit its control tokens as plain text in `content` (for example ``, `<\|tool_call>`) so that the tool_call is not parsed and is lost. This comes from the model and its parser, not from how the continuation prefix is built | > | Gemma-4-26B-A4B-it | ✅ Verified | Same as Gemma-4-31B-it. Its reasoning may also occasionally end up in `content` | +> | DeepSeek-V4.1-Flash-1017 | ⚠️ Supported with issues | Tested with the W4A8 weights. Requests are continued and tool_calls are produced, but the model's internal special tokens may leak into the continued output as plain text (for example `…`, ``, `@@INLINE_ICON@@`), in which case the answer may be cut short or end up in `reasoning` instead of `content`. A continuation from a short prefix may also drift off topic or switch language. The seam can occasionally lose or gain a character | > | Qwen3-Omni-30B-A3B-Instruct | ❌ Not supported | | > > The model names in the tables are exactly the strings `CONTINUATION_MODEL` should be set to diff --git a/README.zh-CN.md b/README.zh-CN.md index 0823bf6..bf5fb6c 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -6,7 +6,7 @@ > 网关支持 **SGLang** 和 **vLLM** 两种推理引擎，通过 `CONTINUATION_ENGINE` 指定。本项目针对 > 不同的「引擎 + 模型」组合各有对应的续写策略，目前已验证的组合如下： > -> 状态：✅ 已验证，❌ 不支持。 +> 状态：✅ 已验证，⚠️ 支持但有问题（续写可用，但存在备注里描述的问题），❌ 不支持。 > > **SGLang（NVIDIA GPU，在 v0.5.19+ 版本上验证）** > @@ -34,8 +34,10 @@ > | Qwen3-30B-A3B | ✅ 已验证 | 使用 W8A8 权重测试 | > | Qwen3-14B | ✅ 已验证 | | > | Qwen3-8B | ✅ 已验证 | | +> | GLM-5.3-Flash | ✅ 已验证 | 使用 W8A8 权重测试 | > | Gemma-4-31B-it | ✅ 已验证 | 模型自身偶发不稳定：可能把控制 token 当作普通文本输出到 `content`（例如 ``、`<\|tool_call>`），导致 tool_call 没有被解析出来而丢失。这来自模型及其解析器，不是续写前缀的构造问题 | > | Gemma-4-26B-A4B-it | ✅ 已验证 | 同 Gemma-4-31B-it；另外它的思考内容偶发会落进 `content` | +> | DeepSeek-V4.1-Flash-1017 | ⚠️ 支持但有问题 | 使用 W4A8 权重测试。请求能够续写、tool_call 也能产出，但模型内部的特殊 token 可能以普通文本泄漏到续写内容里（例如 `…`、``、`@@INLINE_ICON@@`），此时回答可能被截断，或落进 `reasoning` 而不是 `content`；前缀较短时续写还可能跑题、切换语言；断开处偶尔会丢字或多出字符 | > | Qwen3-Omni-30B-A3B-Instruct | ❌ 不支持 | | > > 表中的模型名就是 `CONTINUATION_MODEL` 应设置的字符串（大小写不敏感，精确匹配）。表中模型