feat(web): configure generation recovery when deploying models

docs/console-design.md

@@ -337,6 +337,25 @@ DeploySheet -> GET /api/llm/route-options[?route=<existing>]
 | Text | Per-key locale translations where present; new options retain engine text |
 | Ownership | Inferences adapts the Swiss metadata without editing the copied upstream module |
 
+### In-flight generation recovery
+
+The deploy and upgrade forms expose the charts' existing `continuationGateway` values (console #63).
+
+```text
+form -> recovery values -> adaptive compose -> optional timeout reset -> validate -> diff -> apply
+nginx -> continuation gateway -> CART -> engine
+```
+
+| Concern | Decision |
+|---|---|
+| Controls | Enable gateway, actual model identity, idle timeout, continuation execution and tool-call buffering |
+| Upgrade state | Seed from the stored plan once; query refreshes preserve edits |
+| Validation | Check all recorded plan layers, chart compatibility, CART, ModelRoute, actual model, timeout and engine |
+| Advanced YAML | Preserve precedence; stop when its effective settings differ from the recovery controls |
+| Timeout reset | Clearing removes a retained form override by recomposing the validated form without `fromRelease`, pinned to the resolved source and chart |
+| Integrity | Recovery and adaptive cleanup share source pinning and normalized plan comparison; reject unrelated changes |
+| Ownership | Existing plan API and chart values; copied Swiss sources remain unchanged |
+
 ### Unpublished Monitor integration
 
 `modules/inferences/features.ts` holds the single compile-time `ENABLE_MONITOR` flag, false until the Monitor backend is public (console #76).

web/src/modules/docs/content/en-US/deploy.md

@@ -34,6 +34,18 @@ Most fields can stay empty and take the site profile's and chart's defaults.
 
 **Adaptive concurrency** groups the master switch, floor fraction, absolute floor, start point, idle hold, and TTFT decisions together; its internal knobs are available under **Advanced adaptive options**. Turning the switch off hides and clears the group's entered values; turning it back on does not restore them. **Default peer maximum** stays in **Capacity & failover** and still applies when adaptive concurrency is off; it also influences the static maximum used for adaptive bounds. Review the diff before confirming an upgrade.
 
+## Generation recovery
+
+Enable **Routing → Generation recovery** in the deploy or upgrade form. Enter the **Actual model** whose weights are deployed, such as `kimi-k3` or `glm-5.3`. This selects the continuation strategy; it is not the client-facing served name.
+
+- Requires **CART**, **Publish a route**, and `sglang` chart **0.8.11+** or `vllm` chart **0.8.9+**. Composing the plan checks these conditions.
+- **Stream idle timeout** is the time without a chunk after streaming starts; clearing removes the stored form timeout override and inherits catalog, site and chart settings.
+- **Resume interrupted responses** is on by default; off detects and logs interruptions without sending a continuation request.
+- **Buffer tool calls** holds arguments until a complete call can be sent together, so interrupted calls can be regenerated.
+- Requests flow through `openresty → continuation gateway → CART → inference instance`. The gateway is deployed with the model release; review and confirm the diff before applying.
+- Unsupported requests, including structured JSON output, pass through. Verify the actual model and engine combination against the [gateway compatibility documentation](https://github.com/modelsphere/continuation_gateway).
+- Upgrades carry the configured recovery settings forward. Turn the switch off, compose and confirm to remove the gateway. Advanced YAML takes precedence over the form; if it overrides the recovery controls, update it before composing the plan.
+
 ## 3. Review and confirm
 
 **Compose plan** opens the review:

web/src/modules/docs/content/zh-CN/deploy.md

@@ -33,6 +33,18 @@
 
 **自适应并发** 的总开关、下限比例、绝对下限、起始并发、空闲保持和 TTFT 判断位于同一区域，内部参数可从 **高级自适应参数** 添加。关闭后隐藏本组输入并清空已填写参数，再开启时不会恢复这些旧值。**默认节点并发上限** 留在 **容量与故障切换** 中，关闭自适应并发时也有效；它也影响自适应上下限使用的静态最大值。升级时仍需先检查差异再确认。
 
+## 生成中断恢复
+
+在部署或升级表单的 **路由 → 生成中断恢复** 中开启，填写权重对应的 **实际模型**，例如 `kimi-k3` 或 `glm-5.3`。它用于选择续写策略，不是客户端调用时使用的服务模型名。
+
+- 需要启用 **CART**、**发布路由**，并选择 `sglang` chart **0.8.11+** 或 `vllm` chart **0.8.9+**；生成计划时会检查这些条件。
+- **流式输出空闲超时** 控制开始输出后多久没有新片段视为卡顿；清空会移除已有表单超时覆盖，沿用目录、站点和 chart 配置。
+- **执行中断续写** 默认开启；关闭时网关仅检测并记录中断，不发送续写请求。
+- **缓冲工具调用** 会将参数保留到调用完整生成后一次性发送，以便在中断时重新生成。
+- 请求路径为 `openresty → 恢复网关 → CART → 推理实例`，恢复网关随模型 release 部署；最终仍需预览变更并确认。
+- 结构化 JSON 等不支持的请求直接透传。实际模型与引擎的兼容范围见 [恢复网关说明](https://github.com/modelsphere/continuation_gateway)，启用前应验证实际组合。
+- 升级会带出已配置的恢复参数；关闭后再生成、确认计划可移除网关。高级 YAML 的优先级高于表单；如果它覆盖了恢复控件的选择，需先修改高级 YAML 再生成计划。
+
 ## 3. 预览并确认
 
 点 **生成计划** 后进入预览：

web/src/modules/inferences/adaptive-concurrency.ts

@@ -1,5 +1,6 @@
 import type { Form } from "@swiss/components/DeploySettings";
 import type { Plan, PlanRequest } from "@swiss/lib/api";
+import { requestFromPlan, samePlanBaseline } from "./plan-composition";
 import { routeDirectives, type RouteOptions } from "./routeOptions";
 
 type Tree = Record<string, unknown>;
@@ -30,14 +31,7 @@ export function withAdaptiveConcurrency(request: PlanRequest, form: Form, option
   modelRoute.nginx = nginx;
   overrides.modelRoute = modelRoute;
   if (!composed) return { ...request, overrides };
-  return {
-    ...request, fromRelease: undefined,
-    catalog: composed.source.catalogName || catalog || request.catalog,
-    model: composed.source.model, version: composed.source.version,
-    variant: composed.source.variant,
-    release: composed.release.name, namespace: composed.release.namespace,
-    chartVersion: composed.chart.version, overrides,
-  };
+  return requestFromPlan(request, composed, overrides, catalog);
 }
 
 export class AdaptivePlanChangedError extends Error {}
@@ -53,21 +47,9 @@ export async function composeAdaptivePlan(request: PlanRequest, form: Form, opti
   return proposed;
 }
 
-function canonical(value: unknown): string {
-  if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
-  if (value && typeof value === "object") return `{${Object.entries(value).sort(([a], [b]) => a.localeCompare(b)).map(([key, child]) => `${JSON.stringify(key)}:${canonical(child)}`).join(",")}}`;
-  return JSON.stringify(value) ?? "undefined";
-}
-
 export function sameAdaptiveBaseline(before: Plan, after: Plan, options?: RouteOptions | null): boolean {
-  const layers = (plan: Plan) => {
-    const copy = structuredClone(plan.layers ?? {});
-    const values = routeValues(copy.form ?? {});
+  return samePlanBaseline(before, after, (layers) => {
+    const values = routeValues(layers.form ?? {});
     for (const key of Object.keys(values)) if (isAdaptiveOption(key, options)) delete values[key];
-    return canonical(copy);
-  };
-  return before.release.name === after.release.name && before.release.namespace === after.release.namespace
-    && before.chart.name === after.chart.name && before.chart.version === after.chart.version && before.chart.repo === after.chart.repo && before.chart.path === after.chart.path
-    && before.engine === after.engine && before.profile === after.profile && layers(before) === layers(after)
-    && (["catalog", "catalogName", "model", "variant", "version", "hf", "digest"] as const).every((key) => before.source[key] === after.source[key]);
+  });
 }

web/src/modules/inferences/components/DeployForm.tsx

@@ -58,9 +58,10 @@ export interface DeployFormProps {
   catalogRoute?: string;
   // The chart's values and swissd's layers for the Override YAML editor.
   helmValues?: HelmValuesSource;
+  recovery?: ReactNode;
 }
 
-export function DeployForm({ form, onChange, upgrade, submitted, target, cluster, defaults, gpuOptions, gpuListed, gpuResources, capacity, expanded, onExpand, idPrefix, routeOptions, catalogRoute, helmValues }: DeployFormProps) {
+export function DeployForm({ form, onChange, upgrade, submitted, target, cluster, defaults, gpuOptions, gpuListed, gpuResources, capacity, expanded, onExpand, idPrefix, routeOptions, catalogRoute, helmValues, recovery }: DeployFormProps) {
   const t = useT();
   const on = effective(form);
   const f = (k: string) => t(`deploy.fields.${k}`);
@@ -187,6 +188,7 @@ export function DeployForm({ form, onChange, upgrade, submitted, target, cluster
       {section(
         "routing",
         <>
+          {recovery}
           <FloatingField layout="inline" label={label("expose_routed_peer", "exposeRoutedPeer")} hint={hint("expose_routed_peer", "exposeRoutedPeerHint")}>
             {sw("exposeRoutedPeer", f("exposeRoutedPeer"))}
           </FloatingField>

web/src/modules/inferences/components/DeploySheet.tsx

@@ -43,6 +43,9 @@ import { DeployForm } from "@/modules/inferences/components/DeployForm";
 import type { HelmValuesSource } from "@/modules/inferences/components/OverrideYaml";
 import { ExperimentalBadge } from "@/modules/inferences/components/InstallMethodLabel";
 import { AdaptivePlanChangedError, composeAdaptivePlan, withAdaptiveConcurrency } from "../adaptive-concurrency";
+import { composeRecoveryPlan, composeRecoveryValuesRequest, RecoveryPlanChangedError } from "../recovery-compose";
+import { RecoveryFields } from "./RecoveryFields";
+import { mergeValues, planValues, recoveryError, recoveryInputError, recoverySettings, withRecovery } from "../recovery";
 import { readRouteOptions } from "@/modules/inferences/api";
 
 export type DeployTarget =
@@ -138,6 +141,11 @@ function DeploySheetBody({ target, onClose }: { target: DeployTarget; onClose: (
   if (current.data && !seedPlan) setSeedPlan(current.data);
   const deployModel = upgrade ? "" : target.model;
   const initial = useMemo<Form>(() => (upgrade ? (seedPlan ? formFromPlan(seedPlan) : EMPTY) : { ...EMPTY, serviceId: defaultServiceId(deployModel) }), [upgrade, seedPlan, deployModel]);
+  const initialRecovery = useMemo(() => recoverySettings(
+    upgrade && seedPlan ? planValues(seedPlan) : mergeValues(entry.data?.variant.values, profile.data?.profile?.extra),
+  ), [upgrade, seedPlan, entry.data?.variant.values, profile.data?.profile?.extra]);
+  const [recoveryPatch, setRecoveryPatch] = useState<Partial<ReturnType<typeof recoverySettings>>>({});
+  const recovery = { ...initialRecovery, ...recoveryPatch };
   const [form, setForm] = useState<Form>(initial);
   const catalogRoute = upgrade && current.data ? initial.route || initial.serviceId || target.release : undefined;
   const routeOptions = useQuery({
@@ -149,7 +157,7 @@ function DeploySheetBody({ target, onClose }: { target: DeployTarget; onClose: (
   });
   useEffect(() => setForm(initial), [initial]);
   const [seed, setSeed] = useState<SwitchDefaults | null>(null);
-  const dirty = useDirty({ form, version, variant, moveTo, chartVersion }, { form: seed ? reseed(initial, {}, seed) : initial, version: upgrade ? "" : (target.version ?? ""), variant: upgrade ? "" : (target.variant ?? ""), moveTo: "", chartVersion: "" });
+  const dirty = useDirty({ form, version, variant, moveTo, chartVersion, recovery }, { form: seed ? reseed(initial, {}, seed) : initial, version: upgrade ? "" : (target.version ?? ""), variant: upgrade ? "" : (target.variant ?? ""), moveTo: "", chartVersion: "", recovery: initialRecovery });
 
   const [step, setStep] = useState<"form" | "review">("form");
   const [submitted, setSubmitted] = useState(false);
@@ -199,11 +207,23 @@ function DeploySheetBody({ target, onClose }: { target: DeployTarget; onClose: (
   // overrides, which the editor merges itself. Cached per request while the
   // sheet is open, so saving the overrides and editing them again does not
   // read the chart again.
+  const compositionError = (error: unknown) => {
+    if (error instanceof RecoveryPlanChangedError) return new Error(t("deploy.recovery.errors.changedPlan"));
+    if (error instanceof AdaptivePlanChangedError) return new Error(t("deploy.fields.adaptiveChanged"));
+    return error;
+  };
   const valuesForm = { ...form, edits: "" };
-  const valuesRequest = upgrade && !recorded ? undefined : withAdaptiveConcurrency(requestFor(valuesForm), valuesForm, routeOptions.data);
+  const valuesRequest = upgrade && !recorded ? undefined : withAdaptiveConcurrency(withRecovery(requestFor(valuesForm), recovery), valuesForm, routeOptions.data);
   const helmValues = useQuery({
     queryKey: ["helmValues", JSON.stringify(valuesRequest ?? null)],
-    queryFn: () => deployApi.helmValues(valuesRequest!),
+    queryFn: async () => {
+      try {
+        const request = await composeRecoveryValuesRequest(requestFor(valuesForm), recovery, (next) => composeAdaptivePlan(next, valuesForm, routeOptions.data, deployApi.plan, catalogName), catalogName);
+        return await deployApi.helmValues(request);
+      } catch (error) {
+        throw compositionError(error);
+      }
+    },
     enabled: false,
     staleTime: Infinity,
     // Dropped when the sheet closes: they last one install or upgrade.
@@ -219,10 +239,14 @@ function DeploySheetBody({ target, onClose }: { target: DeployTarget; onClose: (
   const compose = useMutation({
     mutationFn: async () => {
       try {
-        return await composeAdaptivePlan(requestFor(form), form, routeOptions.data, deployApi.plan, catalogName);
+        const inputError = recoveryInputError(recovery);
+        if (inputError) throw new Error(t(`deploy.recovery.errors.${inputError}`));
+        const proposed = await composeRecoveryPlan(requestFor(form), recovery, (request) => composeAdaptivePlan(request, form, routeOptions.data, deployApi.plan, catalogName), catalogName);
+        const invalid = recoveryError(proposed, recovery);
+        if (invalid) throw new Error(t(`deploy.recovery.errors.${invalid}`));
+        return proposed;
       } catch (error) {
-        if (error instanceof AdaptivePlanChangedError) throw new Error(t("deploy.fields.adaptiveChanged"));
-        throw error;
+        throw compositionError(error);
       }
     },
     onSuccess: (pl) => {
@@ -446,6 +470,7 @@ function DeploySheetBody({ target, onClose }: { target: DeployTarget; onClose: (
               onChange={change}
               upgrade={upgrade}
               submitted={submitted}
+              recovery={<RecoveryFields settings={recovery} onChange={(patch) => setRecoveryPatch((previous) => ({ ...previous, ...patch }))} />}
               target={targetFields}
               cluster={cluster.data}
               defaults={{ serviceId: upgrade ? undefined : defaultServiceId(target.model), localPath: entry.data?.localPath, localPathTemplate: entry.data?.pathTemplate, image: imageOf(v, entry.data?.imageRepository), cache }}

web/src/modules/inferences/components/RecoveryFields.tsx

@@ -0,0 +1,35 @@
+import { FieldInput, FloatingField, Switch } from "@modelsphere/ui";
+import { useT } from "../i18n";
+import type { RecoverySettings } from "../recovery";
+
+export function RecoveryFields({ settings, onChange }: { settings: RecoverySettings; onChange: (patch: Partial<RecoverySettings>) => void }) {
+  const t = useT();
+  return (
+    <FloatingField
+      layout="inline"
+      label={t("deploy.recovery.title")}
+      hint={t("deploy.recovery.hint")}
+      expanded={settings.enabled}
+      expandedContent={
+        <div className="space-y-3 pb-3">
+          <p className="text-xs text-muted-foreground">{t("deploy.recovery.requirements")}</p>
+          <FloatingField label={t("deploy.recovery.model")} required hint={t("deploy.recovery.modelHint")}>
+            <FieldInput value={settings.model} onChange={(event) => onChange({ model: event.target.value })} />
+          </FloatingField>
+          <FloatingField label={t("deploy.recovery.timeout")} hint={t("deploy.recovery.timeoutHint")}>
+            <FieldInput value={settings.idleTimeout} inputMode="decimal" onChange={(event) => onChange({ idleTimeout: event.target.value })} />
+          </FloatingField>
+          <FloatingField layout="inline" label={t("deploy.recovery.resume")} hint={t("deploy.recovery.resumeHint")}>
+            <Switch aria-label={t("deploy.recovery.resume")} checked={settings.resumeEnabled} onCheckedChange={(value) => onChange({ resumeEnabled: value })} />
+          </FloatingField>
+          <FloatingField layout="inline" label={t("deploy.recovery.buffer")} hint={t("deploy.recovery.bufferHint")}>
+            <Switch aria-label={t("deploy.recovery.buffer")} checked={settings.bufferToolCalls} onCheckedChange={(value) => onChange({ bufferToolCalls: value })} />
+          </FloatingField>
+          <p className="text-xs text-muted-foreground">{t("deploy.recovery.compatibility")}</p>
+        </div>
+      }
+    >
+      <Switch aria-label={t("deploy.recovery.title")} checked={settings.enabled} onCheckedChange={(value) => onChange({ enabled: value })} />
+    </FloatingField>
+  );
+}

web/src/modules/inferences/locales/en-US.json

@@ -419,6 +419,30 @@
       "routing": "Routing",
       "advanced": "Advanced"
     },
+    "recovery": {
+      "title": "Generation recovery",
+      "hint": "Deploy a continuation gateway to resume a stalled or interrupted streaming response once.",
+      "requirements": "Requires CART, a published route, and sglang chart 0.8.11+ or vllm chart 0.8.9+. Advanced YAML overrides these fields.",
+      "model": "Actual model",
+      "modelHint": "The model whose weights are deployed, for example kimi-k3 or glm-5.3. This selects the continuation strategy; it is not the client-facing served name.",
+      "timeout": "Stream idle timeout (seconds)",
+      "timeoutHint": "Time without a chunk after streaming starts. Clearing removes the form override and inherits catalog, site and chart settings.",
+      "resume": "Resume interrupted responses",
+      "resumeHint": "Off detects and logs interruptions without sending a continuation request.",
+      "buffer": "Buffer tool calls",
+      "bufferHint": "Hold tool-call chunks until complete so interrupted calls can be regenerated. Applies only when resuming is on; arguments arrive together.",
+      "compatibility": "Model and engine compatibility must be verified. Structured JSON responses and other unsupported requests pass through without recovery.",
+      "errors": {
+        "changedPlan": "The plan changed while clearing the recovery timeout. Generate and review a new plan.",
+        "overridden": "Advanced YAML overrides the recovery controls. Update its continuationGateway settings before continuing.",
+        "chart": "Generation recovery needs sglang chart 0.8.11+ or vllm chart 0.8.9+. Select a supported chart version.",
+        "cart": "Generation recovery requires CART. Enable CART or turn recovery off.",
+        "route": "Generation recovery requires a published route. Publish a route or turn recovery off.",
+        "model": "Enter the actual model for generation recovery; the client-facing served name is not sufficient.",
+        "timeout": "The recovery idle timeout must be a finite number greater than zero.",
+        "engine": "The continuation engine must match the selected sglang or vllm chart. Check Advanced YAML."
+      }
+    },
     "routeOptions": {
       "advanced": "Advanced route options",
       "advancedAdaptive": "Advanced adaptive options",