Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"schemaVersion": 1,
"name": "ai-node-canvas",
"displayName": "节点画布",
"version": "0.86.0",
"version": "0.87.0",
"description": "无限画布节点编排:把 MiniMax 模型生成的文本、图片、视频串成工作流,连线即数据流,上游产物可直接被下游引用。",
"author": "antianqi",
"icon": "icon.png",
Expand Down
15 changes: 10 additions & 5 deletions plugins/antianqi/ai-node-canvas/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ English | [简体中文](README.zh-CN.md)
An infinite-canvas node graph for MiniMax Code, modelled on the canvas of
`work.batvid.com`.

Author: [antianqi](https://github.com/antianqi) · Version: `0.86.0`
Author: [antianqi](https://github.com/antianqi) · Version: `0.87.0`

![The canvas with the 图片复刻 template on it: a reference picture goes in, the prompt is read back out of that picture, and a new picture is made from the prompt](docs/preview.png)

Expand Down Expand Up @@ -668,10 +668,15 @@ separate pictures.
- **How long a connector call may take** — the Host puts a timeout on one
connector call (30 seconds by default). A call that runs over it is aborted
and reported as an unknown outcome: it may have been dispatched, and it may
have been charged, so it must never be retried automatically. A slow
generation (an image generation with a reference is the one that hits it
most often) can therefore fail once; the message names the tool and how long
it ran. Video is submitted and then polled, so it is not affected.
have been charged, so it must never be retried automatically. A picture has no
task number to ask about, so the one the provider makes cannot be found
afterwards — and a slow generation (an image generation with a reference is
the one that hits the limit most often) would be lost with it. So a picture
that runs over the limit does not fail: the run goes on by the other road.
The Agent is handed the very request the direct call would have sent and runs
it as a turn, which the Host does not time out, and the picture comes back
from there. The page is shown no failure; the run log says the road changed.
Video is submitted and then polled, so it is not affected.
- **Processes spawned** — `ffmpeg` and `ffprobe`, and nothing else. Both are
looked up on the machine and run with an argument array, never through a
shell, so no part of a path or a parameter can be read as shell syntax.
Expand Down
8 changes: 5 additions & 3 deletions plugins/antianqi/ai-node-canvas/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
MiniMax Code 里的无限画布节点图。每个节点是一次模型运行,每条连线是一条数据通路——
把文本节点连到图片节点,上游文本会自动注入下游提示词。

作者:[antianqi](https://github.com/antianqi) · 版本:`0.86.0`
作者:[antianqi](https://github.com/antianqi) · 版本:`0.87.0`

![画布:铺了「图片复刻」模板——参考图进去,提示词从这张图里反推出来,再照着它重新生成一张](docs/preview.png)

Expand Down Expand Up @@ -419,8 +419,10 @@ provider 有**几种不同的请求**把素材变成视频,不是同一个请
(回环地址除外),请求由运行时自己发出,**不走**宿主连接器。
- **连接器调用的时限** — 宿主对一次连接器调用有超时上限(默认 30 秒),超时的调用会被
中止并报「结果未知」:调用可能已经发出去了,也可能已经扣费,所以**不能自动重试**。
慢的生成(带参考图的图片生成最容易碰到)因此可能失败一次,失败信息里会写明是哪一个
工具、跑了多少秒。视频走「提交 + 轮询」,不受这个时限影响。
图片没有任务号可查,provider 那边做出来的那张也就找不回来——出图超限**不会失败**:
这次运行改走另一条路,把直连本来要发的那份请求原样交给 Agent 跑一轮(宿主不限制 Agent
回合的时长),图从那条路回来。页面上不弹失败,运行日志里写明换了路。
视频走「提交 + 轮询」,不受这个时限影响。
- **启动的进程** — 只有 `ffmpeg` 与 `ffprobe`。两者都在本机查找,调用时用参数数组、
绝不经过 shell,所以路径或参数里的任何字符都不会被当成 shell 语法。具体命令:
- `ffmpeg -hide_banner -loglevel error -y <输入> …` — 每个媒体工具一次调用:
Expand Down
70 changes: 70 additions & 0 deletions plugins/antianqi/ai-node-canvas/miniapp/node/agent-runner.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ const MODES = new Set(['image', 'video', 'audio', 'music', 'text']);
// so it keeps the Agent turn.
const DIRECT_MODES = new Set(['image', 'video', 'audio', 'music']);

// The codes that mean the Host stopped a connector call it had already sent:
// the provider may still be working on it, and a picture has no task number
// to ask about afterwards.
const STOPPED_CODES = new Set(['CONNECTOR_TIMEOUT', 'CONNECTOR_OUTCOME_UNKNOWN']);

const ERROR_TEXT = {
NOT_PERMITTED: '没有 Agent 访问权限(需要 MiniMax Code 3.1.2 或更新版本)',
BUSY: 'Agent 正忙,请稍后重试',
Expand Down Expand Up @@ -383,6 +388,35 @@ function buildPrompt({ mode, segments, prompt, references, params, model, path =
return { agentPrompt: sections.join('\n\n'), compiled, providerPrompt, assets };
}

/**
* What the Agent is told when the direct call was stopped.
*
* The request is the one the direct path would have sent, written out as the
* tool call to make. The Agent decides nothing here: it is handed the same
* request and asked to run it by the other road, because the Host does not
* time a turn out the way it times a connector call out.
*/
function agentImageInstruction({ prompt, params, assets, count = 1 }) {
const request = { prompt: String(prompt ?? '') };
if (params?.ratio) request.aspect_ratio = params.ratio;
if (params?.resolution) request.resolution = params.resolution;
const urls = (assets?.images ?? [])
.map((item) => String(item?.url ?? item ?? ''))
.filter(Boolean);
if (urls.length) request.input_urls = urls;
// A batch is one call with one request per asset: the direct path gives
// every asset a window of its own, but a turn has no such window.
const total = Math.max(1, Number(count) || 1);
const requests = [];
for (let index = 0; index < total; index += 1) requests.push({ ...request });
return [
'刚才那条直连生成通道超时了,现在改走另一条路:请调用图片生成工具重新生成,只调用这一次。',
'参数严格照抄下面这一份,不要改动,也不要额外生成别的:',
JSON.stringify({ requests }),
'生成之后,对每一张图片调用 get_asset_url 换成能访问的链接,并把每条链接都写在回复里。',
].join('\n\n');
}

export function createAgentRunner(context) {
const tasks = new Map();
const nodeTasks = new Map();
Expand Down Expand Up @@ -696,6 +730,9 @@ export function createAgentRunner(context) {
assets: { images: [], videos: [], audios: [] },
// Which path owns this run: the connector (direct) or an Agent turn.
runner: 'agent',
// Set when the direct call was stopped and the run went on by the
// Agent road instead.
degraded: false,
providerTaskId: null,
providerModel: null,
providerStatus: null,
Expand Down Expand Up @@ -836,6 +873,9 @@ export function createAgentRunner(context) {
// plan quota. The Host owns the credentials; no key is handled here.
async function executeConnector(task, { mode, prompt, params, assets, modelId, count = 1, path, frames }) {
const signal = armController(task);
// Set when the direct call was stopped and the run goes on by the Agent
// road instead.
let degrade = false;
try {
const result = await connector.submit({ mode, prompt, params, assets, modelId, signal, count, path, frames });
if (task.cancelled) {
Expand Down Expand Up @@ -876,6 +916,15 @@ export function createAgentRunner(context) {
task.status = 'aborted';
task.error = '';
task.log.push('已取消');
} else if (mode === 'image' && STOPPED_CODES.has(error?.code)) {
// The Host stopped the call and the outcome is unknown: the provider
// may still be working on it. A picture has no task number to ask
// about, so the one it makes cannot be found afterwards.
//
// The page is not told the run failed. The run goes on by the other
// road: the Host does not time an Agent turn out, so the same
// request can run there as long as it needs.
degrade = true;
} else {
task.status = 'error';
task.error = describeError(error?.code ?? 'UNKNOWN', error?.message);
Expand All @@ -884,6 +933,27 @@ export function createAgentRunner(context) {
} finally {
releaseController(task);
}
if (degrade) {
task.degraded = true;
task.runner = 'agent';
task.log.push('直连超过 30 秒,改走 Agent 通道重新生成');
await execute(task, { text: agentImageInstruction({ prompt, params, assets, count }) });
// The Agent's links are the provider's own and they expire, so the
// canvas keeps a local copy: the node still has its picture later.
if (task.status === 'done' && task.media?.length) {
task.media = await Promise.all(task.media.map(async (item) => {
const local = await store.publishRemote(item.url, item.kind).catch(() => null);
return local ? { ...item, url: local } : item;
}));
}
// A turn that ended without a picture is not a picture: the run says
// so, instead of finishing with nothing to show.
if (task.status === 'done' && !(task.media ?? []).length) {
task.status = 'error';
task.error = 'Agent 通道没有返回图片,请再试一次';
task.log.push(`失败:${task.error}`);
}
}
return task;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -332,15 +332,23 @@ export function createConnectorRunner(context, options = {}) {
// calls themselves took. The two are told apart, because only one of
// them is the provider's answer.
const timing = { waitedMs: 0, callMs: 0 };
// The first failure keeps its own code: whether the Host stopped the call
// or the provider refused it decides what the run does next, so the code
// is not flattened into one thing.
let firstError = null;
for (let index = 0; index < wanted; index += 1) {
try {
media.push(...(await callOneAsset(tool, argsFor(index), kind, signal, timing)));
} catch (error) {
if (!firstError) firstError = error;
warnings.push(`第 ${index + 1} 个没有生成出来:${error?.message ?? error}`);
}
}
if (!media.length && warnings.length) {
throw Object.assign(new Error(warnings[0].replace(/^第 \d+ 个没有生成出来:/, '')), { code: 'CONNECTOR_ERROR' });
throw Object.assign(
new Error(warnings[0].replace(/^第 \d+ 个没有生成出来:/, '')),
{ code: firstError?.code ?? 'CONNECTOR_ERROR' },
);
}
return { media, warnings, waitedMs: timing.waitedMs, callMs: timing.callMs };
}
Expand Down
2 changes: 1 addition & 1 deletion plugins/antianqi/ai-node-canvas/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ai-node-canvas",
"version": "0.86.0",
"version": "0.87.0",
"private": true,
"description": "Node-graph canvas MiniApp for MiniMax Code: wire MiniMax model runs into a visual workflow.",
"mcode": {
Expand Down
77 changes: 71 additions & 6 deletions plugins/antianqi/ai-node-canvas/tests/smoke.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2994,30 +2994,30 @@ test('a call that waited for the machine says how long it waited when it fails',
// call, how long it ran, and how long it stood behind another call.
const connector = createMockHostConnector(undefined, {
callDelayMs: 60,
timeoutCall: { tool: 'generate_image', nth: 2 },
timeoutCall: { tool: 'synthesize_speech', nth: 2 },
});
const h = await createHarness({ hostConnector: connector });
t.after(() => h.cleanup());

const first = await fetch(`${h.base}/api/runs`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ nodeId: 'n_ok', mode: 'image', prompt: '第一张' }),
body: JSON.stringify({ nodeId: 'n_ok', mode: 'audio', prompt: '第一段' }),
});
const firstRun = (await first.json()).run;
await waitForCall(connector.calls, (call) => call.toolRef.endsWith('generate-image'));
await waitForCall(connector.calls, (call) => call.toolRef.endsWith('synthesize-speech'));
const second = await fetch(`${h.base}/api/runs`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ nodeId: 'n_cut', mode: 'image', prompt: '第二张' }),
body: JSON.stringify({ nodeId: 'n_cut', mode: 'audio', prompt: '第二段' }),
});
const secondRun = (await second.json()).run;

const doneFirst = await waitFor(h.base, firstRun.id, (item) => item.status === 'done' || item.status === 'error');
const failed = await waitFor(h.base, secondRun.id, (item) => item.status === 'done' || item.status === 'error');
assert.equal(doneFirst.status, 'done');
assert.equal(failed.status, 'error');
assert.match(failed.error, /generate_image/, 'the error does not say which call failed');
assert.match(failed.error, /synthesize_speech/, 'the error does not say which call failed');
assert.match(failed.error, /前面排队/, 'the error does not say how long the call waited');
assert.match(failed.error, /耗时/, 'the error does not say how long the call itself ran');
// A call that the Host stopped leaves nothing behind: the outcome is
Expand All @@ -3028,13 +3028,78 @@ test('a call that waited for the machine says how long it waited when it fails',
const third = await fetch(`${h.base}/api/runs`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ nodeId: 'n_after', mode: 'image', prompt: '第三张' }),
body: JSON.stringify({ nodeId: 'n_after', mode: 'audio', prompt: '第三段' }),
});
const thirdRun = (await third.json()).run;
const doneThird = await waitFor(h.base, thirdRun.id, (item) => item.status === 'done' || item.status === 'error');
assert.equal(doneThird.status, 'done', 'a failed call blocked the one behind it');
});

test('a stopped image call goes on by the Agent road instead of failing', async (t) => {
// The Host stops a call that runs over its window, and a picture has no
// task number to ask about, so the one the provider makes cannot be found
// afterwards. The page is therefore not told the run failed: the run goes
// on by the other road, because the Host does not time a turn out.
const connector = createMockHostConnector(undefined, {
timeoutCall: { tool: 'generate_image', nth: 1 },
});
const h = await createHarness({ hostConnector: connector });
t.after(() => h.cleanup());

const started = await fetch(`${h.base}/api/runs`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
nodeId: 'n_degrade',
mode: 'image',
prompt: '把参考图里的字搬到牌匾上',
params: { ratio: '9:16', resolution: '1K' },
references: [
{
id: 'n_ref',
title: '门头照',
kind: 'image',
text: '',
media: [{ url: 'https://cdn.example.com/ref.png', kind: 'image' }],
},
],
}),
});
assert.equal(started.status, 202);
const runId = (await started.json()).run.id;

// The Agent turn asks for its tool call to be approved before it makes it.
const waiting = await waitFor(h.base, runId, (run) => Boolean(run.pendingPermission));
assert.ok(waiting.pendingPermission, 'the run never reached the Agent');
assert.equal(waiting.degraded, true, 'the run did not say it changed road');
assert.equal(waiting.status, 'running', 'the page was told the run failed');
assert.ok(!waiting.error, 'the page was shown a failure');
assert.ok(
waiting.log.some((line) => line.includes('改走 Agent 通道')),
'the run did not say it changed road',
);

// The Agent is handed the very request the direct path would have sent.
const runCall = h.agent.calls.find((call) => call.kind === 'run');
assert.ok(runCall, 'the Agent was not asked to run');
assert.match(runCall.payload.text, /把参考图里的字搬到牌匾上/, 'the request lost the prompt');
assert.match(runCall.payload.text, /"aspect_ratio":"9:16"/, 'the request lost the ratio');
assert.match(runCall.payload.text, /"resolution":"1K"/, 'the request lost the resolution');
assert.match(runCall.payload.text, /cdn\.example\.com\/ref\.png/, 'the request lost the reference');

const approved = await getJson(h.base, `/api/runs/${runId}/permission`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ decision: 'allowOnce' }),
});
assert.equal(approved.status, 200);

const done = await waitFor(h.base, runId, (run) => run.status === 'done' || run.status === 'error');
assert.equal(done.status, 'done', `the degraded run died: ${done.error}`);
assert.ok(!done.error, 'the degraded run ended with an error');
assert.equal(done.media.length, 1, 'the degraded run produced no picture');
});

test('a finished video that cannot be fetched says so instead of finishing empty', async (t) => {
// The task is done, but the answer carries no link and the video has no
// drive asset. Saying "完成" with nothing behind it is a lie the user would
Expand Down
Loading