Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,9 @@ Issue 检视结果明确要求代码变更、存在原文证据且无需补充

控制面在自己的数据卷上为每个已启用仓库保留一份 bare mirror(`/data/git-cache/<仓库 ID>.git`,`BOT_GIT_CACHE_DIR` 可改位置),任务不必每次从 GitHub 重新下载整个仓库。任务准备时先把镜像更新到这次要检视或开发的提交,PR 任务还包括读取规则的默认分支提交:镜像里已有这些提交就不访问 GitHub;缺少时按 SHA `git fetch` 这些提交,GitHub 不按 SHA 提供时改取对应的 PR head 或分支 ref,而不是只更新默认分支。同一镜像的更新逐个进行;拉到的每个提交固定在 `refs/bot/commits/<sha>` 下,后续更新和维护不会删掉。安装令牌只作为一次性 HTTP 头经这次 fetch 的进程环境传给 git,只对 GitHub 返回的 clone 地址所在源生效,不写进 remote URL、镜像配置、FETCH_HEAD 或日志,也不进入项目容器;`github_read_token` 仍是只读,语义不变。规则文件和工作区文件随后都从镜像在固定提交上读取,镜像里已有的对象不再逐个调用 GitHub blob API;只有镜像不可用(fetch 失败、仓库没有 clone 地址、拉取后读不出等)或镜像缺某个 blob 时,才回到下面的 Git 数据 API 逐文件读取,任务不会因此失败,日志写明原因。工作副本按文件经 manager 以约 30 KB 分块写进该任务自己的工作目录,两个任务不共用工作目录;runner 校验 blob 与已接收文件的 tree,构建本地 Git 快照,`.git` 里没有 remote 和凭据。副本固定在任务的提交上,之后对镜像的 fetch 不改动已拷出的副本,任务也从不写镜像。副本不是完整上游历史克隆,模型上下文和评论覆盖说明明确标识这一点。文件数超过 2000、单文件超过 1 MiB、合计超过 32 MiB 都不再使准备失败;递归 tree 被截断时丢弃不完整列表,改为逐目录非递归列举。子模块不克隆,子模块和不支持的 Git mode 不下载,日志记录路径和 mode;回到 Git 数据 API 读取时,[GitHub blob API](https://docs.github.com/en/rest/git/blobs#get-a-blob) 支持约 100 MB 以内的 blob,接口明确因文件过大拒绝返回时记录路径、大小和 HTTP 状态并跳过,coverage 列出跳过数量。权限错误、限流和普通 HTTP 错误仍按原有故障处理。跳过条目保留在上游,发布时沿用源 tree 并校验合并后的完整 tree,避免误删。控制面一次仅处理一个文件的内容;快照 blob 下载不受普通 GitHub 响应的 48 MiB 限制,管理接口请求体上限保持不变。runner 的 Git 快照操作不再受 4 MiB 输出缓冲和 60 秒超时限制,读取候选 blob 时落盘后分块返回;模型命令的超时策略不变。初始上下文仍使用 `contextBytes < 100000` 和单文件 12000 字切片,大文件保留在工作区供工具读取。二进制和符号链接按 Git 对象处理,相对文件工具不跟随越界符号链接。哈希不一致、路径越界和工作区被替换仍然失败,说明包含路径、期望与实际 SHA 或具体原因。

任务入队和准备、Agent 执行、策略测试、GitHub 发布阶段开始时即写入任务日志,包含仓库、固定提交 SHA、检视或开发模式、实际测试命令或发布目标;同一次尝试内的阶段开始去重,策略测试按实际命令操作记录,没有策略命令时不写测试开始日志。准备过程中每写完 100 个文件记录已写入数量、字节数和刚写完的路径,每个跳过条目立即记录路径及原因,构建本地 Git 树前再记一条,结束时保留 `snapshot` 汇总,其中记录文件来源(`git_cache` 或 `github_api`)和镜像状态(已有 `hit`、本次拉取 `fetched`、不可用 `unavailable`),跳过说明过长时标明省略条数。Pi 保留模型决策和工具结果事件,并在工具执行前写入 `tool_start`(文件路径、命令或动作名,不含写入内容),经现有 poll 复制到 `task_logs`;runner 在事件落盘前屏蔽已兑换的令牌,控制面继续经 Redactor 脱敏。运行中和成功后、失败或 `waiting_*`、`retry_wait` 的详情均保留完整时间线,故障步骤和侧栏直接展示 fault `detail`,无需打开原始数据;相同故障码、状态和说明在自动重试时只记录一次,自动重试用尽转为失败时另记一条。`src/agent/failures.ts` 把每个故障码归入一类原因并给出下一步,任务和日志接口在读取时附上 `error_cause`/`cause`,旧日志同样适用。GitHub 和模型请求失败写明方法与路径(或模型名与主机)、HTTP 状态、对方的简短说明,以及超时、DNS、连接重置、TLS 等网络原因;GitHub 5xx 附带 `X-GitHub-Request-Id`。日志截断保持合法 JSON,并保留故障说明。
任务入队和准备、Agent 执行、策略测试、GitHub 发布阶段开始时即写入任务日志,包含仓库、固定提交 SHA、检视或开发模式、实际测试命令或发布目标;同一次尝试内的阶段开始去重,策略测试按实际命令操作记录,没有策略命令时不写测试开始日志。准备过程中每写完 100 个文件记录已写入数量、字节数和刚写完的路径,每个跳过条目立即记录路径及原因,构建本地 Git 树前再记一条,结束时保留 `snapshot` 汇总,其中记录文件来源(`git_cache` 或 `github_api`)和镜像状态(已有 `hit`、本次拉取 `fetched`、不可用 `unavailable`),跳过说明过长时标明省略条数。Pi 保留模型决策和工具结果事件,并在工具执行前写入 `tool_start`(文件路径、命令或动作名,不含写入内容),经现有 poll 复制到 `task_logs`;runner 在事件落盘前屏蔽已兑换的令牌,控制面继续经 Redactor 脱敏。运行中和成功后、失败或 `waiting_*`、`retry_wait` 的详情均保留完整时间线,故障步骤和侧栏直接展示 fault `detail`,无需打开原始数据;相同故障码、状态和说明在自动重试时只记录一次,自动重试用尽转为失败时另记一条。`retry_wait` 的自动重试依次等待 5、15、45、135 秒(限流按 `Retry-After`),第 5 次仍失败转为失败。`src/agent/failures.ts` 把每个故障码归入一类原因并给出下一步,任务和日志接口在读取时附上 `error_cause`/`cause`,旧日志同样适用。GitHub 和模型请求失败写明方法与路径(或模型名与主机)、HTTP 状态、对方的简短说明,以及超时、DNS、连接重置、TLS 等网络原因;GitHub 5xx 附带 `X-GitHub-Request-Id`。日志截断保持合法 JSON,并保留故障说明。

开发候选由容器内 Git 生成,控制面保存候选 tree、文件清单、测试输出与本地提交映射,经 Git 数据 API 上传 blob/tree/commit。只有 GitHub 返回 tree 与已验证候选完全一致,才创建或更新登记的 bot 分支。更新固定 force:false;未知或变化的分支 head 会阻止发布。容器里没有 GitHub remote 认证、私钥或模型 Key;控制面只对仓库镜像运行 git fetch、ls-tree 和 cat-file,不检出代码,也不运行仓库里的脚本或钩子,命令仍不在 bot 宿主机上执行。真实仓库访问仍需为 App/账号提供相应 Contents、Issues 和 Pull requests 权限。
开发候选由容器内 Git 生成,控制面保存候选 tree、文件清单、测试输出与本地提交映射,经 Git 数据 API 上传 blob/tree/commit。tree 只把改动、新增和删除的路径叠加在源快照的 tree 上(`base_tree`),不再每次列出整个仓库:大仓库的完整清单会让 GitHub 超过单个请求 10 秒的处理上限,每次重试都返回 502/504。路径在文件和目录之间转换,或 GitHub 算出的 tree 与候选不一致时,退回完整清单。只有 GitHub 返回 tree 与已验证候选完全一致,才创建或更新登记的 bot 分支。更新固定 force:false;未知或变化的分支 head 会阻止发布。容器里没有 GitHub remote 认证、私钥或模型 Key;控制面只对仓库镜像运行 git fetch、ls-tree 和 cat-file,不检出代码,也不运行仓库里的脚本或钩子,命令仍不在 bot 宿主机上执行。真实仓库访问仍需为 App/账号提供相应 Contents、Issues 和 Pull requests 权限。

评论、Review、分支和 PR 都有持久 outbox。请求结果未知时按任务 marker、head 和远端对象先对账;未能确认就保持 waiting_publication,不盲目重复;人工重试视为确认,结果未知的写入先再对账一次,仍查不到才重新发送。分支已推送后,基础分支前进不再阻止创建对应 PR。控制面重启后恢复未完成阶段,已完成任务不重新入队;manager 中断的命令不当作通过,重新检查环境后在新 attempt 工作目录继续,旧目录保留。同代已完成工具通过原操作 ID 回放,generation 变化会阻止旧任务直接写入新环境。

Expand Down
5 changes: 4 additions & 1 deletion src/agent/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,10 @@ export class TaskEngine {
const failure=error instanceof TaskError?error:new TaskError('task_runtime_failure',String((error as Error)?.message).includes('manager')?'waiting_environment':'failed',`任务在 ${task.phase} 阶段发生运行异常:${redactor.text(String((error as Error)?.message||'未知错误')).replace(/\s+/g,' ').slice(0,300)}`);
const current=this.store.get('SELECT * FROM tasks WHERE id=?',id);
const retries=Number(current?.retry_count||0)+(failure.state==='retry_wait'?1:0),state=failure.state==='retry_wait'&&retries>=5?'failed':failure.state;
const delay=failure.code==='github_rate_limited'?(failure.retryAfterMs??60000):state==='waiting_configuration'?30000:5000;
// Automatic retries back off (5 s, 15 s, 45 s, 135 s), so a GitHub or model outage of a few minutes is outlasted
// instead of using up all five attempts in half a minute.
const delay=failure.code==='github_rate_limited'?(failure.retryAfterMs??60000):state==='waiting_configuration'?30000
:state==='retry_wait'?Math.min(5000*3**(retries-1),300000):5000;
if(current?.claim_token===token&&!['cancelled','superseded'].includes(current.status))update({status:state,error_code:failure.code,retry_count:retries,next_run_at:Date.now()+delay});
if(failure.code==='execution_interrupted'&&task.phase!=='publish'&&task.attempt<3){
log({type:'restart_attempt',preserved_workspace:task.workspace_id});
Expand Down
2 changes: 1 addition & 1 deletion src/agent/failures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ const codeActions: Record<string,string> = {
};
const causeActions: Record<FailureCause,(state:string)=>string> = {
external: state => automatic.has(state) ? '不是配置问题,bot 会自动重试,一般无需处理。'
: '已自动重试多次仍失败。确认 GitHub、模型服务和网络恢复后点「重试」。',
: '已自动重试 5 次仍失败。先看 GitHub 状态页、模型服务和网络是否正常;都正常而同一个请求反复返回 502/504 时,多半是请求太大或太慢(GitHub 处理单个请求超过 10 秒会中止),请把任务链接反馈给 bot 维护者。恢复后点「重试」。',
github_access: () => '按说明核对 GitHub App 是否安装到该仓库、授权的权限和仓库范围、账号凭据是否有效,' + retry,
github_rejected: () => 'GitHub 收到了请求但不接受,通常是仓库当前状态与请求冲突(例如分支或 PR 已存在、分支保护规则)。按说明核对仓库后点「重试」;无法解释时请反馈给 bot 维护者。',
model: () => '检查项目的模型地址、模型名和 API Key;模型输出不符合要求时可以点「重试」,或换用其他模型。',
Expand Down
29 changes: 19 additions & 10 deletions src/github/publisher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,22 @@ export class Publisher {
const blob=await this.github.request('POST',this.github.path('/git/blobs'),{content:file.base64,encoding:'base64'});
if(blob.sha!==file.sha)stop('github_blob_mismatch',undefined,`GitHub 写入文件 ${file.path} 后返回的 SHA 不一致:预期 ${file.sha},实际 ${blob.sha}。`);
}
const entries=candidate.files.map((f:any)=>({path:f.path,mode:f.mode,type:'blob',sha:f.sha}));
let expected=candidate.tree_sha;
if(candidate.skipped_files?.length){
// Keep excluded upstream entries, including blobs GitHub will not let us download.
// Hashing the merged manifest also rejects a candidate that overwrites an excluded path.
expected=treeSha([...candidate.files,...candidate.skipped_files]);
const paths=new Set(candidate.files.map((f:any)=>f.path));
for(const file of candidate.snapshot_files)if(!paths.has(file.path))entries.push({path:file.path,mode:file.mode,type:'blob',sha:null});
}
const tree=await this.github.request('POST',this.github.path('/git/trees'),{...(candidate.skipped_files?.length?{base_tree:candidate.source_tree}:{}),tree:entries});
// Skipped upstream entries (submodules, blobs GitHub will not let us download) stay as they are in the source tree.
// Hashing the merged manifest also rejects a candidate that overwrites an excluded path.
const skipped=candidate.skipped_files||[],expected=skipped.length?treeSha([...candidate.files,...skipped]):candidate.tree_sha;
const entry=(f:any,sha=f.sha)=>({path:f.path,mode:f.mode,type:'blob',sha});
const snapshot=candidate.snapshot_files||[],upstream=new Map<string,any>(snapshot.map((f:any)=>[f.path,f])),paths=new Set(candidate.files.map((f:any)=>f.path));
const deletions=snapshot.filter((f:any)=>!paths.has(f.path)).map((f:any)=>entry(f,null));
const changes=candidate.files.filter((f:any)=>upstream.get(f.path)?.sha!==f.sha||upstream.get(f.path)?.mode!==f.mode).map((f:any)=>entry(f));
// Rebuilding a large repository's tree from every path can take GitHub longer than its 10-second request limit,
// which answers 502/504 on every retry, so only the changes go on top of the source tree. A path that turns from a
// file into a directory or back is not an update GitHub applies the same way; it, and any tree GitHub builds
// differently, falls back to the complete listing.
const before=[...upstream.keys(),...skipped.map((f:any)=>f.path)],files=new Set(before),directories=new Set(before.flatMap(parents));
const reshaped=changes.some((f:any)=>directories.has(f.path)||parents(f.path).some(path=>files.has(path)));
const create=(body:any)=>this.github.request('POST',this.github.path('/git/trees'),body);
let tree=candidate.source_tree&&changes.length+deletions.length&&!reshaped?await create({base_tree:candidate.source_tree,tree:[...changes,...deletions]}):null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F1 — medium, suggestion — robustness — a rejected incremental tree request never falls back to the complete listing

  • Where: src/github/publisher.ts:90
  • Problem: the fallback on the next line only runs when GitHub answers with a tree SHA different from expected. If GitHub rejects the POST that carries base_tree instead (4xx such as 422 for an entry it cannot apply on top of the base tree, or a 5xx), create() throws and the complete listing is never tried. Before this change a repository without skipped files always sent a complete listing without base_tree, so there was no base tree for GitHub to refuse.
  • Conditions: a candidate whose delta GitHub refuses to apply to source_tree — a reshape the local reshaped check does not model, or a base tree GitHub no longer resolves. The local check covers file↔directory conversion for every changed/deleted path of the snapshot, so this is hardening rather than a known failing input; I could not produce a concrete rejection on a real repository.
  • Impact: the publish fails (github_http_422 → github_rejected) or repeats the identical request through all five automatic retries, which is the same failure loop this PR is removing, just with a different trigger.
  • Fix: treat a rejected incremental request like a SHA mismatch — try the complete listing once, then fail:
let tree=null;
if(candidate.source_tree&&changes.length+deletions.length&&!reshaped){
  try{tree=await create({base_tree:candidate.source_tree,tree:[...changes,...deletions]});}
  catch(error){if(!(error instanceof TaskError))throw error;tree=null;}
}

An unreferenced tree object left behind is harmless; trees are content-addressed, so the following complete-listing request cannot corrupt the branch.

if(tree?.sha!==expected)tree=await create(skipped.length?{base_tree:candidate.source_tree,tree:[...candidate.files.map((f:any)=>entry(f)),...deletions]}:{tree:candidate.files.map((f:any)=>entry(f))});
if(tree.sha!==expected)stop('github_tree_mismatch',undefined,`GitHub 创建的文件树不一致:预期 ${expected},实际 ${tree.sha}。`);
candidate.published_tree_sha=expected;
const author={name:'Department Bot',email:this.github.authorEmail(),date:candidate.date};
Expand Down Expand Up @@ -136,6 +142,9 @@ export class Publisher {
}
}

// a/b/c.txt -> ['a','a/b']
function parents(path:string):string[]{const parts=path.split('/');return parts.slice(1).map((_,index)=>parts.slice(0,index+1).join('/'));}

export function assertOwnedPR(owned:Row|undefined,pr:any,actorId:string,repositoryId:string): asserts owned is Row {
if(!owned)throw new TaskError('bot_pr_identity_changed','waiting_permission',`PR #${pr.number} 没有对应的机器人归属记录。`);
if(String(pr.id)!==owned.pr_id)stop('bot_pr_identity_changed','waiting_permission',`PR #${pr.number} 的标识与已登记的 PR ${owned.pr_id} 不一致。`);
Expand Down
2 changes: 2 additions & 0 deletions test/agent-process.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,8 @@ test('create_pull_request with empty policy never reruns agent evidence and retu
const state = await fetch(d.endpoint + '/fixture/state').then(r => r.json());
assert.match(state.prs.at(-1).body, /Tests: agent-already-ran-tests/);
assert.match(state.prs.at(-1).body, /not rerun by the control plane/); assert.doesNotMatch(state.prs.at(-1).body, /Exit code:/);
// Only the one edited file goes to GitHub, on top of the snapshot's source tree, and GitHub's tree is the verified one.
assert.deepEqual(state.trees.at(-1), { repository_id: 101, base_tree: d.candidate.source_tree, entries: 1 });
});

test('a later call for the same Issue appends to the same bot branch and PR', async t => {
Expand Down
10 changes: 7 additions & 3 deletions test/failure-causes.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,17 @@ test('a write that landed despite a 500 is found on retry and never sent twice',
h.due();const task=await h.run();assert.deepEqual([task.status,h.comments.length],['succeeded',1]);
});

test('the retry that gives up is logged as its own 失败 entry with the next step',async t=>{
test('automatic retries back off, and the retry that gives up is logged as its own 失败 entry with the next step',async t=>{
const h=await emptyRepositoryFixture(t,{github(path,send){if(path!=='/user')return false;send({message:'Server Error'},502);return true;}});
for(let attempt=1;attempt<=5;attempt++){h.due();await h.run();}
for(let attempt=1;attempt<=5;attempt++){
h.due();const before=Date.now(),task=await h.run(),after=Date.now();if(attempt===5)break;
// Automatic retries back off: 5 s, 15 s, 45 s, 135 s.
const delay=5000*3**(attempt-1);assert.ok(task.next_run_at>=before+delay&&task.next_run_at<=after+delay,`attempt ${attempt}`);
}
const task=h.store.get('SELECT * FROM tasks WHERE id=?',h.taskId);assert.deepEqual([task.status,task.error_code,task.retry_count],['failed','github_http_502',5]);
assert.deepEqual(faults(h).map(log=>log.state),['retry_wait','failed']);
assert.equal(faults(h)[1].detail,'GitHub 请求 GET /user 返回 HTTP 502:Server Error');
const detail=(await h.admin('/api/tasks/'+h.taskId)).value.task;assert.equal(detail.error_cause.cause,'external');assert.match(detail.error_cause.action,/已自动重试多次仍失败/);
const detail=(await h.admin('/api/tasks/'+h.taskId)).value.task;assert.equal(detail.error_cause.cause,'external');assert.match(detail.error_cause.action,/已自动重试 5 次仍失败/);
});

test('model failures name the model, its host and the provider explanation without the key',async t=>{
Expand Down
Loading
Loading