fix(tooling): 文档审计范围改为从目录派生,清单与磁盘不一致时判红 (#4851) - #4921
Merged
Conversation
… of a hand-kept list (#4851) `.claude/workflows/docs-accuracy-audit.js` carried its default audit scope inline, as a hand-kept `ALL_HANDWRITTEN` array behind a "keep in sync with `affected-docs.mjs --all`" comment. Nothing checked that promise, and it had rotted in BOTH directions: - 16 listed paths no longer existed — 10 of them the whole `content/docs/protocol/objectos/**` directory, renamed to `protocol/kernel/`. An audit agent pointed at a non-existent file reads nothing and reports `fixCount: 0`, indistinguishable in the run summary from a doc that was checked and found accurate. That is how #4781 and #4817 sat in `protocol/kernel/` for ~2 months under green "full" audits. - 48 existing docs were absent from the list — including all 9 of `protocol/kernel/**` and the whole `content/docs/capabilities/` directory. A run logging `FULL audit` was auditing 130 of 178 hand-written docs. The list stays inline because it must: a workflow script runs in a `node:vm` context with no require/import/filesystem, so it can neither walk `content/docs/` nor read a JSON artifact. So it is GENERATED instead — `scripts/docs-audit/check-audit-scope.mjs --write` derives it from `affected-docs.mjs --all` (one definition of "hand-written doc"), and `pnpm check:docs-audit-scope` fails in lint.yml when the block and `content/docs/` disagree in either direction, naming every entry. Missing markers or an unparseable block fail too — a gate that cannot find its subject must go red, not green. A CI gate only sees the default list, so two more nets cover the rest: - the workflow preflights its resolved scope (including a caller-supplied `args.docs`) and refuses to start, naming every path that does not exist; the preflight's own arithmetic is reconciled against the scope, so a verdict that cannot account for every path exactly once is a failed preflight, not a pass; - every audit agent reports `docExists` from the path that actually opens the file, and the run throws if any comes back false — a self-check that runs somewhere other than the real read path proves nothing about it (#4868). Same discipline as #4690 / #4777 / #4804 / #4835 / #4868 / #4890. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018iARDqtrhQgz6fVHDeDkbQ
…s-audit-dead-roots # Conflicts: # package.json
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
📓 Docs Drift CheckNo hand-written docs reference the 0 changed package(s). ✅ |
xuyushun441-sys
marked this pull request as ready for review
August 3, 2026 16:23
This was referenced Aug 3, 2026
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4851
实情先行:议题说的是「11 条 objectos 路径」,实测烂得更彻底
议题正文的写法本身就是本单要防的东西,所以我没有信它,而是从仓库现状重新测量:
runtime-capabilities.mdx)ai/chatbot-integration、deployment/cloud-artifact-api、getting-started/cli、getting-started/validating-metadata、permissions/roles、ui/role-based-interfacesprotocol/kernel/**全部 9 页、整个content/docs/capabilities/目录、releases/v12–v17等也就是说:一次打印
FULL audit (no args.docs given)的运行,实际审的是 178 篇里的 130 篇。议题只问了第一个方向(死条目),而 第二个方向(漏列)烂得多三倍,并且没有任何人问起过 —— 这正是「手写清单」的性质:它两个方向同时腐烂,且两个方向都不出声。
为什么这 16 条会烂,以及为什么烂了两个月没人知道
清单头上原本写着「Keep in sync with
node scripts/docs-audit/affected-docs.mjs --all」—— 一句没有任何东西检查的承诺。目录改名(protocol/objectos/→protocol/kernel/)是一次纯content/**的改动,不触发任何与该清单相关的门禁,清单就地失效。失效之后没有任何信号,是因为审计 agent 拿到一条不存在的路径,读不到文件,返回
fixCount: 0—— 在运行摘要里,这和「这篇审过了,没问题」长得一模一样。#4781(runtime-capabilities.mdx整页教一个 #3605 已删的 schema)和 #4817(http-protocol.mdx把 dispatcher 响应挂在/api/v1/discovery名下)两个真实失准就是这样在绿色的 full audit 底下活了约两个月。清单能不能派生?能 —— 但不能在运行时派生
先回答议题里那个更根本的问题。答案分两半,分界线是工作流脚本的执行环境:
content/docs/**/*.mdx减去references/,没有任何人工挑选。这个定义已经存在于affected-docs.mjs --all。node:vm上下文里,全局只有log/phase/console/budget/定时器,加上agent/parallel/pipeline/workflow/args,并且codeGeneration关闭 —— 没有require、没有import、没有文件系统、没有eval。所以脚本既无法自己遍历content/docs/,也无法去读一个 JSON 产物。清单必须内联。结论:在生成时派生。清单改为一段生成产物,只是恰好住在一个手写文件里:
派生源刻意复用
affected-docs.mjs --all(子进程调用)而不是再写一遍目录遍历 —— 「手写文档」只能有一个定义;第二次遍历就是下一个会漂移的东西。新校验的双向证明
绿(当前分支状态)
红 #1 —— 把一条路径改回改名前的写法
红 #2 —— 在磁盘上真的把目录改名回去(即 #4851 的原始机制本身)
红 #3 —— 把生成标记删掉(门禁找不到自己的审查对象)
红 #4 —— 手工从清单里删掉一条(清单本身没有死条目,只是漏列)
恢复后重新判绿,
git status干净。四种红都不是「跑完了报个漂亮的绿」,而是点名到具体条目。CI 门禁只看得见默认清单 —— 所以另外补了两张网
args.docs是这个工作流最常见的调用方式(affected-docs.mjs的输出直接传进来),而没有任何 CI 门禁能看见调用方传了什么。一条坏路径在那里失败的方式,和 #4851 里那份烂清单完全一样。DOCS之后、进入昂贵的 pipeline 之前,先派一个 agent 机械地ls每一条路径,任何一条解析不到就throw并点名。这一步必须走 agent,因为 VM 没有文件系统 —— 也正因如此它是「报告」而不是「测量」,所以它自己的算术要被核对:返回的路径必须与 scope 不重不漏地一一对应,对不上就按「前置校验失败」处理,而不是当成通过。(自检本身没人检 =merge.os-regen.driver指向「上一个装过依赖的 worktree」的绝对路径 —— 该 worktree 一删,全容器的生成物合并驱动就坏了 #4868。)docExists:每个审计 agent 现在必须报告它到底有没有打开成文件,任何一条false都让整轮throw。前置校验和真正读文件的不是同一条调用路径,而merge.os-regen.driver指向「上一个装过依赖的 worktree」的绝对路径 —— 该 worktree 一删,全容器的生成物合并驱动就坏了 #4868 的教训正是:跑在别处的自检,证明不了真实路径上的任何事。于是三层各管一段:门禁管默认清单,前置校验管调用方的清单,真实读取路径两个都复核。
扫到
protocol/kernel之后有没有发现真实失准按 PM 要求不顺手修,只如实报告。我这边跑不了那个 LLM 工作流(Workflow 工具不在我的工具集里,且 9 页 ≈ 18 个 agent),所以做的是一轮机械抽查,不能替代真正的审计:
HOUSE_FACTS词表(os studio/client.ai.chat//_studio/AUTH_SECRET/objectstack-ai/spec/ …)零命中;@objectstack/*包名逐一对照 workspace,3 处不存在的(@objectstack/service-ai、@objectstack/ui、@objectstack/testing)都带着显式的 "proposed / target / 不存在" 说明,不是失准;@objectstack/service-ai那条更是与packages/spec/src/system/core-services.zod.ts:142逐字相符;objectos残留:0。结论:机械抽查未发现新的失准。 这不等于那 9 页干净 —— 真正的实现级审计还没跑过,建议合并后按正常流程跑一轮 scoped audit。
顺带发现(已按 Prime Directive #10 立单,未在本 PR 修)
#4920 —— 审计工作流的交付物是就地改写 mdx,而它的范围包含
content/docs/releases/**,与 AGENTS.md「releases 是 release-owned,禁止在代码 PR 中编辑」直接冲突。这不是本 PR 引入的(旧清单里已有 3 页 releases),但派生后从 3 页变成 9 页,第一次显眼。修它等于在本仓引入第二套「手写文档」定义,是需要维护者拍板的口径决定,故立单不改。共享文件
本 PR 触到两个本仓最热的冲突点,均为新增一行/一步,无改写:
package.json:新增check:docs-audit-scope(合并main时与 fix(scripts): check:doc-authoring 覆盖 .claude 语料,worktrees 进 SKIP_PATHS (#4913) #4915 在相邻行冲突过,已保留对方的--self-test改动);.github/workflows/lint.yml:在lintjob 里新增一步。放这里是刻意的 —— 会打破这份清单的改动是一次文档改名,packages/**的 paths filter 会让门禁恰好对自己的失效模式失明(.claude/skills/**的 markdown 不被任何门禁扫描 —— check:nul-bytes 只看 JS/TS,check:doc-authoring 的 ROOTS 不含 .claude/ #4890 那一族)。验证
新增的
--self-test13 条用例是密封的(不依赖仓库状态),覆盖:标记缺失必须 throw、手写形态的数组必须 throw、渲染/解析往返、以及差异的两个方向各自单独钉住(议题只问了dead,而unlisted才是三倍的洞)。Generated by Claude Code