chore(scripts): the raw-byte gate scans the whole C0 control set, not only NUL (#5157) by os-zhuang · Pull Request #5461 · objectstack-ai/objectstack

os-zhuang · 2026-08-05T12:50:27Z

Fixes #5157
Closes #4958 —— 见下方"与 #4958 的关系"。

前提核验

成立,而且比 issue 说的更严重。scripts/check-nul-bytes.mjs 在 origin/main(01c0baef9)上确实只扫 0x00(扫描循环是 buf.indexOf(0))。但按 issue 建议的区间对全仓做逐字节扫描,发现当下就有 4 个受追踪源文件、共 6 枚裸 C0 控制字节从这道门禁下溜着,不是理论缺口:

文件	行	字节
`packages/cli/src/commands/login.ts`	38	0x03
`packages/cli/src/commands/register.ts`	34	0x03
`packages/services/service-analytics/src/strategies/cross-object-rebucket.ts`	121、131	0x01 × 2
`packages/services/service-storage/src/verify-file-references.ts`	107	0x01 × 2

一处对 issue 论证的修正(实测,不是外推)

issue 正文猜测"grep/ripgrep 的二进制判定对其他控制字节同样敏感(实现相关)"。实测不成立,这一点很重要,所以没有照抄进门禁文案:

with-nul.txt  → grep: binary file matches      / rg: binary file matches (found "\0" byte around offset 11)
with-soh.txt  → grep: 1:alpha sep='' beta      / rg: 1:alpha sep='' beta        (0x01,照常匹配)
with-etx.txt  → grep: 1:alpha sep='' beta      / rg: 1:alpha sep='' beta        (0x03,照常匹配)

(GNU grep 3.11 / ripgrep 14.1.0)

所以本次扩面不是把 NUL 的论证外推到别的字节 —— 那会是一条自己证不了的门禁文案。真正的理由是另一条独立的、落在整个 C0 集上的危害,而且上面那三行输出本身就是证据:这些字节渲染为空,连 grep 的输出里都是空的。本 PR 清掉的两个真实样本,读起来都是空串,而它们是承重的:

const key = keyParts.join('U+0001 的裸字节');   // 所有人看到的:keyParts.join('')
return `${object}U+0001${recordId}U+0001${field}`;  // 所有人看到的:三段直接拼接

第一处是"一个显然多余、下一个读者会顺手删掉"的 join('');第二处更糟 —— "复合键没有分隔符"正是经典碰撞 bug 的样子,读者会去修一个并不存在的缺陷。cross-object-rebucket.ts 的注释同样是自我抹除的:// … is a separator no group value contains``,反引号之间本应是那个字符。对每一个读者说谎的代码,比 grep 找不到的代码更糟,因为没有任何信号提示还存在第二种读法。而本意写该转义的作者,两种拼写都搜不到:搜转义文本搜不到(文件里是字节),那个字节也没法敲进搜索框。

第三条理由是事故源:本仓每一例都来自"作者正在写关于这个字节的内容时,编辑工具把转义落成了真字节"(#4763、#4890、PR #5140)。该事故源不挑字节值,所以门禁也不该挑。

双向证明(先定方向,再跑)

预期方向:旧门禁在未修的树上判绿(盲区),新门禁在同一棵树上判红(扩面生效),修完判绿。三次都如预期:

① 旧门禁(origin/main 版脚本)× 未修的树
   check-nul-bytes: OK (scanned 5442 tracked text file(s); … no raw NUL bytes).   EXIT=0

② 新门禁 × 同一棵未修的树
   check-nul-bytes: 4 files contain a raw C0 control byte
     • packages/cli/src/commands/login.ts:38:15 -- 1 occurrence of 0x03, first at byte offset 1336
     • packages/cli/src/commands/register.ts:34:15 -- 1 occurrence of 0x03, first at byte offset 1176
     • …/cross-object-rebucket.ts:121:67 -- 2 occurrences of 0x01, first at byte offset 5087
     • …/verify-file-references.ts:107:20 -- 2 occurrences of 0x01, first at byte offset 4461
   Write the escape sequence instead of the byte:
       0x01  ->  (U+0001 的转义序列)
       0x03  ->  (U+0003 的转义序列)                                              EXIT=1

③ 新门禁 × 修完的树
   check-nul-bytes: OK (scanned 5442 tracked text file(s); skipped 5 binary, 1 non-regular; no raw C0 control bytes).   EXIT=0

自测里也把这条钉住了,而且钉的是"为什么当年判绿"而不只是"现在判红":每个新样本文件整个文件都不含 NUL,所以旧的 buf.indexOf(0) 扫描确实无事可做 —— 断言直接检查这一点,免得日后有人以为样本是靠构造才变红的。把扩面回退成 NUL-only,自测立刻 8 条失败(变异测试已跑)。

关键设计点:二进制判据要不要跟着扩(PM 点名)

要,而且这一步是承重的,不是顺手对齐。

原判据是"剔除 NUL 后整文件 UTF-8 严格解码,解不通才算二进制"。issue 说"0x01 是合法 UTF-8 单字节,不会被解码判定挡住"—— 单独出现时确实如此,但塞进多字节序列中间就会:E4 B8 01 AD 是「中」被塞进一个 0x01,只剔 NUL 的话它解码失败 → 文件判二进制 → 跳过 → 那个 0x01 成了自己的不在场证明。这正是本门禁存在的理由(git 对 NUL 掉进去的那个循环),只是换了一个字节值。自测有专门样本 docs/split-sequence.md 钉住这条,并同时断言"只剔 NUL 的话它会被误判二进制"。

反方向不会误伤:扫描集全部不超过 0x1f,而合法 UTF-8 多字节序列只由不小于 0x80 的字节构成,所以剔除它们永远不会破坏一个本来合法的序列。验收自测按 PM 的两条标准做了实测 —— 全部 5448 个受追踪路径,扩面后文本/二进制判定零变化(仍是 4 PNG + 1 ICO 跳过,5442 个文本文件被扫)。也就是说这条反循环性质是白拿的。

仓内没有合法含 C0 字节的受追踪文本(那 6 枚全是缺陷),所以不需要按 #4890 先例开豁免 —— 门禁的"无 per-file 豁免口"保持不变。

命名:保留旧名(按"引用点 × 破坏面"自判)

保留 scripts/check-nul-bytes.mjs 与 pnpm check:nul-bytes。功能引用点 2 个(package.json、lint.yml),散文引用点 5 个,其中三个改不动或不该改:packages/lint/scripts/check-doc-formula-expressions.mjs:263(该包文件面由 #5417 在飞占用,本单明令不触)、.claude/agents/os-dev.md 与 .claude/skills/pm-dispatch/SKILL.md(agent 配置)。改名换来的是名字更准,代价是一次跨文件、只改了一半的重命名 —— 而半改的重命名正是下一个 agent 会踩的坑。语义变化改为写在三处:脚本头、报错文案、CI 步骤名(Raw NUL byte guard → Raw control-byte guard)。#4958 建议过改名成 check:control-bytes,#5417 落地后可作为独立的一次性重命名,已在报告里留作 open question。

与 #4958 的关系

搜重时发现 #4958(08-03,未认领、未入队)是本单的姊妹单:它点名的正是本 PR 修的那两处 0x01(cross-object-rebucket.ts:131、verify-file-references.ts:107),给出的建议方向 (1) 就是本 PR 实现的东西,并且记录了这个字节已经造成的实际损失 —— #4821 整张 issue 的机制、复现、方向裁决全部建立在一段复制时字节被吃掉的引文上,复现其实不成立,直到实施阶段逐字节 cat -A 才发现。本 PR 把它的两处样本和它推荐的门禁一起落地,故一并 close;若维护者认为该单应独立走,把 Closes 行删掉即可。

顺带发现(未在本 PR 修)

login.ts:47 / register.ts 各有一枚裸 DEL(0x7f) 当 Backspace 键值,与本 PR 刚转义的 0x03 同一个 switch、隔九行。0x7f 不是 C0 控制符,不在 issue 明文给的区间里 —— 扩不扩是门禁扫描面的定义问题,不该由实施顺手扩,已按 Prime Directive #10 单独记录为 #5460(finding,未入队)。自测里有一条断言故意把 0x7f 钉在扫描面之外,说明这条边界是选出来的而不是漏掉的。

测试

node scripts/check-nul-bytes.mjs --self-test
  ✓ check-nul-bytes --self-test: 34 assertions over a temp git repo (real scan() path)   (原 16)
node scripts/check-nul-bytes.mjs
  check-nul-bytes: OK (scanned 5442 tracked text file(s); skipped 5 binary, 1 non-regular; no raw C0 control bytes).

pnpm --filter @objectstack/cli test                → Test Files 79 passed (79) / Tests 767 passed (767)
pnpm --filter @objectstack/service-analytics test  → Test Files 47 passed (47) / Tests 745 passed (745)
pnpm --filter @objectstack/service-storage test    → Test Files 21 passed (21) / Tests 283 passed (283)
pnpm --filter @objectstack/cli typecheck           → tsc --noEmit, EXIT=0
eslint(5 个改动文件)                              → EXIT=0
node scripts/check-changeset-fixed.mjs             → ✓ in sync with 69 public workspace packages

耗时 505ms → 586ms(全仓 5443 个文本文件),扩面成本可忽略。

一条实测到的元证据

本 PR 的 changeset 与 #5460 的第一版正文,在写"不要写裸控制字节"的过程中各自落进了裸控制字节 —— changeset 那两枚是被本 PR 新加的扫描面当场抓住的(旧门禁会放行)。这是 #4890 那次事故的第五、六例,也是 #4958 里"工具链本身就会不断重新引入这类字节,人工守则挡不住"的实测复现。

🤖 Generated with Claude Code

https://claude.ai/code/session_01GX3sL71LFq8m2usg6VqTSE

Generated by Claude Code

… only NUL (#5157) `check:nul-bytes` shipped from #4890 scanning 0x00 alone, because 0x00 was the byte whose harm its own failure message could prove: a raw NUL makes grep and ripgrep classify the whole file as binary and silently return zero matches. Measured, that argument really is NUL-specific — GNU grep 3.11 and ripgrep 14.1 report "binary file matches" for a file carrying 0x00 and keep matching normally for one carrying 0x01 or 0x03 — so this change is not that argument extended by assertion. It is a second harm that lands on the whole C0 set. The other C0 controls render as NOTHING wherever a human reads the code. Both specimens this commit removed from the tree read as an empty string while being load-bearing: const key = keyParts.join('<0x01>'); // shows as: keyParts.join('') return `${object}<0x01>${recordId}`; // shows as: plain concatenation grep prints the match, the diff prints the line, and review sees `join('')` — an obviously pointless call a later reader is invited to delete, or a separator-less composite key a later reader is invited to "fix". Code that lies to every reader is worse than code grep cannot find, because nothing signals a second reading exists. Nor can the author search for it: not the escape text (the file holds a byte) and not the byte (nobody can type it). And the accident source does not pick byte values — every occurrence in this repo came from an editing tool materialising an escape while someone wrote ABOUT the byte (#4763, #4890, and PR #5140, where the caught NUL was fixed and a 0x01 fourteen bytes away was not). Scanned set is now `[\x00-\x08\x0b\x0c\x0e-\x1f]`: the C0 range minus tab, LF and CR — the pattern #4890's own manual sweep used before the gate narrowed to NUL. The binary probe widens with it, and that step is load-bearing rather than cosmetic tidiness. A control byte CAN break an otherwise-valid multi-byte sequence: `E4 B8 01 AD` is 中 with a 0x01 spliced into it, and stripping only NUL leaves that undecodable, so the file is skipped as binary and the 0x01 becomes its own alibi — the exact circularity this gate exists to break, one byte value over. Widening cannot err the other way: every scanned byte is <= 0x1f while valid UTF-8 multi-byte sequences are built only from bytes >= 0x80. Measured over all 5448 tracked paths, the widened stripping moves zero files between text and binary. Six raw control bytes already in the tree are escaped here; the NUL-only gate was green over all of them. `login.ts` / `register.ts` carried a 0x03 Ctrl+C case label, `cross-object-rebucket.ts` a 0x01 bucket-key separator (in the comment and in the `join`), `verify-file-references.ts` two 0x01 in `slotKey`. Escaped strings are byte-identical at runtime, so no behaviour changes and nothing ships. The script and its `pnpm check:nul-bytes` command keep their historical names: those strings are referenced from CI, from other gates' comments and from agent instruction files, several owned by other in-flight work, and a rename buys a more accurate name at the price of a half-applied one. The widened semantics are stated in the script header, the failure message and the CI step instead. `--self-test` goes 16 -> 34 assertions. The new ones pin the widening in both directions: every new specimen file contains no NUL anywhere, so the pre-#5157 `buf.indexOf(0)` scan had nothing to find, and reverting the widening turns the self-test red with 8 failures rather than leaving it quietly green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GX3sL71LFq8m2usg6VqTSE

vercel · 2026-08-05T12:50:35Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 5, 2026 12:50pm

github-actions · 2026-08-05T12:50:48Z

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/cli, @objectstack/service-analytics, @objectstack/service-storage.

27 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

content/docs/ai/skills-reference.mdx (via packages/cli)
content/docs/api/client-sdk.mdx (via @objectstack/cli)
content/docs/api/data-api.mdx (via @objectstack/service-analytics)
content/docs/api/data-flow.mdx (via @objectstack/cli)
content/docs/api/environment-routing.mdx (via @objectstack/cli)
content/docs/api/error-catalog.mdx (via @objectstack/cli)
content/docs/api/index.mdx (via @objectstack/service-analytics)
content/docs/api/plugin-endpoints.mdx (via @objectstack/service-storage)
content/docs/automation/hook-bodies.mdx (via packages/cli)
content/docs/deployment/backup-restore.mdx (via @objectstack/cli)
content/docs/deployment/cli.mdx (via @objectstack/cli)
content/docs/deployment/self-hosting.mdx (via @objectstack/cli)
content/docs/deployment/validating-metadata.mdx (via packages/cli)
content/docs/getting-started/your-first-project.mdx (via @objectstack/cli)
content/docs/kernel/runtime-services/data-service.mdx (via packages/cli)
content/docs/kernel/runtime-services/index.mdx (via packages/cli)
content/docs/kernel/services-checklist.mdx (via @objectstack/service-analytics, @objectstack/service-storage)
content/docs/permissions/authentication.mdx (via @objectstack/cli)
content/docs/permissions/sharing-rules.mdx (via @objectstack/service-analytics)
content/docs/plugins/index.mdx (via @objectstack/cli)
content/docs/plugins/packages.mdx (via @objectstack/cli, @objectstack/service-analytics, @objectstack/service-storage)
content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/cli)
content/docs/protocol/kernel/realtime-protocol.mdx (via @objectstack/cli)
content/docs/releases/implementation-status.mdx (via @objectstack/cli, @objectstack/service-analytics, @objectstack/service-storage)
content/docs/releases/v16.mdx (via @objectstack/cli)
content/docs/releases/v17.mdx (via @objectstack/cli, @objectstack/service-analytics)
content/docs/releases/v9.mdx (via @objectstack/service-analytics)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

github-actions Bot added documentation Improvements or additions to documentation ci/cd tooling labels Aug 5, 2026

github-actions Bot added the size/m label Aug 5, 2026

os-zhuang mentioned this pull request Aug 5, 2026

裸控制字节(U+0001)作为分隔符写进源码:不可见、能让 issue 写错前提,而 check:nul-bytes 只拦 0x00 #4958

Closed

os-zhuang marked this pull request as ready for review August 5, 2026 12:55

os-zhuang mentioned this pull request Aug 5, 2026

check:nul-bytes 只扫 NUL(0x00)—— 0x01-0x08 等控制字节不在扫描面,#5140 实测一个 0x01 会从 NUL-only 修复下溜走 #5157

Closed

os-zhuang enabled auto-merge August 5, 2026 12:55

os-zhuang mentioned this pull request Aug 5, 2026

DEL(0x7f)在 C0 扫描面之外:login.ts / register.ts 各有一枚裸 0x7f 当 Backspace 键值,与刚转义的 0x03 同处一个 switch #5460

Closed

os-zhuang added this pull request to the merge queue Aug 5, 2026

os-zhuang mentioned this pull request Aug 5, 2026

Check Changeset 的失败文案把「空 changeset」推荐为出路 —— 而那正是 #4898 静默卡死发布的输入 #5292

Closed

Merged via the queue into main with commit b375f08 Aug 5, 2026
24 checks passed

os-zhuang deleted the claude/issue-5157-control-bytes branch August 5, 2026 13:03

os-zhuang mentioned this pull request Aug 5, 2026

静态扫描:job 级 if: 读 needs.*.outputs.* 必须显式带状态函数(#4928 沉淀的规则 + publish-smoke.yml 唯一存量违规) #5343

Closed

5 tasks

claude Bot mentioned this pull request Aug 5, 2026

chore(ci): job 级 if: 读 needs.*.outputs.* 必须显式点名状态函数 (#5343) #5477

Merged

Navigation Menu

Uh oh!

chore(scripts): the raw-byte gate scans the whole C0 control set, not only NUL (#5157) - #5461

chore(scripts): the raw-byte gate scans the whole C0 control set, not only NUL (#5157)#5461
os-zhuang merged 1 commit into
mainfrom
claude/issue-5157-control-bytes

os-zhuang commented Aug 5, 2026

Uh oh!

vercel Bot commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

os-zhuang commented Aug 5, 2026

前提核验

一处对 issue 论证的修正(实测,不是外推)

双向证明(先定方向,再跑)

关键设计点:二进制判据要不要跟着扩(PM 点名)

命名:保留旧名(按"引用点 × 破坏面"自判)

与 #4958 的关系

顺带发现(未在本 PR 修)

测试

一条实测到的元证据

Uh oh!

vercel Bot commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

📓 Docs Drift Check

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants