Skip to content

refactor(db): hash 规范形态收敛为小写→大写 - #14

Closed
leeefun wants to merge 2 commits into
masterfrom
refactor/hash-canonical-form
Closed

leeefun wants to merge 2 commits into
masterfrom
refactor/hash-canonical-form

Conversation

@leeefun

@leeefun leeefun commented Oct 6, 2026

Copy link
Copy Markdown
Owner

问题

应用层对 hash 大小写不敏感,而唯一索引 ux_h_user_type_hash 是大小写敏感的(SQLite 默认 BINARY)⇒ 同一份内容可能插出 ABC 与 abc 两条记录、R2 里出现两个工作目录。

改动(第一版:只做规范化,不重建表、不引入 COLLATE 迁移)

  • src/hash.ts:新增 normalizeProfileHash(hash) = hash.toUpperCase()。
  • src/db.ts:写边界 entityParams 规范化 Hash;purgeDeletedRecord 的绑定同样规范化;getByTypeAndHash 暂留 LOWER(Hash)=LOWER(?)(兼容历史库里可能存在的小写行)。
  • tools/check-hash-case-conflicts.mjs:只读检测脚本,逐组列出只差大小写的同 (UserId, Type) 行。零冲突才可把查询收敛为 Hash = ?。绝不自动删数据。
  • docs/protocol.md §10:更新「落库 hash 的大小写」「hash 的匹配方式」两行(去掉已漂移的行号,改用函数锚点)。

验证

  • test/fixes.test.ts:写 'abc' 落库为大写,再写 'ABC' 命中同一条(不双份)。
  • test/hash.test.ts:normalizeProfileHash 单测。
  • test/docs.test.ts:findCaseConflicts 判据 + 工具只读性守卫。

tsc/eslint 干净;全量套件 22/22 通过。

后续(不在本 PR)

确认线上库零折叠冲突后,把 getByTypeAndHash 收敛为 Hash = ?。

应用层大小写不敏感,而唯一索引 ux_h_user_type_hash 是大小写敏感的
(SQLite 默认 BINARY)⇒ 同一份内容可能插出 ABC 与 abc 两条、R2 出现两个工作目录。

第一版只做**规范化**,不做表重建、不引入 COLLATE 迁移:
- src/hash.ts:新增 normalizeProfileHash(hash) = hash.toUpperCase()。
- src/db.ts:写边界 entityParams 规范化 Hash;purgeDeletedRecord 的绑定同样规范化;
  getByTypeAndHash 暂留 LOWER()(兼容历史库里可能的小写行)。
- tools/check-hash-case-conflicts.mjs:只读检测脚本,逐组列出只差大小写的同
  (UserId,Type) 行;零冲突才可把查询收敛为 Hash=?。绝不自动删数据。
- test/fixes.test.ts:写 'abc' 落库为大写,且再写 'ABC' 命中同一条(不双份)。
- test/hash.test.ts:normalizeProfileHash 单测。
- test/docs.test.ts:check-hash-case-conflicts 的判据与只读性。
- docs/protocol.md §10:更新「落库 hash 的大小写」「hash 的匹配方式」两行,
  去掉已漂移的行号、改用函数锚点。
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T15:01:06.855699Z b38f468 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b38f4689ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/db.ts Outdated
e.transferDataHash ?? '',
JSON.stringify(e.filePaths),
e.hash,
normalizeProfileHash(e.hash),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Migrate R2 directories before uppercasing stored hashes

When an existing legacy lowercase record is updated (for example, starred, restored, or reused), entityParams writes its Hash back in uppercase while its R2 data remains under the original lowercase working-directory key. Subsequent /data reads construct the uppercase key and fail, and listReferencedWorkingDirs then identifies only the uppercase directory, allowing the orphan-cleanup phase to delete the actual lowercase data. If an uppercase twin already exists, the same update instead fails the case-sensitive unique constraint. Preserve the stored hash on updates, or migrate the R2 directory and resolve case-fold conflicts before changing it.

Useful? React with 👍 / 👎.

Comment thread src/db.ts Outdated
`DELETE FROM HistoryRecords WHERE UserId = ?1 AND Type = ?2 AND Hash = ?3 AND IsDeleted != 0`,
)
.bind(HARD_CODED_USER_ID, type, hash)
.bind(HARD_CODED_USER_ID, type, normalizeProfileHash(hash))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep per-record purge compatible with lowercase legacy hashes

For an untouched legacy row whose stored hash is lowercase, the UI submits that lowercase value from the record, but this binding converts it to uppercase while the SQL comparison remains case-sensitive. The DELETE therefore affects zero rows and the UI reports the permanent deletion as failed. Use a case-insensitive match for this compatibility period, or delete the exact hash value supplied by the listed record.

Useful? React with 👍 / 👎.

评审反馈:原版把 normalizeProfileHash 塞进写边界,还改了 purgeDeletedRecord 的绑定,
但查询仍用 LOWER(Hash)=LOWER(?) ⇒ 历史小写行"能读、能进回收站、却删不掉"(明确 bug);
且检测脚本只查 case-fold 冲突,漏了"无冲突但含小写行"这一档,安全结论不成立。

按评审收敛为第一阶段:**只加诊断工具,不动 DB 行为**。
- 撤掉 src/db.ts、src/hash.ts 的全部改动(写边界/查询绑定恢复原样)。
- tools/check-hash-case-conflicts.mjs:两项体检 —— ① case-fold 冲突组 ② 非规范(非大写)行;
  两者都为 0 才判"可安全收敛为 Hash=?"。只读(只 SELECT)。
- test/docs.test.ts:两条判据各一组用例(含"无冲突但有小写行"的判别力)。
- docs/protocol.md:把 "hash 的匹配方式" 行的易漂移行号换成函数锚点,并注明由体检决定是否收敛。

实测:本地 D1 5818 行两项 0;注入一条小写行后工具如实报 ②=1 并退出 1(判别力已验证)。
@leeefun

leeefun commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

关闭,不合并(评审结论)。

项目自初始提交起,正常写路径一直是 hash.toUpperCase()(addProfile / addRecordDto),SHA256 输出本身也是大写 ⇒ 正常产品路径从不产生小写 Hash;且上游数据库不能直接导入当前 D1 schema。因此本 PR 防的实际上只有「人工改 D1 / 第三方脚本绕过应用层」这类带外写入——不值得为它长期增加约 150 行工具 + 测试 + 文档。

现有 LOWER(Hash) = LOWER(?) 保持不动即可;个人规模下为「完全吃到索引」而引入数据迁移/诊断体系不划算。将来真出现性能问题,再按真实数据处理。

(已被撤掉的运行时改动:本 PR 早先版本曾把 normalizeProfileHash 写进写边界并改了 purgeDeletedRecord 的绑定,但查询仍用 LOWER() ⇒ 历史小写行会「能读、能进回收站、却删不掉」。那部分已全部还原,本 PR 现在只剩只读诊断工具。)

@leeefun leeefun closed this Oct 6, 2026
@leeefun
leeefun deleted the refactor/hash-canonical-form branch October 7, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant