Skip to content

feat: bound many-shard lifecycle queries with private scratch storage - #231

Merged
codeasier merged 3 commits into
mainfrom
fix/218-bounded-lifecycle-scratch
Oct 10, 2026
Merged

codeasier merged 3 commits into
mainfrom
fix/218-bounded-lifecycle-scratch

Conversation

@codeasier

Copy link
Copy Markdown
Owner

依赖与范围

Stacked on #230。Related to #218,不自动关闭其全部剩余范围。

超过实际 SQLite ATTACH 上限时,使用一次查询私有的磁盘 SQLite 窄表归约 refs 与 lifecycle state,源分片始终只读并逐个挂载。<=cap 的既有快路径保留。证明、生命周期归并及冲突检测在过滤/分页前完成,最终仅取回有界结果,保留 source hydration、scope/NULL/free/tie 等语义。

  • QueryBudget.max_scratch_db_bytes 限制派生 main 数据库页;0 禁用新路径。默认候选512MiB正在更大规模验证。
  • 此限制不是总临时磁盘或RSS硬上限;journal、SQLite排序/临时文件仍可能有额外开销。保留原 fetched/output 行数和序列化字节预算、同一deadline及取消/失败清理。
  • 私有临时目录/文件权限、源写入拒绝、SQLITE_FULL/I/O/timeout域错误和清理已独立验证。异常进程终止可能留下私有临时数据,不保证崩溃时清理。
  • 文档明确本次改变旧的 no-temporary-merge-database 约定,仅在超cap路径写必要派生数据,不复制完整trace/stack/frame。

当前验证

最终tree654b156bf7ed20e98ec7fd5c6ed6ef4e4d3fb82f已冻结。

  • 独立同树840例语义差分、270次源写拒绝及FULL/timeout/I/O清理通过;源hash保持。
  • Focused104 passed/2 intentional skips;Ruff/Black/diff通过;typecheck0errors,总1630warnings,含新增83项非阻断类型标注债,不声称全为既有。
  • 独立16片/160k生命周期:低重复160k观察与高重复1.36M观察均通过,派生DB约8.8MB,取回32行。测量不是通用延时/RSS保证。
  • 12×500k短生命周期6M事件:三个模板均通过,派生DB226820096B。更重的持续存活/39M观察场景仍待结果,最终全套coverage和安装wheel gates仍运行,不宣称完成。
  • 已安装最终wheel的跨12个owner小规模CLI/API及页限额/禁用fallback错误检查通过。

当前为草稿;合入前还需最终完整本地证据与exact-head CI。

@codeasier
codeasier changed the base branch from fix/219-content-verified-cache to main October 10, 2026 15:19

Copy link
Copy Markdown
Owner Author

为运行现有 CI,暂将本 PR 目标设为 main:仓库工作流仅匹配 pull_request.branches=[main],原 stacked base 不触发检查。#230 仍是实际父提交和审阅/合入依赖,当前页面会包含其改动;请先审阅/合入 #230 后复核本 PR 净差异。未修改 workflow 或绕过保护,本 PR 仍为草稿。

Copy link
Copy Markdown
Owner Author

最终冻结树654b156bf7ed20e98ec7fd5c6ed6ef4e4d3fb82f本地验收完成:全套含slow及本轮wheel验收2198 passed / 2 intentional skips(不支持slice leak的参数组合);branch coverage92.73%,critical92/90/85%全过,scratch模块98%。构建/审计、独立安装CLI/API、Ruff/Black和typecheck0errors通过;1630warnings中新增83项为已复核的非阻断标注债。

规模实测(同一512MiB派生main页限额、默认fetched预算,查询实验deadline300s):

  • 12×500k事件短生命周期,3M lifecycles:三个模板16.2–17.4s,派生DB226820096B;冷inspect另25.9s。
  • 6M持续存活生命周期、39M观察:block118.45s、leak114.29s(页5/精确total6M,各28 fetched行),lifetime105.70s(13行);派生DB351571968B,低于512MiB。
  • 重样本冷inspect另239.34s,整进程峰RSS约1.76GiB包含验证阶段,不是归约RSS或硬上限。300s为本次查询实验配置,不是默认端到端耗时保证。

840项同树独立差分、真实源写拒绝、磁盘页限额/FULL/I/O/timeout清理和源hash不变通过。512MiB只承诺派生main页限额,journal/sort和RSS仍另有开销;任意更大数据仍可正确预算失败。最终exact-head CI另行等待,PR保留草稿。

Copy link
Copy Markdown
Owner Author

精确最终head a5298ea 已通过完整CI(全部6 jobs):https://github.com/codeasier/pt-snap-cli/actions/runs/38063116328 。本地2198项与安装wheel/规模/独立复核证据如上一评论。PR仍为草稿,保留#230合入依赖,等待顺序合入决定。

Copy link
Copy Markdown
Owner Author

After #230 was squash-merged, reconciled ancestry using a fast-forward dual-parent commit f7b61c0. Its tree remains exactly the tested 654b156bf7ed20e98ec7fd5c6ed6ef4e4d3fb82f; no source content changed. The new main parent has exactly the former #230 head tree. GitHub now reports the PR mergeable.

Final local results supersede the earlier pending full-suite status: 2198 passed / 2 intentional skips, 92.73% coverage; the 6M-event scale checks and 840 independent semantic differential cases passed. Original exact-head CI 38063116328 passed all six jobs. Fresh exact-head CI for the ancestry-only commit is running: https://github.com/codeasier/pt-snap-cli/actions/runs/38065401174 . Merge will wait for it.

This is partial work for #218, which remains open. The scratch page cap is not a total disk/RSS cap, and crash cleanup is not guaranteed.

@codeasier
codeasier marked this pull request as ready for review October 10, 2026 16:02
@codeasier
codeasier merged commit c467aee into main Oct 10, 2026
6 checks passed

Copy link
Copy Markdown
Owner Author

Squash-merged as c467aee. The final main tree is verified as 654b156bf7ed20e98ec7fd5c6ed6ef4e4d3fb82f, exactly the frozen tested tree.

Before merge, fresh exact-head CI 38065401174 completed successfully with all six jobs passing. The preceding #230 main push CI 38065366192 also completed successfully with all six jobs passing. Final main push CI is being monitored. #218 remains open for its remaining scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant