Skip to content
25 changes: 24 additions & 1 deletion docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Context LRU. One diminishing deadline includes all shards, sources, grouping and
totals; reports explicitly share **one** budget across peak selection and attribution. Cumulative
fetched/output work is limited to **100000 rows / 64 MiB serialized values**, including
repeated source reads. Exceeding either fails the entire operation, never an alleged
complete partial result. Materialization/sort are bounded by this work ceiling, **not**
complete partial result. Python materialization/sort are bounded by this work ceiling, **not**
by max_rows. This is not a process RSS limit: Python/SQLite, one batch and individual
cells have overhead. Strong manifest/member validation and hashing remain the default
and share that deadline: hash chunks, Python row batches and SQLite progress handlers
Expand All @@ -161,6 +161,29 @@ only when the caller guarantees immutable finalized files and reliable filesyste
change times; see [dataset validation and cache contracts](sharded-snapshotdb.md).
No temporary merge DB, repair or artifact write.

Global peak queries fetch scalar range evidence and at most three counter rows per
shard. Event/allocation pages apply SQL filters and ordering per shard, merge at most
`offset + limit` candidates per shard, and count matches separately. Range evidence
is measured before non-range filters. Ordinary text-stack statistics merge full
per-shard groups before global thresholds; averages use summed sizes and counts.
Canonical ordered-frame statistics retain the bounded source-reading path because
formatted text is not their identity. Block/leak/lifetime reductions use read-only
attached shards for proof, conflicts and latest observations before pagination;
the attachment limit (normally ten) counts selected observation shards plus any
other shards owning their allocation/free references. Slice queries discover those
proof owners before filtering or paging; exceeding the limit retains the budgeted
source-reading path. SQLite internal scans and temporary space are not bounded by
the fetched-row/byte budget. Summary and page (or lifetime buckets) each evaluate
the full lifecycle CTE, sharing the same deadline. No intermediate merged database
is created.

A million-event dataset can therefore answer aggregates and small pages within the
default work budget. Use `-n 5` or an explicit finite `limit` for large row-returning
queries. Unlimited output, large offsets, many distinct stack groups, ordered-frame
reads or the attachment-limit fallback can still exceed the budget and fail honestly.
The limit counts rows fetched into Python and serialized output, not internal SQL
scans, SQLite sorting space or total process memory.

## Dataset point-event attribution

A complete compatibility-v1 or native-v2 directory/manifest supports `event`
Expand Down
17 changes: 16 additions & 1 deletion docs/zh/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,13 +130,28 @@ exactness 标记诚实。`has_more/truncated` 描述窗口,不表示来源/范
数据集读取先分离批次再查外片来源,使用借用的有界 Context LRU。所有片、来源、分组、
精确总数共用递减 deadline;报告的峰值选择与归因明确共用**一个**预算。累计取回/输出工作上限为
**100000 行 / 64 MiB 序列化值**,含重复来源读取;超限整体失败,不返回冒充完整的部分
结论。物化/排序由工作上限约束,**不由 max_rows 约束**;不是进程 RSS 上限,Python/
结论。Python 物化/排序由工作上限约束,**不由 max_rows 约束**;不是进程 RSS 上限,Python/
SQLite、一个批次及单个 cell 有额外开销。默认仍对 manifest/成员执行完整校验和 hash,
并共用该 deadline:hash 分块、Python 行批次及 SQLite progress handler 都检查取消。
单个文件系统调用不提供 OS 级硬抢占。仅当调用方保证最终产物不可变且文件系统变更时间
可靠时,才可通过 `PT_SNAP_DATASET_CACHE=immutable` 显式启用进程内校验复用;参见
[数据集校验与缓存合同](sharded-snapshotdb.md)。不生成合并临时 DB、不修复或写产物。

全局峰值查询每片只取范围汇总及最多三条计数器记录。event/allocation 在各片 SQL 中筛选、
排序,每片最多取 `offset + limit` 个候选后全局归并,并单独统计匹配总数;实际范围证据在
非范围筛选前计算。普通文本栈统计先合并各片完整分组,再应用全局阈值,平均值由总大小
与总计数计算。规范化原始有序帧的身份不能用格式化文本替代,因此仍走受预算约束的来源
读取路径。block/leak/lifetime 通过只读附加分片先证明身份、检测冲突、选最新观察,再分页;
附加上限(通常十片)按选定观察分片与其分配/释放引用所属的其他分片合计。
slice 查询在筛选、分页前发现所需的证明来源分片;超限时回退到原有受预算约束的来源
读取路径。SQLite 内部扫描与临时空间不受取回行数/字节预算约束;汇总和分页(或存活时长
分桶)各自执行完整的生命周期 CTE,共用同一 deadline。不创建中间合并数据库。

因此,百万事件数据集的聚合与小页面可在默认工作预算内完成。大量逐行结果请指定 `-n 5`
或有限 `limit`。无限制输出、大 offset、大量不同栈分组、有序帧读取及超过附加上限的回退
路径仍可能超预算,并明确失败。预算统计进入 Python 的实际取回行和序列化输出,不限制
SQL 内部扫描、SQLite 排序空间或整个进程内存。

## 数据集定点事件归因

完整兼容 v1 或原生 v2 目录/manifest 支持 `event` 寻址,以及接受真实 `event_id` 的
Expand Down
Loading
Loading