Repository navigation
perf: skip Frame reconstruction during database import (#84) - #85
Merged
Merged
Conversation
added 3 commits
August 7, 2026 17:32
Database import now keeps raw trace/segment/block frame dictionaries instead of building a Frame object per frame reference, and formats callstacks with reversed iterator and list comprehension. Public snapshot replay and split keep eager Frame construction unchanged. Baseline benchmark on 131k/628k sanitized samples (same machine, before on dafaf4f, after on this change): | sample | metric | before | after | change | |--------|----------|--------|-------|--------| | 131k | wall | 9.600s | 3.510s | -63% | | 131k | peak RSS | 1415.6MB | 233.0MB | -84% | | 628k | wall | 43.000s | 17.930s | -58% | | 628k | peak RSS | 5615.8MB | 706.5MB | -87% | DB output is byte-for-byte identical (same row counts, callstacks, representative queries, quick_check ok).
Owner
Author
|
已处理本轮评审意见:
本地验证: |
This was referenced Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
针对高事件数、深调用栈场景优化数据库导入的对象重建与调用栈处理成本(issue #84)。
核心改动
_raw_frames),不再为每个事件构建Frame对象;snapshot2db.dump()通过内部_raw_frames=True显式选用该模式。DeviceSnapshot.from_dict()/TraceEntry.from_dict()/ replay / split 仍保持 eager 构造,frames契约与 mutation 语义不变。filename/line/name,保持畸形输入原有失败时机。reversed()+ list comprehension,消除frames[::-1]。snapshot2db.dump()使用同目录临时数据库并在成功后原子替换;失败不会暴露多设备部分结果,也不会破坏已有目标数据库。benchmarks/baseline_import.py:新增 frame 引用总数与平均/最大调用栈深度;使用 Pythonperf_counter()+resource.getrusage()采集 wall/user/sys/RSS,支持 Linux 与 macOS(含 Darwin RSS 单位归一化),不再依赖 GNU/BSD/usr/bin/time。🔗 Related Issue
Fixes #84
🧪 Testing
全量测试:
585 passed, 1 xfailed基线测试(同一台机器、同一版 Python 标准库测量脚本,优化前 = base commit,优化后 = 本 PR)
正确性:两样本
PRAGMA quick_check = ok,trace/block 行数、memory_peak / allocator_gap / block_summary 代表查询结果优化前后完全一致。✅ Checklist
Snapshot Runtime Provenance
Snapshot provenance decision: updated
Snapshot provenance no-update reason:
📋 Additional Notes
131k/628k LFS 样本已在本机拉取并用于同机优化前后实测。benchmark 的资源输出缺失或格式错误会显式失败,不再静默记录为 0。