Repository navigation
feat: read v1 and v2 SnapshotDB callstack layouts without migrating - #135
Conversation
codeasier
left a comment
There was a problem hiding this comment.
Cross-review report (3 independent reviewers: qwen3.8-flash, deepseek-v4.1-flash, muse-spark-1.3-contributor; quorum 2/3, all findings re-verified by the judging session against head d518ef4).
Verdict: approve with fixes. The v1/v2 read-compat mechanism is correct and well tested: read-only detection inside mode=ro, injection-safe identifier handling, byte-equivalent v2 SQL, per-layout compiled-template caching, and consistent CLI/API/MCP plumbing. No security issues found.
Inline findings (severity order):
- Medium — construction-time layout rejection breaks all surfaces for conflicting external DBs, and the focus display path swallows the error silently (context.py, focus_service.py, database.md).
- Low —
_has_tablecase-sensitivity vs SQLite's case-insensitive identifiers (context.py). - Low — metadata-vs-structure conflict skipped when the layout is unrecognized (context.py).
- Low —
focus --device/--sessiondon't print the callstack layout the docs promise (cli.py). - Low — stale "re-import the snapshot" advice now only fires for custom templates (executor.py).
- Low — explicit
querysilently discarded whenquery_variantsis present (config.py).
Test gaps (informational): the v1 trailing-LIMIT pushdown path is untested in tests/query/test_query_max_rows_pushdown.py (only v2 DBs are built); test_unknown_layout_errors_only_for_variant_templates doesn't assert that a non-variant template still succeeds on an unknown-layout DB; no CLI/MCP test covers focus display on a conflicting-layout DB.
Legacy inline-callstack databases became unusable for callstack templates after schema v2. Detect the layout read-only and select matching SQL so focus and query keep working on both schemas. Fixes #134
Construction-time rejection blocked focus, metadata, and every template for conflicting external databases. Detect the conflict, surface it on focus, and fail only variant templates.
d518ef4 to
0ddd7eb
Compare
codeasier
left a comment
There was a problem hiding this comment.
Follow-up cross-review (same 3 independent reviewers: qwen3.8-flash, deepseek-v4.1-flash, muse-spark-1.3-contributor; quorum 2/3, findings re-verified by the judging session against head 0ddd7eb).
Verdict: all six findings from review 5237886082 are properly fixed — approve. Each fix has focused test evidence, the v2 SQL is preserved in the variants, non-variant templates keep working on unrecognized/conflicting layouts, and the read-only/no-migration guarantees hold. en/zh docs are consistent with the implemented behavior (including the intentional --session export-only output).
| Finding | Status | Key evidence |
|---|---|---|
| 1 (Medium) construction-time rejection + silent focus error | fixed | _validate_schema only checks dictionary; _detect_callstack_layout returns (None, message); get_focus propagates callstack_layout_error and keeps devices; CLI warns on all focus paths |
2 (Low) _has_table case-sensitivity |
fixed | _find_table with COLLATE NOCASE; _import_format_version resolves the real table name |
| 3 (Low) metadata check skipped when unrecognized | fixed | layout is None now conflicts with import_format_version 1/2 |
4 (Low) --device/--session missing layout echo |
fixed | --device echoes; --session intentionally export-only, documented en/zh |
| 5 (Low) stale re-import advice | fixed | layout-aware _execution_error, no re-import text; docs updated |
6 (Low) query silently overwritten |
fixed | from_dict rejects declaring both query and query_variants |
Three low-severity, non-blocking residuals are noted inline (dead _has_table helper, an inverted message branch in _execution_error, and an over-broad Warning: prefix match in the contract-test normalizer), plus one optional doc precision nit on callstack_layout_error in snapshot-analyzer-api.md (en/zh). None of these blocks merge.
| return str(row[0]) if row is not None else None | ||
|
|
||
|
|
||
| def _has_table(cursor: sqlite3.Cursor, name: str) -> bool: |
There was a problem hiding this comment.
[Low, non-blocking] Dead helper after the case-sensitivity fix
_has_table is now referenced nowhere in src/ or tests/ (whole-worktree grep matches only this definition) — all detection call sites use _find_table directly. Since it was the site of the original case-sensitivity finding, leaving it behind invites future reuse. Suggest deleting it. (The still case-sensitive name='dictionary' check in _validate_schema predates this PR and is out of scope.)
| "but this database uses the v1 inline-callstack layout" | ||
| ) | ||
| if layout == "v2": | ||
| return QueryExecutionError( |
There was a problem hiding this comment.
[Low, non-blocking] Inverted message in the v2 branch
When layout == "v2" and the SQLite error matches the callstack patterns, this claims the SQL "expects the v1 inline-callstack layout" — but the only plausible trigger is a custom template referencing a nonexistent column such as callstack.callstackId on a v2 database, where that wording is misleading (the SQL references a v2-style column in the wrong place, not a v1 expectation). Unreachable for packaged variant templates since selection guarantees matching SQL. Suggest falling back to the generic "requires a recognized v1 or v2 callstack layout" message for the non-v1 case.
| ] | ||
| elif line.startswith("Callstack layout: "): | ||
| focus["callstack_layout"] = line.removeprefix("Callstack layout: ").split(" ", 1)[0] | ||
| elif line.startswith("Warning: "): |
There was a problem hiding this comment.
[Low, non-blocking] Over-broad Warning: prefix match
Any Warning: … line is mapped to callstack_layout_error, but the same CLI focus branch can also print Warning: Database file does not exist! (cli.py:156), which would be misclassified as a layout error against MCP's None. No current test routes a missing-database focus through this normalizer, so it passes today — it will produce a confusing failure for the first contract test that does. Suggest matching only the layout-conflict prefix (e.g., Warning: Incompatible callstack layout).
📝 Description
Read both SnapshotDB callstack layouts without rewriting the file.
Contextdetects v1 inline text vs v2callstackIdfrom table columns, andevent/callstack_analysis/active_memory_callstack_at_eventselect matching SQL under the same template contract. Focus, query, API, and MCP stay read-only; layout is not stored in.pt-snap/focus.json, and there is no implicit v1→v2 migration.🔗 Related Issue
Fixes #134
🧪 Testing
Focused query/context/focus/report/CLI/API/MCP/contract tests plus full
pytest(807 passed, 1 xfailed).ruff checkpassed.✅ Checklist
Snapshot Runtime Provenance
Snapshot provenance decision: no-update
Snapshot provenance no-update reason: This change only adds read-only v1/v2 query compatibility in Context and query templates; snapshot import, replay, and PROVENANCE.md are unchanged.
📋 Additional Notes
v2 queries still aggregate by
callstackId. Import cache invalidation on format change is unchanged. Conflicting columns, mixed device layouts, or metadata that contradicts a recognized structure raise a schema error.