Skip to content

feat: read v1 and v2 SnapshotDB callstack layouts without migrating - #135

Merged
codeasier merged 2 commits into
mainfrom
issue-134-callstack-schema-compat
Sep 17, 2026
Merged

codeasier merged 2 commits into
mainfrom
issue-134-callstack-schema-compat

Conversation

@codeasier

Copy link
Copy Markdown
Owner

📝 Description

Read both SnapshotDB callstack layouts without rewriting the file. Context detects v1 inline text vs v2 callstackId from table columns, and event / callstack_analysis / active_memory_callstack_at_event select matching SQL under the same template contract. Focus, query, API, and MCP stay read-only; layout is not stored in .pt-snap/focus.json, and there is no implicit v1→v2 migration.

🔗 Related Issue

Fixes #134

🧪 Testing

  • Tests added/updated
  • Manual testing completed

Focused query/context/focus/report/CLI/API/MCP/contract tests plus full pytest (807 passed, 1 xfailed). ruff check passed.

✅ Checklist

  • Code follows project guidelines
  • Documentation updated (if applicable)
  • No new warnings introduced
  • Changes tested locally

Snapshot Runtime Provenance

  • Snapshot provenance evaluation completed when snapshot runtime files change
    Snapshot provenance decision: no-update
    Snapshot provenance no-update reason: This change only adds read-only v1/v2 query compatibility in Context and query templates; snapshot import, replay, and PROVENANCE.md are unchanged.

📋 Additional Notes

v2 queries still aggregate by callstackId. Import cache invalidation on format change is unchanged. Conflicting columns, mixed device layouts, or metadata that contradicts a recognized structure raise a schema error.

@codeasier codeasier left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-review report (3 independent reviewers: qwen3.8-flash, deepseek-v4.1-flash, muse-spark-1.3-contributor; quorum 2/3, all findings re-verified by the judging session against head d518ef4).

Verdict: approve with fixes. The v1/v2 read-compat mechanism is correct and well tested: read-only detection inside mode=ro, injection-safe identifier handling, byte-equivalent v2 SQL, per-layout compiled-template caching, and consistent CLI/API/MCP plumbing. No security issues found.

Inline findings (severity order):

  1. Medium — construction-time layout rejection breaks all surfaces for conflicting external DBs, and the focus display path swallows the error silently (context.py, focus_service.py, database.md).
  2. Low — _has_table case-sensitivity vs SQLite's case-insensitive identifiers (context.py).
  3. Low — metadata-vs-structure conflict skipped when the layout is unrecognized (context.py).
  4. Low — focus --device / --session don't print the callstack layout the docs promise (cli.py).
  5. Low — stale "re-import the snapshot" advice now only fires for custom templates (executor.py).
  6. Low — explicit query silently discarded when query_variants is present (config.py).

Test gaps (informational): the v1 trailing-LIMIT pushdown path is untested in tests/query/test_query_max_rows_pushdown.py (only v2 DBs are built); test_unknown_layout_errors_only_for_variant_templates doesn't assert that a non-variant template still succeeds on an unknown-layout DB; no CLI/MCP test covers focus display on a conflicting-layout DB.

Comment thread src/pt_snap_cli/context.py Resolved
Comment thread src/pt_snap_cli/core/focus_service.py Resolved
Comment thread docs/en/database.md Resolved
Comment thread src/pt_snap_cli/context.py Resolved
Comment thread src/pt_snap_cli/context.py Resolved
Comment thread src/pt_snap_cli/cli.py Resolved
Comment thread src/pt_snap_cli/query/executor.py Resolved
Comment thread src/pt_snap_cli/query/config.py Resolved
codeasier added 2 commits September 17, 2026 23:35
Legacy inline-callstack databases became unusable for callstack templates after schema v2. Detect the layout read-only and select matching SQL so focus and query keep working on both schemas.

Fixes #134
Construction-time rejection blocked focus, metadata, and every template for
conflicting external databases. Detect the conflict, surface it on focus, and
fail only variant templates.
@codeasier
codeasier force-pushed the issue-134-callstack-schema-compat branch from d518ef4 to 0ddd7eb Compare September 17, 2026 15:36

@codeasier codeasier left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up cross-review (same 3 independent reviewers: qwen3.8-flash, deepseek-v4.1-flash, muse-spark-1.3-contributor; quorum 2/3, findings re-verified by the judging session against head 0ddd7eb).

Verdict: all six findings from review 5237886082 are properly fixed — approve. Each fix has focused test evidence, the v2 SQL is preserved in the variants, non-variant templates keep working on unrecognized/conflicting layouts, and the read-only/no-migration guarantees hold. en/zh docs are consistent with the implemented behavior (including the intentional --session export-only output).

Finding Status Key evidence
1 (Medium) construction-time rejection + silent focus error fixed _validate_schema only checks dictionary; _detect_callstack_layout returns (None, message); get_focus propagates callstack_layout_error and keeps devices; CLI warns on all focus paths
2 (Low) _has_table case-sensitivity fixed _find_table with COLLATE NOCASE; _import_format_version resolves the real table name
3 (Low) metadata check skipped when unrecognized fixed layout is None now conflicts with import_format_version 1/2
4 (Low) --device/--session missing layout echo fixed --device echoes; --session intentionally export-only, documented en/zh
5 (Low) stale re-import advice fixed layout-aware _execution_error, no re-import text; docs updated
6 (Low) query silently overwritten fixed from_dict rejects declaring both query and query_variants

Three low-severity, non-blocking residuals are noted inline (dead _has_table helper, an inverted message branch in _execution_error, and an over-broad Warning: prefix match in the contract-test normalizer), plus one optional doc precision nit on callstack_layout_error in snapshot-analyzer-api.md (en/zh). None of these blocks merge.

return str(row[0]) if row is not None else None


def _has_table(cursor: sqlite3.Cursor, name: str) -> bool:

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low, non-blocking] Dead helper after the case-sensitivity fix

_has_table is now referenced nowhere in src/ or tests/ (whole-worktree grep matches only this definition) — all detection call sites use _find_table directly. Since it was the site of the original case-sensitivity finding, leaving it behind invites future reuse. Suggest deleting it. (The still case-sensitive name='dictionary' check in _validate_schema predates this PR and is out of scope.)

"but this database uses the v1 inline-callstack layout"
)
if layout == "v2":
return QueryExecutionError(

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low, non-blocking] Inverted message in the v2 branch

When layout == "v2" and the SQLite error matches the callstack patterns, this claims the SQL "expects the v1 inline-callstack layout" — but the only plausible trigger is a custom template referencing a nonexistent column such as callstack.callstackId on a v2 database, where that wording is misleading (the SQL references a v2-style column in the wrong place, not a v1 expectation). Unreachable for packaged variant templates since selection guarantees matching SQL. Suggest falling back to the generic "requires a recognized v1 or v2 callstack layout" message for the non-v1 case.

]
elif line.startswith("Callstack layout: "):
focus["callstack_layout"] = line.removeprefix("Callstack layout: ").split(" ", 1)[0]
elif line.startswith("Warning: "):

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low, non-blocking] Over-broad Warning: prefix match

Any Warning: … line is mapped to callstack_layout_error, but the same CLI focus branch can also print Warning: Database file does not exist! (cli.py:156), which would be misclassified as a layout error against MCP's None. No current test routes a missing-database focus through this normalizer, so it passes today — it will produce a confusing failure for the first contract test that does. Suggest matching only the layout-conflict prefix (e.g., Warning: Incompatible callstack layout).

@codeasier
codeasier merged commit 53f2196 into main Sep 17, 2026
6 checks passed
@codeasier codeasier mentioned this pull request Sep 17, 2026
6 of 7 tasks
@codeasier
codeasier deleted the issue-134-callstack-schema-compat branch October 3, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] 兼容 callstack 去重前后的 DB schema:v1 旧库直接只读查询/focus,不迁移

1 participant