Repository navigation
feat(data-compare): compare 2,000 rows of both sides in key order, paged and filtered - #559
Merged
Merged
Conversation
…ged and filtered The grids page each side on its own with OFFSET, so their pages never line up and a one-sided row on a cut-short side could only be 'unresolved'. Compare 2,000 rows reads both sides again ordered by the key columns (the order goes on the outer query: /sql/execute takes orderBy, plain column names only), 2,001 rows each to learn whether more follow. - A row found on one side only is classified when the other side holds everything, or (numeric keys) when its key sorts at or before that side's last key. Text keys stay unresolved past a cut: engines sort them by their own collation. - Filter chips with counts for the whole window (Changed, Only in source, Only in target, Unresolved, Matching), browser-side pages of 50-500, and Next / Previous difference across pages. Sync boxes keep their state. - Data migrate reads the compared window; it still writes only when both sides are complete (raising its cap is Phase B). Phase A of docs/plans/2026-10-17-data-compare-and-sql-tracer.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 12815453-008a-472c-aece-938027be9e0e) |
Contributor
Test IDsAdded (12)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase A of
docs/plans/2026-10-17-data-compare-and-sql-tracer.md(Part 1).Problem
Compare data only saw the page each grid had loaded. Each side pages on its own with OFFSET, so the pages never line up. Every one-sided row on a cut-short side came out
unresolved, and paging through a compare wasn't meaningful.Change
/sql/executetakesorderBy(plain column names only, at most 8) and puts the order on the outer query, in each engine's paging syntax.Tests
sql-page-wrap.service.test.ts: the ordered page on Postgres, MySQL, Oracle, Db2 and SQL Server, including the ORDER BY strip;orderByvalidation refuses anything but plain names.sql-execute.service.test.ts: live SQLite. A page comes back in key order withhasNext. It fails without the change, because the statement's ownORDER BY name DESCwould win.resultKeyAlign.test.ts: the window boundary, numeric including bigint text, and text staying unresolved. The boundary cases fail on the old code.compareView.test.ts: filter counts, paging, next/previous difference, the slice, and the last-row key.sql-editor-compare-window.test.ts, 4/4: two SQLite files of 2,500 rows with known differences. Exact counts (Changed 3, Only in source 2, Unresolved 2, Matching 1,995, All 2,002), the changed filter, 21 pages of 100, and difference jumps to pages 15 and 20.eslint .and the security lint are clean. Unit suite: 6559 pass.Next (Phase B)
Raise migrate's cap to 2,000 through one shared constant, and take the restore snapshot in chunks of 500 keys (Oracle's IN-list limit).
🤖 Generated with Claude Code
Note
Medium Risk
Touches SQL execution paging (
orderBywrapping) across dialects and compare/migrate data paths; mitigated by strict column-name validation and broad tests, but wrong paging SQL would affect any ordered execute calls.Overview
Adds Compare 2,000 rows so side-by-side compare no longer depends on mismatched grid pages: both connections re-run the statement with
orderBykey columns (up to 2,000+1 rows each) via extendedPOST /sql/executeand dialect-specificwrapSqlForOrderedPage(validated plain column names; SQL Server strips inner ORDER BY when needed).The compare toolbar gains filter chips with full-window counts (changed, source-only, target-only, unresolved, matching), in-browser paging (50–500 rows), and previous/next difference jumps.
useCompareWindowholds fetched windows;compareViewslices/filters aligned rows. Key alignment usesleftWindowEnd/rightWindowEndso numeric keys classify one-sided rows before a truncated side’s last key; text keys stay unresolved past a cut. Data migrate reads the compared window when active. Redis/Mongo subset dialects skip the action.Coverage: unit tests for ordered paging, alignment boundaries, and compare view; new e2e
sql-editor-compare-window.test.tswith seeded SQLite fixtures.Reviewed by Cursor Bugbot for commit e70bbfe. Bugbot is set up for automated code reviews on this repo. Configure here.