Skip to content

Read the extension's schema and page the dashboard instead of capping it - #2

Open
EnesYilmazcode wants to merge 28 commits into
rebuild/p0-sitefrom
rebuild/p4-dashboard
Open

EnesYilmazcode wants to merge 28 commits into
rebuild/p0-sitefrom
rebuild/p4-dashboard

Conversation

@EnesYilmazcode

@EnesYilmazcode EnesYilmazcode commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Stack, merge in order: #1 > #2 (this one). Based on #1, so the diff here is only this step.
The extension stack: EnesYilmazcode/AmazonSellerScraper#5, EnesYilmazcode/AmazonSellerScraper#6, EnesYilmazcode/AmazonSellerScraper#7, EnesYilmazcode/AmazonSellerScraper#8, EnesYilmazcode/AmazonSellerScraper#9.

The dashboard read fields from a seed fixture the extension never wrote (F-25, F-29f). This vendors the extension's packages/schema and types runs and pages from it. Every document the dashboard reads, and every write in the rules tests, is checked against it. check-schema fails if the copy drifts from the extension.

  • Pagination (F-45): one live listener covers every loaded page, so a sync can't drop rows between pages. The count is recounted as the sync lands and never reads below what is loaded. The run inbox pages instead of stopping at 30. Export walks the whole scope a page at a time.
  • Movers (F-46): deltas and movers are computed per source from its latest two runs, page by page. The composite indexes no query used are dropped.
  • Accounts (F-50, F-55 partly): Google sign-ins can set a password for the extension. New passwords need 8 characters.
  • An empty cache-only snapshot keeps loading instead of saying "No scans yet" (F-48).
  • npm run seed writes through the extension's own sync module, and an e2e run checks the production bundle against that data.

The last commit is a small fix to the schema check. With core.autocrlf the extension's copy checks out with CRLF, so the byte comparison failed on Windows even though the committed files are identical.

Tested: npm test passes on Windows. That covers typecheck, the schema check, 7 unit tests, build, the bundle check, and the rules tests, including 43 adversarial probes. The e2e passes locally against the extension's rebuild/p4-sync, with one known failure marked (NEW-SYNC-1). CI checks out the extension's main branch, so I expect the schema check there to fail until the extension stack is merged.

Does not change permissions. This repo has no manifest.json.

…with the extension's own module

Each document lists the keys it may carry and their types. ASIN ids must match the ASIN pattern, run ids must be {sourceId}_{startMs}, scan times may not be in the future, firstSeenAt and firstRunId are create-only, events and stage moves are stamped with request.time, and owners can delete their own documents. The sync rules test now runs the extension's engine and sync.js from its checkout instead of a hand-copied port.
The first 200 products are live and Load more pages on from the last loaded document, so the header never presents a window as the whole workspace (F-45). An exact ASIN in the search box is read directly.
Movers was the global top 100 by delta.pPct filtered on the client, so a source with real drops showed none (F-46). It now diffs each source's latest two runs page chunk by page chunk. A source-scoped board shows deltas against that source's previous run, on screen and in the export.
The extension only signs in with email and password, so a dashboard account made with Google could never sync (F-50). A card above every view links an email credential to the same account, confirming the Google session first when Firebase asks for a recent sign-in.
…s taken

The Auth emulator checks every account for the email, including the one being linked, and refuses. With one account per email that holder can only be this account, so updatePassword adds the same email provider.
…on bundle is checked

The real engine and sync.js write 1,050 products over three runs on different days. vite preview serves the emulator-flagged production bundle, and Playwright checks the promised columns, true counts, Load more, exact ASIN search, the full export, run-against-run movers, and a Google account setting a password the extension then signs in with. Amazon hosts are answered locally.
…nges

Load more now grows the listener instead of freezing a cursor page, so
rows a sync pushes past a page boundary stay on screen. The header count
recounts after live changes, never drops below what is loaded, and
leaves out documents that failed the schema check.
PROSCAN_ALLOW_NO_EXT=1 used to exit 0 from inside requireExtension,
after the index.d.ts check had already failed.
… fails

vite preview of the dashboard alone 404s /icon128.png, which failed the
console check. A failed chromium launch left vite running, and a later
run then reused it; preview now refuses a port that already answers.
The extension's sync now needs it to keep every source a product was
seen in, and that fix also clears the NEW-SYNC-1 known failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant