Repository navigation
rate-lab: Rate Lab module (M5) - #157
Merged
Merged
Conversation
quoteParcel, laneProviders and checkAddress move to carriers/quote.ts so Rate Lab can quote and check addresses the way rates.fetch does. The order fetch now calls them; behaviour is unchanged. Refs #145
- Rate Lab role (ratelab.use + ratelab.manage), assignable on Users, en/es labels. - The module's one migration: every Rate Lab table and enum, the role, the run-cap setting, FKs from rate_fetches' lab columns, and the hand-written constraints (one active suite run, group names unique per kind ignoring case, ad hoc fields on one-off runs only), each proven by a DB test. - core/rate-lab/catalog.ts: addresses (create/update/recheck/duplicate/archive, lock rules) through the shared checkAddress, and groups (create/rename/archive). - Rate Lab sidebar entry and tabbed shell; the Addresses tab with group filter, search, archived toggle, typed vs. corrected, and a groups dialog. Refs #146
- core/rate-lab/parcels.ts: list, create, update, duplicate, archive and unarchive test parcels, plus a physical-product lookup for contents. A box copies its inner size and keeps the link; the parcel's own box keeps the size it copied when the box is edited later. Locked parcels refuse size, weight and contents changes (unchanged values pass); title, description and groups still change. Contents: physical products only, 1-999 each, each product once. Parcel groups only. Every change audited. - catalog.ts shares resolveGroups (now by kind), groupRefs and copyTitle. - Parcels tab: table with group filter, search, archived toggle and lock icon; add/edit dialog with box or typed size, weight, contents picker and groups. The groups dialog takes a kind. - en/es strings; desktop e2e for a parcel from a box with contents and one with a typed size, in a group. Refs #147
Admins (report.read) see, for active labels linked to their logged quote (default last 30 days, fake data hidden unless asked): cheapest-pick share and overspend by reason, carrier and service share, median/mean lane cost per week, label cost vs. shipping charged, and each package's alternatives. Filters: dates, carrier, service, country, state, box. Adds Recharts 3.10.1 (exact, React 19 peer) with react-is, a theme hook that reads daisyUI colours, and ChartFigure, which pairs each chart with a table for screen readers. Refs #152
# Conflicts: # packages/core/src/index.ts # packages/i18n/src/messages/en.json # packages/i18n/src/messages/es.json
Rate Lab quotes are never bought, and ShipStation v2's POST /v2/rates with an inline shipment creates a pending shipment per call. RateRequest.quick maps to rate_options.rate_type "quick". Answers SPEC-rate-lab open question 1. Refs #148
New run dialog with the live combination count and cap, startLabRun / previewLabRun / listLabRuns, and the rateLab.run job (new pg-boss queue, retryLimit 0): three combinations at a time through the shared quoting, each logged as a LAB fetch, customs from contents, addresses and parcels locked. The runs list polls every 3 s while a run is active. Refs #148
# Conflicts: # apps/web/src/server/router.ts # packages/contracts/src/rate-lab.ts # packages/core/src/index.ts # packages/i18n/src/messages/en.json # packages/i18n/src/messages/es.json
The run page shows its header, insight cards (cheapest and fastest per address x parcel, price moves against the most recent earlier suite run, carrier wins, failures per provider), every quote grouped by combination with carrier/provider/status filters, and Run again. Refs #150
- cancelLabRun (POST /rate-lab/runs/{id}/cancel, ratelab.use, audited): a queued
run is cancelled at once and its job quotes nothing; a running one stops between
combinations and ends CANCELLED, keeping what it logged.
- recoverStalledLabRuns: startLabRun and the runs list fail any RUNNING run whose
heartbeat (or start) is over 10 minutes old with errors.rateLab.stalled. The job's
end writes only while the run is still RUNNING, so a late worker can't undo that.
- Settings: rateLabMaxCombinations (1-2000, admins) and a Rate Lab card on
Settings -> Shipping; the New run dialog re-reads its count on open.
- Runs list: Cancel on active runs, "stopping" once asked (LabRunRow.cancelRequested).
- Dialog: scrolls when taller than the screen, so its buttons stay reachable.
Refs #149
# Conflicts: # apps/web/src/server/router.ts # packages/contracts/src/index.ts # packages/contracts/src/rate-lab.ts # packages/i18n/src/messages/en.json # packages/i18n/src/messages/es.json
# Conflicts: # apps/web/src/server/router.ts # packages/contracts/src/rate-lab.ts # packages/i18n/src/messages/en.json # packages/i18n/src/messages/es.json
A typed address against 1-20 saved, grouped or typed parcels, quoted inline as a ONE_OFF run (quick quotes, never bought), with Save as test address and Save parcel. The PII purge clears one-off typed addresses past the retention period and keeps their fetches. Suite runs and one-offs share labQuoter (rate-lab/combination.ts). Refs #151
Report pages in a (report) route group: no app nav, light theme, at most 180 mm wide, page breaks between sections. "PDF report" on each screen opens the page with print=1, which calls window.print() once the data is in. Core adds labRunTrends (each combination's cheapest price over its last 10 suite runs up to the run) and labTrendsReport (the grid plus up to 60 combinations' series, counting the rest). Charts skip animation on reports. Refs #155
Export CSV on a run's page and on Real orders streams one row per quote (RFC 4180, UTF-8 with BOM, CSV-injection guard) from session-checked routes that need ratelab.use and report.read. Refs #154
# Conflicts: # packages/core/src/index.ts
# Conflicts: # apps/web/src/app/(app)/rate-lab/real-orders/real-orders-dashboard.tsx # apps/web/src/app/(app)/rate-lab/runs/[id]/run-detail.tsx # packages/i18n/src/messages/en.json # packages/i18n/src/messages/es.json
…had it Price moves now look up, per address x parcel x service, the latest earlier SUITE quote with the same fake flag, so a partial run no longer makes services read NEW next time. MISSING stays measured on the last run with quotes. A move from a free price is CHANGED with no percentage instead of NaN.
Label cost vs. charged summed only the labels inside the date window and filters against the order's whole shipping charge. The filters still pick which orders are listed; each one is now costed with all its active package labels.
(status, purchased_at) replaces the (status) index it starts with.
labTrendsReport (and the run report's trends) asked for every address with every parcel; they now pass the exact pairs, as run detail does with unnest, within the grid's date filters. Shared groupBy replaces the copy-on-append grouping.
…e ids over /api/v1; cap Real orders at 366 days
…mid-stream failures and maps every refusal
…on snapshot, lost queued runs, double submit) - rateLab.run queue expires after 4 h (sized for the 2000-combination ceiling at the provider timeout); createQueue now updates expiry on existing queues too - the worker passes the job's signal plus a shutdown signal; an aborted run starts no new combination and ends FAILED (errors.rateLab.runFailed), freeing the one-active slot - startLabRun snapshots the resolved address/parcel ids in selection.resolved; the job quotes only those, minus archived ones; the chosen selection stays as it was for the list and Run again - a run QUEUED past the job expiry + stall window is failed with errors.rateLab.notStarted - a P2002 on start replays the same key's stored result before answering runActive - tests for the unexpected-error and providers-factory failure paths
…y number When one combination throws, the other workers stop before their next one and the run is failed once those in flight finish, so nothing calls a provider or bumps the run afterwards. Typed parcels are titled #1, #2 (as the Quote tab numbers them) instead of the English "Parcel 1" that reached customsIncomplete params.
- Rate errors translate with their params (customsIncomplete named its products as a raw key). - Real orders: a new carrier or country resets its service or state, choices the range no longer offers drop, and the CSV link uses reportQuery. - New run: Start waits for the new selection's preview; a stale start error clears. - Pickers say None only once loaded; address and parcel searches keep their rows. - WeightInput takes name/isRequired so weightOz field errors show. - Chart data tables print; Trends keys runs by id with the time, and focuses the chart on a pick. - Shared AddressFields, ParcelSizeFields, useMessage (lib/errors), money (lib/money, with a currency) and formatAddress (lib/address); run detail shows each quote's currency. - Address and parcel forms no longer scroll inside the scrolling dialog.
…oms one-off Runs are picked by their group, and the journey deletes its run and job in finally.
… in core groupBy, byName, DAY_MS and lastQuoted live once in rate-lab/shared.ts, and the CSV export groups quotes with groupBy instead of copying on every append. renameLabGroup, duplicateLabAddress and duplicateLabParcel Zod-parse their input like their siblings.
…election test allows the snapshot
A partial unique index on started_by_id for running ONE_OFF runs decides it, so two at once can't race; the conflict maps to errors.rateLab.oneOffActive. The run is created before the address check so a refused quote asks no provider, and stalled runs are recovered first so a crashed request doesn't hold the slot.
…e abort status rule A run backdated past the stall limit mid-quote stays RUNNING through stall recovery once a combination logs. Cancel races with the job's claim and with itself, deterministic ones with the run reads slowed. Abort as the last combination quotes ends SUCCEEDED; cancel plus abort ends CANCELLED with no error.
The lock's transaction timed out at 120 s, counting the wait for the lock, so a long Rate Lab body could lose the lock mid-test. It now lives for 30 minutes: only a hung body frees it.
…the journey's runs Runs quote on a fake whose getRates waits for the test, so cancel and progress assertions hold whatever the timing. Runs come from their row link or page URL, not the newest queued run. The journey deletes the runs it created, not every recent one-off. A worker that claims the run first fails the test saying so.
…is kept Each pair's last earlier suite fetch is a LATERAL LIMIT 1 down the (address, parcel, fetched_at) index instead of a DISTINCT ON over every earlier quote; only services that fetch lacks walk further back, per pair and service. Two queries as before.
…eued-stall comment orderLabelCosts now follows includeFake like the rows it costs. LAB_QUEUED_STALL_MS: pg-boss expiry only applies to started jobs, so the reason is that the worker has been gone longer than any run could last.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the
rate-labmodule per SPEC-rate-lab (M5).Closes #140
What it adds
Rate Lab is a role (
ratelab.use,ratelab.manage; decision 13). With it you can:rateLab.runpg-boss queue runs them, with no retries, one suite run at a time, a cap set in Settings, cancel, and recovery of stuck runs;report.read);Tickets
Review and ship fixes are included: run lifecycle (4 h job expiry, abort on shutdown, selection snapshot, recovery of runs left queued), price comparisons per service with fake runs matched to fake runs, cost vs. charged over each whole order, GET inputs that work over
/api/v1, a 366-day cap on Real orders ranges, CSV hardening, and UI and e2e fixes.Migrations
20261103130000_rate_lab: the module's tables and its hand-written constraints.20261105163427_shipping_labels_status_purchased_at: replaces thestatusindex onshipping_labelswith(status, purchased_at).20261105190000_rate_lab_one_off_per_user: allows one running one-off per user (a partial unique index). It first fails any duplicate running one-offs left over.The worker updates the
rateLab.runqueue's expiry when it starts. Runningpnpm jobs:installdoes the same on an existing database.Dependencies
recharts3.10.1 (decision 9) and its peerreact-is19.3.1.pnpm audit --prodreports nothing new.Verification
Run locally on
d59882e, the last merge:pnpm lint,pnpm typecheckandmigrate:checkall pass.pnpm test: core 2873 passed, 4 failed. All 4 failures were timeouts inpacking/service.guarantees.test.tswhile the machine was under load, and that file passes 56/56 on its own. Contracts, web, worker, logger and i18n all pass.pnpm buildpasses.Commit after
d59882e:SPEC.mdmarksrate-labasdone.