Skip to content

feat: AAA data integration, PADD regions, historical charts, and test coverage (v1.1.5-beta.4)#11

Merged
WFord26 merged 20 commits intomainfrom
beta
Mar 20, 2026
Merged

feat: AAA data integration, PADD regions, historical charts, and test coverage (v1.1.5-beta.4)#11
WFord26 merged 20 commits intomainfrom
beta

Conversation

@WFord26
Copy link
Owner

@WFord26 WFord26 commented Mar 20, 2026

Summary

This PR merges all beta work into main, advancing the version from 1.1.0 to 1.1.5.


What's changed

Test coverage

  • @fuelripple/db queries — 38 new tests; �aa.ts and supply.ts both 0% → 100%; overall package statement coverage 16.89% → 58.39%
  • @fuelripple/api integration — 16 new endpoint tests (AAA routes + Supply routes) + 9 missing DB mocks; total API test count 51 → 67; overall API statement coverage 37.5% → 53.59%

Historical fuel prices page overhaul

  • Full support for all 4 AAA fuel grades (Regular, Mid-Grade, Premium, Diesel)
  • Regional drill-down: National → PADD region → Individual state
  • Time range buttons: 7D, 30D, 90D, 1Y, 5Y, All-Time
  • Recharts LineChart with multi-series, tooltip, and legend
  • 7-day Moving Average overlay toggle
  • Enhanced statistics cards per grade (current price, 7d/30d change, min/max/avg)

Daily crude oil & gas correlation endpoints

  • \getDailyCrudePrices, \getDailyAaaGasPrices, \getDailyCorrelationSeries\ DB helpers
  • Three new API endpoints: /correlation/daily-crude, /correlation/daily-gas-aaa, /correlation/daily-series\
  • Correlation page updated with real-time Yahoo Finance WTI chart

AAA PADD regional aggregates

  • New \�aa_padd_aggregates\ hypertable — daily per-PADD mean and population-weighted averages for all 4 grades
  • \STATE_POPULATIONS\ constant (2020 Census) added to @fuelripple/shared\
  • Three new API endpoints: /aaa/regions, /aaa/region/:padd/latest, /aaa/region/:padd\
  • \�ackfill-aaa-padd\ script for historical backfill

Regional comparison page

  • All 4 fuel grades across all 5 PADD regions using daily AAA data
  • Dual-method selector (arithmetic mean vs. population-weighted mean)
  • US state choropleth map with PADD region borders
  • Collapsible per-PADD cards with per-state price breakdown

AAA data reliability

  • \�aaClientV2\ — parallel batch requests (5 concurrent), exponential backoff, multiple HTML parsing strategies
  • Optional intraday scraping at 4 PM ET (\AAA_INTRADAY_ENABLED=true)
  • \docs/AAA_DATA_INTEGRATION.md\ reference guide

Wayback Machine historical backfill

  • \�ackfill-aaa-wayback\ script recovers AAA state prices back to Nov 2017 from Internet Archive
  • CDX API enumeration, two HTML layout parsers, polite 1,200 ms delay, idempotent \onConflict().ignore()\

State detail page

  • 4-row current prices table (all grades vs. national average with % diff)
  • Full-width price history comparison table across all 4 grades × 5 time periods

Infrastructure

  • \latest_prices\ snapshot table — O(1) lookup for /prices/current\
  • \�aa_national_averages\ hypertable + \�aa_state_aggregates\ table
  • \price_changes_cache\ table pre-computes week/month/3-month/year deltas
  • \CACHE_TTL.AAA_NATIONAL\ (24 h) constant in @fuelripple/shared\

Bug fixes

  • Downstream Sankey pass-through modeling corrected (15% CPI / 85% absorbed)
  • Stale \T06:00:00Z\ duplicate rows fixed via \Date.UTC()\ normalization
  • Redis not cleared in script context fixed by calling \initializeCache()\ first

Migrations required

\
npm run db:migrate
\
New tables: \�aa_national_averages, \�aa_state_aggregates, \latest_prices, \price_changes_cache, \�aa_padd_aggregates, \�aa_state_changes_cache\

Test results

\
Tasks: 8 successful, 8 total — all packages pass
\\

WFord26 added 20 commits March 16, 2026 13:06
- Created a new migration to establish the aaa_national_averages table for storing daily US nationwide average gas prices.
- Implemented upsert and retrieval functions for national averages in the aaa queries module.
- Updated the main index to export the new aaa queries.
- Added a new BlogPostMeta schema to shared schemas for blog post metadata management.
…ding backfill script and caching improvements
- Added `__APP_VERSION__` declaration in `vite-env.d.ts` for version tracking.
- Updated `vite.config.ts` to read version from `package.json` and included `remark-gfm` for GitHub Flavored Markdown support.
- Created a new migration for `aaa_state_changes_cache` to store pre-computed price changes for states and fuel grades.
- Added seed data for `aaa_state_averages` to populate historical price data for testing.
- Implemented new queries in `aaa.ts` for fetching the latest state prices and historical data.
- Added new PADD aggregates to the job queue for daily and intraday updates.
- Implemented new API endpoints to fetch PADD aggregates and individual region data.
- Enhanced the Comparison and Dashboard pages to display PADD regional prices.
- Introduced a new database migration for the `aaa_padd_aggregates` table.
- Updated data processing logic to compute population-weighted averages for PADD regions.
- Improved error handling and logging for the AAA data scraping process.
- Added documentation for the AAA data integration and its current status.
- Updated Historical.tsx to replace EIA data with AAA national averages for gas prices.
- Introduced new fuel grade options (regular, mid-grade, premium, diesel) and adjusted time ranges.
- Implemented moving average calculations and statistics for selected fuel grades.
- Enhanced charting with Recharts for better visualization of fuel price trends.
- Added new API functions in prices.ts for fetching daily gas prices and correlation series using AAA data.
- Improved UI components for better user experience and clarity in data representation.
@WFord26 WFord26 merged commit 99a0f51 into main Mar 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant