Repository navigation
Warm the newest weekly report first, and build a cold page once - #137
Merged
Merged
Conversation
After a restart on the VPS, /weekly/2026-09-22 answered 504 at nginx's 60s and then 200 in 54s. Profiled locally with the real stores: a cold render is ~3.8s, essentially all of it get_symbols_data's first per-market build (append_trading_signals). The explicit-date matrix does share that cache with the boards' newest-week build (the second costs ~0.1s), so the note blaming a missing cache was wrong. The VPS journal shows what happened instead: the home-page prewarm, the crowd warmer and every piled-up request built the same frames in parallel on a ~9x slower box, and the two weekly requests finished the moment the home prewarm did, 3.5 minutes after boot. - report_page renders under a lock: lru_cache does not coalesce concurrent misses, so waiters now read the page the first render cached. - weekly_reports.warm_newest, called first in warm_page_caches (boot and after a release, still after the email per #136). It costs the crowd warmer nothing, both ride the same per-market cache. - The poller also warms after a send on a tick that saw no new week, since a navbar poll can consume refresh_if_stale's True and leave the page the email links to cold. Locally, warmer plus four concurrent requests for the cold newest page: one matrix build, all four served at 4.9s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 25, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Right after a restart on the VPS,
/weekly/2026-09-22returned 504 at 60.4s, then 200 in 54.2s, andverify-deploy.shsaw both weekly checks as 000.What the profile found
Local cold render of
report_page(newest)with the real stores (47-market params): 3.8s, of which 3.4s isget_symbols_data(mostlyappend_trading_signals, the trendline-breakpolyfitrolling apply the largest single piece at ~1.1s).The existing note blamed a missing cache between the explicit-date matrix and the boards' None-date one. Measurement does not support that. Both go through
get_symbols_data's lru_cache, so building one after the other costs ~0.1s.VPS journal for the 16:17:52 restart: the home-page prewarm (
pages/home.py::_prewarm_cache) ran 16:18:49 to 16:22:11, and nothing was served from 16:20:10 to 16:22:10. The two queued weekly requests completed at 16:22:10, the moment the prewarm finished. The warmer's heatmap and divergence steps ran ~8-9x slower than locally. So the 54s came from the same per-market frames being built in parallel by the home prewarm, the crowd warmer and each piled-up request (lru_cache does not coalesce concurrent misses) on a slower, GIL-bound box.Change
report_pagerenders under a lock, so concurrent requests for a cold page build it once and the rest read the cache.weekly_reports.warm_newest()now runs first inwarm_page_caches(boot, and after a release). It still runs after the email, per Send the weekly email before warming the page caches #136. It costs the crowd warmer nothing, since both share the per-market cache.sentoutcome on a tick that saw no new week. A navbar poll can consumerefresh_if_stale's True, which would leave the page the email links to cold._rendereddocstring's cache claim.Local check: warmer plus four concurrent requests for the cold newest page gave one matrix build, with all four served at 4.9s.
Not done here
cotmetrics.signals.append_trading_signals, which should be its own cotmetrics change.verify-deploy.shcan still land in the warm window. The newest page is ready once the log saysweekly pages: warmed <date>.Tests
pytest tests/gives 886 passed (CI env: dummy stores,PYTHONPATH=src).ruff check src testsis clean (ruff 0.15.22). New tests: concurrent cold requests build once,warm_newestfills the newest page and never raises, and the warm order pins weekly first.Not deployed.
🤖 Generated with Claude Code