Skip to content

Send the weekly email before warming the page caches #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mspinola merged 1 commit into from
Sep 25, 2026
Merged

Send the weekly email before warming the page caches #136

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions src/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ def store_poll_loop():

The navbar callback polls the same way and is the primary trigger, but dcc.Interval
is client-side: it only ticks while a browser tab is open. Without this loop the
first visitor after a release pays the ~90 second rebuild. With it, an unattended
first visitor after a release pays the rebuild. With it, an unattended
app is current before anyone arrives.

Every 5 minutes, always, rather than a window around the Friday release. The store
Expand Down Expand Up @@ -137,17 +137,11 @@ def store_poll_loop():
utils.cot_logger.info(weekly_email_trigger.startup_message())
while True:
time.sleep(STORE_POLL_SECONDS)
new_week = False
try:
if get_indexer().refresh_if_stale():
new_week = get_indexer().refresh_if_stale()
if new_week:
utils.cot_logger.info("Store poller: picked up a new COT week.")
# The new week keys every entry in the crowd board's and the
# heatmap's join caches, so the refresh just invalidated them
# all; re-warm in the background so the first visitor of the new
# week does not pay the cold renders. A thread rather than
# inline, to keep this loop's tick short.
threading.Thread(
target=warm_page_caches, name="page-warmer", daemon=True
).start()

# After the refresh, never before: refresh_if_stale blocks until the index
# matches the store, so by here the matrix the email builds is the new
Expand All @@ -166,6 +160,19 @@ def store_poll_loop():
# watching. Log and wait for the next tick.
utils.cot_logger.error(f"Store poller failed, will retry: {e}")

# The new week keys every entry in the crowd board's and the heatmap's join
# caches, so the refresh just invalidated them all; re-warm in the background
# so the first visitor of the new week does not pay the cold renders. A thread,
# to keep this loop's tick short. AFTER the email, deliberately: the warmers
# are CPU-bound Python in this process, and started first they contended for
# the GIL with the email's matrix build, so the email a subscriber is waiting
# for queued behind cache fills for pages nobody had opened yet. Outside the
# try, so a failed send still warms the pages.
if new_week:
threading.Thread(
target=warm_page_caches, name="page-warmer", daemon=True
).start()


def daily_options_update_scheduler():
import datetime
Expand Down
Loading