Conversation
* Add marketing campaign vanity links with funnel attribution Marketing can now create campaigns and per-role links entirely from Django admin, publishing clean portal URLs (/join/builders/ethcc) that need no deployment per campaign. An Amplify rule proxies the reserved /join/ namespace to a new campaigns backend app, which resolves the alias, logs a privacy-scrubbed redirect hit with bot classification, and 302s to the stored destination with server-built UTMs (plus forwarded ad click IDs so paid traffic keeps its GA join). Unknown links fail with 404, expired ones with 410, and destinations are validated against a portal-path allowlist on write and again before every redirect. Attribution is durable and first-touch: the frontend stores a structured campaign touch (30-day localStorage first touch plus a session touch), sends the opaque link tracking ID with wallet login, and the backend resolves it server-side onto the pending signup, never trusting browser UTM text and never letting attribution fail a signup. Email confirmation copies the snapshot into a write-once per-user acquisition record inside the same transaction that creates the user (savepointed so a failure cannot abort registration). GA now receives a sign_up event exactly once per created account, and the visible URL is cleaned of tracking params (including ref) while preserving all other query parameters, fixing the old referral cleanup that erased the whole query string. Campaign performance (redirect hits, wallet connects, registrations, and per-role activations computed from durable records, with community activation gated by an explicit per-task flag) renders on the campaign admin page for a restricted Marketing staff group. ## Claude Implementation Notes - backend/campaigns/models.py: MarketingCampaign (unique tracking_key -> utm_campaign), CampaignLink (unique role+alias, server-generated immutable tracking_id -> utm_id, destination allowlist via validate_destination_path), CampaignRedirectHit (append-only, no PII), UserAcquisitionAttribution (OneToOne user, snapshot columns + SET_NULL link FK) - backend/campaigns/services.py: UA bot classification, resolver lookup, best-effort hit logging, click-ID forwarding (CLICK_ID_FORWARD_PARAMS only), apply_pending_attribution (first-touch, window check, reset on expired-pending reuse), record_user_acquisition (nested atomic savepoint), campaign_report (Portal-DB funnel counts, distinct users) - backend/campaigns/views.py + urls.py: anonymous GET/HEAD resolver at /campaigns/redirect/<role>/<alias>, 302 + Cache-Control no-store, 404/410 fail-closed, throttle scope campaign_redirect - backend/campaigns/admin.py: campaign admin with link inline + performance table, link admin with hit/signup annotations and locked role/alias/tracking_id, read-only hit and attribution admins - backend/campaigns/migrations/: 0001 models, 0002 Marketing group with campaign-only permissions (runs create_permissions explicitly for fresh DBs) - backend/campaigns/management/commands/purge_campaign_hits.py: retention purge, default CAMPAIGN_HIT_RETENTION_DAYS=90, --dry-run - backend/ethereum_auth/models.py + migration 0005: PendingWalletSignup gains acquisition_campaign_link/acquisition_snapshot/acquisition_captured_at - backend/ethereum_auth/views.py: login applies pending attribution after update_or_create, resetting stale data when an expired pending row is reused; wrapped so it can never fail signup - backend/ethereum_auth/email_verification.py: record_user_acquisition called inside the confirm transaction right after user creation - backend/social_tasks/models.py + admin.py + migration 0007: SocialTask.counts_as_activation explicit allowlist flag for community activation - backend/tally/settings.py: campaigns app, campaign_redirect throttle rate, CAMPAIGN_HIT_RETENTION_DAYS, CAMPAIGN_ATTRIBUTION_WINDOW_DAYS - backend/tally/urls.py + backend/utils/throttling.py: campaigns URLconf include and CampaignRedirectRateThrottle - backend/campaigns/tests/: models/resolver/attribution/reporting/admin-permission suites (61 tests), incl. SIWE login and email-confirm integration and savepoint verification - frontend/src/lib/analytics.js: structured first/session campaign touches, getAcquisitionAttribution, clearAcquisitionAttribution, cleanTrackingParamsFromUrl (preserves non-tracking params), trackSignUp (created:true only, clears touches) - frontend/src/lib/auth.js: login payload carries attribution via dynamic analytics import (avoids module cycle); not cleared on success - frontend/src/App.svelte: ref capture no longer rewrites the URL; consolidated cleanTrackingParamsFromUrl(['ref']) in onMount - frontend/src/components/ProfileCompletionGuard.svelte: trackSignUp fired from the email-confirm response before finishProfileCompletion - frontend/src/tests/analytics.test.js: new campaign-attribution describe block (9 tests) - amplify.yml: /join/<*> reverse-proxy rule before the SPA catch-all - backend/CLAUDE.md + frontend/CLAUDE.md: campaigns app and attribution documentation * Update changelog * Harden campaign links against history loss and stale attribution Campaign links can no longer be deleted or moved between campaigns from the admin (pausing replaces deletion, so redirect-hit history is never cascaded away), and a campaign's tracking key is now immutable at the model level once links exist, keeping historical utm_campaign meaning and snapshot reporting stable. Destination validation additionally rejects percent-encoded syntax so encoded traversal cannot bypass the allowlist, and rolling back the marketing-group migration no longer deletes a group it may not have created. On the frontend, only touches carrying the opaque campaign link ID are stored, so a plain utm_source share can no longer lock the 30-day first-touch slot and silently block a later real campaign click from being attributed. ## Claude Implementation Notes - backend/campaigns/models.py: reject '%' in validate_destination_path (encoded traversal); MarketingCampaign.clean() rejects tracking_key changes once links exist; CampaignRedirectHit docstring notes the deliberate non-BaseModel append-only-log exception - backend/campaigns/admin.py: CampaignLinkInline can_delete=False and save_formset no longer deletes; CampaignLinkAdmin freezes campaign FK after create and limits delete permission to superusers - backend/campaigns/migrations/0002_marketing_group.py: reverse operation is now RunPython.noop (a pre-existing Marketing group must survive rollback) - backend/campaigns/tests/test_models.py: %2e%2e destination cases; tracking_key immutability coverage (locked with links, editable without) - backend/campaigns/tests/test_admin_permissions.py: exact (app_label, codename) allowlist assertion instead of spot checks - frontend/src/lib/analytics.js: buildStructuredAttribution requires utm_id, so utm_id-less landings store nothing and cannot shadow the session touch in getAcquisitionAttribution - frontend/src/tests/analytics.test.js: regression test that a utm_id-less landing does not lock the first-touch slot - frontend/CLAUDE.md: blank line after the new analytics heading (MD022) * Ignore whitespace-only campaign IDs in browser attribution A campaign link ID consisting only of whitespace (for example a mangled utm_id=%20 share) is now treated as absent: it is never stored, never locks the 30-day first-touch slot, and a stored first touch without a resolvable ID falls through to the session touch instead of shadowing it. ## Claude Implementation Notes - frontend/src/lib/analytics.js: resolvableUtmId helper trims and validates utm_id; buildStructuredAttribution stores the trimmed ID or nothing; getAcquisitionAttribution prefers the first touch only when its ID is resolvable, else falls through to the session touch - frontend/src/tests/analytics.test.js: whitespace-only utm_id case added to the first-touch lock regression test
Wallet session authentication resolved the signed-in account by case-insensitive wallet address on every single API request. PostgreSQL cannot answer that with the account table's only address index, which is case-sensitive, so each request scanned the whole table. During the 27 July degradation this one statement was 61.6% of database CPU. The authenticator now resolves the account through Django's own session machinery, which validates the session's auth backend, its auth hash and the active flag, and costs a single primary-key lookup. The session's wallet address must still bind to the resolved account, compared case-insensitively because sign-in stores a lowercased address while email confirmation stores database casing and production holds mixed-case rows. A mismatch raises rather than returning nothing, so the next authenticator in the chain cannot grant the request off the same session. CSRF enforcement stays at exactly the same point, before any account resolution. Three latent defects disappear with the address lookup: authenticating as whoever later took over a changed address, an uncaught multiple-objects error turning into a 500 for accounts differing only by case, and deactivated accounts staying signed in. Sessions predating the introduction of Django login carry no account id and are rejected; those wallets sign in again once, which is preferable to keeping the scan alive indefinitely for them. ## Claude Implementation Notes - backend/ethereum_auth/authentication.py: Replace User.objects.get(address__iexact=...) with request._request.user (DRF SessionAuthentication's own pattern). Add same_wallet() case-insensitive binding check that raises AuthenticationFailed on mismatch. Drop the now-unused get_user_model and timezone imports. - backend/ethereum_auth/views.py: verify_auth reuses request.user instead of repeating the address query (the endpoint cost two scans per call). refresh_session gates on request.user.is_authenticated so a rejected session stops rolling its own expiry forward. Response contracts unchanged. - backend/ethereum_auth/testing.py: New login_wallet_session() helper building a real session via force_login; mirrors service_accounts/testing.py. - backend/ethereum_auth/test_authentication.py: New. Covers inactive/deleted/password-rotated/address-changed/null-address, the old-address-takeover regression, mixed-case binding both directions, admin-only sessions, wallet switch, CSRF enforcement ordering and exempt views, and the verify/refresh contracts. WalletSessionQueryShapeTests asserts no address literal reaches SQL, paired with a test proving the assertion is live on whichever backend runs (iexact compiles to UPPER on PostgreSQL but LIKE on SQLite). - backend/ethereum_auth/tests.py, backend/poaps/tests/test_poaps.py: Switch the two hand-seeded sessions to the helper so they exercise the real flow.
Public profile and leaderboard reads accept a wallet address as a dual key and
resolve it case-insensitively. On PostgreSQL that compiles to a comparison on
the uppercased column, which the account table's only address index cannot
serve because it is case-sensitive, so those reads scan the table.
Adds a non-unique functional index on the uppercased address. Address
uniqueness semantics are untouched: the existing exact-match conditional unique
constraint stays exactly as it is, and accepted address formats do not change.
The index is built concurrently on PostgreSQL. Migrations run at container
start under an advisory lock with no lock timeout, so a plain index build would
take an exclusive lock and could wait indefinitely behind a long-running query,
stalling the deploy at precisely the moment a fix is most wanted. Other
backends get the ordinary index, since concurrent builds are PostgreSQL-only.
## Claude Implementation Notes
- backend/users/models.py: Add Meta.indexes with models.Index(Upper('address'), name='users_user_address_upper_idx'). Required in model state for two reasons: CI runs makemigrations --check, and tally/test_settings.py disables migrations so the test database is built from model state.
- backend/users/migrations/0022_user_address_upper_index.py: New, atomic = False. SeparateDatabaseAndState pairs the shared AddIndex state operation with a RunPython that emits CREATE INDEX CONCURRENTLY on PostgreSQL and a plain CREATE INDEX elsewhere, so SQLite dev and CI still migrate. Verified against a scratch PostgreSQL database with 60k rows: index is valid, and the planner switches from Seq Scan (cost 2113) to Index Scan (cost 8.43).
Community score and ranking queries were the second largest source of database load during the 27 July degradation. The ranking query annotates every visible account with correlated subqueries and then filters and orders on the computed total, so no index can serve it and each evaluation scans the whole population. Dashboard statistics evaluated that scan twice: once for the points total, and once more to build a set of account ids that nothing ever read. Removing the dead half is exactly behaviour-preserving and halves the cost of every statistics request. A second dead query object in the same function is gone too. The remaining scan is now shared. Both the ranking snapshot and the statistics summary take no request input and contain no per-account data, so each is computed once per minute rather than once per request. Everything personalized stays live per request: search, a caller's own rank, profile context, and the bounded detail hydration. Ranks may lag by up to a minute, which sits well inside the existing freshness envelope given chat XP already syncs hourly. Worth knowing: no cache backend is configured, so this is per-process local memory. Each worker on each container keeps its own copy, which means the relief scales with worker count and is strongest at steady state rather than at maximum fan-out. Making it independent of container count needs a shared cache tier, which is deliberately out of scope here. ## Claude Implementation Notes - backend/community_xp/cache.py: New. cached_or_compute() with 60s TTL, versioned keys, miss detected via `is None` so an empty ranking is a valid cached value, and nothing cached when compute raises. clear_community_caches() for tests. Module docstring records the LocMemCache scaling caveat. - backend/leaderboard/views.py: Delete the unused `'user_ids': set(score_queryset.values_list(...))` and the dead community_contribs queryset. Route the ranking snapshot and the stats summary through cached_or_compute. Deliberately does NOT reuse the memoized member set for the since= membership call: that call filters Creator rows by the eligible set WITHOUT POAP claimants, so substituting would silently widen "new community members". - backend/leaderboard/tests/test_community_query_counts.py: New. First query-count guards in the backend. Markers match on the users_user FROM clause plus the correlated MEE6 subquery, because values_list() inlines expressions and drops annotation aliases; a companion test proves the marker still detects a real scan. - backend/community_xp/tests/test_ranking_cache.py: New. Hit, miss, expiry, empty result, zero TTL, exception isolation, explicit clear, and cache-alias isolation. - backend/leaderboard/tests/test_community_search.py, test_stats.py, community_xp/tests/test_mee6_sync.py: clear_community_caches() in setUp. LocMemCache is not reset between tests.
The global shell issued several requests per navigation where one would do. In
the 27 July window that meant roughly six unread-count calls, three CSRF calls
and two profile reads for every app load, all landing on an already saturated
database.
The notification bell fetched the full notification list on every route change
and on every auth-store emission, even while closed, and each of those also
dragged a redundant unread-count along. Route changes now refresh only the
badge; the list loads when the panel is actually opened. Polling stays at sixty
seconds and keeps its shared timer and hidden-tab skip, but failed polls now
back off so a struggling backend is not hammered at a fixed rate by every open
tab.
The CSRF token could never be read from the cookie in production, because the
API and the portal are on different hosts, so every state-changing request
fetched a fresh one first. The token is now held in memory for the session and
cleared on sign-in, sign-out, wallet switch and email confirmation, the paths
where the server rotates it. A genuine CSRF rejection clears it too, told apart
from an ordinary permission rejection by the error detail. Nothing is retried:
POAP claims share the same client and must never be replayed.
The current profile was re-read on every role-gated navigation. Route guards
are a convenience gate rather than the security boundary, since the backend
authorizes every request independently, so successful reads are now cached for
thirty seconds. Every other caller asks for a fresh read explicitly, so sign-in,
wallet switch, profile edits and journey completion behave exactly as before; a
role change can take up to thirty seconds to show in navigation while the
backend refuses the data immediately.
Session refresh on tab focus is throttled to once a minute, and a verify call
that fails because the backend is unreachable now waits thirty seconds before
retrying instead of repeating on every subsequent trigger. Transient failures
still never sign anyone out.
## Claude Implementation Notes
- frontend/src/components/NotificationCenter.svelte: The $effect now only calls loadUnreadCount(); loadLatest() runs solely from toggleOpen. Keeps $location tracked so the badge refreshes on navigation.
- frontend/src/lib/notificationStore.js: Add skipPollUntil backoff (60s/180s/240s, reset on first success) checked by the poll tick and reset by reset(). markRead deliberately still refetches the count rather than decrementing locally: a count request can observe the server-side mark-read before the POST resolves, so a blind decrement double-subtracts (pinned by notificationPolling.test.js).
- frontend/src/lib/csrf.js: Add in-memory cachedCsrfToken (never persistent storage), plus exported clearCsrfToken() and isCsrfFailure(). Cookie still wins when readable, which keeps same-origin dev on Django's rotation.
- frontend/src/lib/api.js: Response interceptor clears the cached token on a real CSRF failure and does not retry. The 401/403 re-verify branch is kept intact: DRF answers 403, not 401, for an expired session because the first authenticator supplies no authenticate_header.
- frontend/src/lib/auth.js: clearCsrfToken() on login, logout and signup email confirm. verifyCooldownUntil (30s) absorbs repeat verifies after a 5xx, cleared on success or definitive rejection. performVerification takes force and forwards it to loadUser. visibilitychange refresh throttled to 60s via lastRefreshAt.
- frontend/src/lib/userStore.js: 30s success-cache TTL on loadUser({ force }); only successful loads set the timestamp, clearUser resets it, setUser starts it.
- frontend/src/components/*, frontend/src/routes/*: Pass { force: true } at every state-change call site so only the route guard uses the TTL.
- frontend/src/tests/csrf.test.js: New. csrf.js previously had zero coverage; all three files referencing it mocked it away. Covers reuse, coalescing, clearing, no persistent storage, and permission-403 vs CSRF-403.
- frontend/src/tests/notificationCenterRequests.test.js: New. Renders the component, which nothing did before, which is why the route-change bug shipped.
- frontend/src/tests/userStore.test.js, authSession.test.js: TTL, force, 5xx-preserves-user, 401-clears; visibility throttle and verify cooldown. Visibility assertions measure growth rather than absolute counts, since resetModules leaves listeners on the shared document.
Production logged only server errors, so the 27 July degradation left almost no application trace: tens of thousands of successful requests averaging nearly five seconds produced no log lines at all. The middleware already measured duration and built the message, then discarded both for anything that was not a 5xx. Requests at or above a configurable threshold, one second by default, now emit a single warning. Everything needed was already in place and is reused rather than rebuilt: the existing duration measurement, the existing path redaction, the existing skip list for static and health paths, and the request path rather than the full URL, so query strings were never logged and still are not. Server errors keep their existing error-level line and are not logged twice. ## Claude Implementation Notes - backend/tally/middleware/api_logging.py: Add an `elif duration_ms >= settings.SLOW_REQUEST_LOG_MS` warning branch after the existing 5xx and DEBUG branches, so a slow 5xx logs once as an error. Update the module docstring. - backend/tally/settings.py: SLOW_REQUEST_LOG_MS, env-overridable, default 1000, so the threshold is tunable without a deploy. The tally.api logger is already at WARNING in production, so the record emits through the existing JSON handler. - backend/tally/tests/test_api_logging.py: New SlowRequestLoggingTest covering the threshold, fast requests logging nothing, a slow 5xx logging only the error, slow 4xx, claim-link redaction in the warning, query strings never appearing, and skipped paths staying silent.
Gunicorn 26 opens a control socket in the working directory, which the container owns as root while the application process runs unprivileged, so every boot logs a permission warning. Nothing in the application uses the control socket. Unrelated to the 27 July degradation. ## Claude Implementation Notes - backend/Dockerfile, backend/deploy-apprunner.sh, backend/deploy-apprunner-dev.sh: Add --no-control-socket to all five copies of the Gunicorn start command. Worker count, bind, timeout and logging flags unchanged. Flag confirmed present in the pinned gunicorn 26.0.0.
## Claude Implementation Notes - backend/CLAUDE.md: Record that EthereumAuthentication runs on every DRF request and must never look a user up by address; document the case-insensitive wallet binding, the login_wallet_session test helper, the Upper(address) index, the community aggregate cache and its clear_community_caches() test requirement, and SLOW_REQUEST_LOG_MS. - frontend/CLAUDE.md: Document the notification route-change rule and poll backoff, why markRead refetches rather than decrementing, the new CSRF section (cross-host cookie, in-memory-only cache, clear paths, why the 403 re-verify branch must stay), the userStore TTL and its force call sites, and the verify cooldown plus refresh throttle.
…eads
Addresses review of the outage fixes.
An interrupted concurrent index build leaves the index in the catalog marked
invalid. Retrying with "if not exists" then silently skips it rather than
rebuilding, so a deploy that was interrupted once would leave the
case-insensitive address lookups permanently unindexed while appearing to
succeed. Verified against PostgreSQL: the retry logs "already exists, skipping"
and the index stays invalid. The migration now detects that state and clears
the leftover before rebuilding.
Forcing a profile refresh could also settle for a response that predated the
change it was meant to observe. Callers force a refresh precisely because they
just altered server state, so a forced read now queues behind any request
already in flight instead of joining it, and the newer response is the one that
lands in the store. Unforced callers still share in-flight work.
Notification poll backoff no longer reacts to superseded responses, so a stale
failure cannot re-arm a delay that a newer success already cleared, or
reintroduce one after sign-out.
## Claude Implementation Notes
- backend/users/migrations/0022_user_address_upper_index.py: Add _index_is_invalid() checking pg_index.indisvalid via to_regclass (search-path aware), and DROP INDEX CONCURRENTLY the leftover before recreating. Non-PostgreSQL path unchanged. Verified end to end against a scratch database: forced an invalid index via the catalog, confirmed CREATE INDEX CONCURRENTLY IF NOT EXISTS no-ops on it, then confirmed create_index() takes it INVALID to VALID and that drop/recreate still works. Not unit-tested: migrations are disabled under test settings and CONCURRENTLY cannot run inside the test transaction.
- frontend/src/lib/userStore.js: loadUser({ force: true }) no longer returns an in-flight promise; it awaits the previous load, then issues its own request. A token guard stops a superseded load from clearing the newer in-flight handle, and clearUser() resets both.
- frontend/src/lib/notificationStore.js: Move notePollResult() after the epoch/version guard in both handlers.
- frontend/src/tests/userStore.test.js: The 5xx TTL test now advances mocked time past the original TTL and reads UNFORCED, which is the only way to catch a failure wrongly refreshing the timestamp; the previous forced read hit the network regardless and proved nothing. Add a test that a forced load started during an in-flight load resolves with the newer response.
- backend/tally/settings.py: String default for os.environ.get (Ruff PLW1508).
- frontend/src/components/NotificationCenter.svelte: Reword comment to "unread count" per the project terminology rule.
A profile request already in flight when the account changes would still write its result into the store when it resolved, restoring the previous account after a sign-out or a wallet switch and starting a freshness window for data that no longer applied. The notification store already guards against exactly this with its epoch counter; the user store did not. Every write is now gated on the load still being the current one, and clearing the user invalidates whatever is in flight. ## Claude Implementation Notes - frontend/src/lib/userStore.js: Gate the success write, lastLoadedAt, and the error write on currentLoadToken === token, and bail before the loading flag if the load was superseded while queued. clearUser() already drops the token, which is what invalidates in-flight work. The request still resolves or rejects for its caller; only the shared store state is protected. - frontend/src/tests/userStore.test.js: Two regression tests covering a deferred request, clearUser(), then resolution: the store must stay empty, and the discarded response must not seed the TTL. Both verified to fail without the guard (the first asserts the logged-out account was being restored), so they are not vacuous. - frontend/CLAUDE.md: Document the token guard and why forced loads queue rather than join.
Writing the user store directly always carries state that is newer than any
read already in flight: the server's own user from a profile save, a role join
or a claim. Those writes could still be overwritten when an earlier request
resolved, reverting a saved profile or dropping a merged field.
Clearing the session already invalidated in-flight work. The two direct writes
now do the same, through one shared helper, so all three behave alike. A read
started after such a write takes a fresh token and is unaffected.
## Claude Implementation Notes
- frontend/src/lib/userStore.js: Extract invalidateInFlightLoad() and call it from setUser, updateUser and clearUser. updateUser was not in the review finding but carries the identical defect across ~10 call sites, and its partial merge is fully replaced by a stale response; fixing only setUser would have left the bug next door and an asymmetry that reads as intentional.
- frontend/src/tests/userStore.test.js: Regression tests for a pending load followed by setUser, and by updateUser, then a late response. Both verified to fail without the fix ('Stale' beating 'Saved' and 'Merged').
- frontend/CLAUDE.md: Note that all three direct writes invalidate, and why.
Starting a wallet signup with an unregistered address marks the session as not
authenticated but deliberately leaves the previous sign-in and address in
place. The verification and refresh endpoints had been narrowed to the resolved
user alone, so that session read as fully signed in as the previous account:
verification never reported the pending signup the client was waiting for, and
refresh kept extending a session the user was in the middle of leaving.
Journey and waitlist flows also refresh the profile after acting, and several of
those refreshes could be served from the new short-lived cache instead of
rereading, leaving waitlist, journey and points state a step behind. Those call
sites now ask for a fresh read; the navigation guard, which is where the saved
requests actually come from, still uses the cache.
Auth requests additionally had no way to notice a rejected security token. They
are the only users of their own client, so a token rotated in another tab was
never cleared and session refresh would keep failing every five minutes until
the tab reloaded.
## Claude Implementation Notes
- backend/ethereum_auth/views.py: Restore the session 'authenticated' flag alongside request.user.is_authenticated in verify_auth and refresh_session. The pending-signup branch of login() sets it False while leaving _auth_user_id and ethereum_address, so DRF's SessionAuthentication still resolves the old user; the flag is what distinguishes the two.
- backend/ethereum_auth/test_authentication.py: Two regression tests for that session shape, verified to fail against the previous gate.
- frontend/src/lib/auth.js: Add the authAxios response interceptor mirroring api.js (clear on a real CSRF failure, never retry). Force the post-login and signup-confirm profile reads, both session boundaries.
- frontend/src/routes/{BuilderJourney,CommunityJourney,ValidatorWaitlist}.svelte, frontend/src/components/funnel/RoleLanding.svelte, frontend/src/components/social-tasks/SocialTaskCard.svelte: Pass { force: true } on post-mutation refreshes. These call loadUser?.() with optional chaining, which an earlier literal search for loadUser() missed. ValidatorWaitlist's read straight after setUser stays cached on purpose: setUser already wrote the authoritative user.
- frontend/src/tests/authSession.test.js: Mock interceptors.response, capture the handler at registration since importAuth clears mock calls, and cover CSRF-403 clearing versus permission-403 not clearing.
…eshes
The pending-signup regression test only set the session key, without the
signup record it points at, so the endpoint took the "no pending signup"
path and the test never checked the response the client actually depends on.
It now creates the record and asserts the pending signup and its address,
which is the half of the contract the fix was really about.
The fire-and-forget profile refreshes in the journey and task flows also had
no rejection handler, so a failed refresh surfaced as an unhandled rejection
rather than being quietly ignored as intended.
## Claude Implementation Notes
- backend/ethereum_auth/test_authentication.py: Create an active PendingWalletSignup and set pending_wallet_signup_id, then assert pending_signup is True and address is the pending one. get_pending_signup_from_session() filters on STATUS_PENDING and is_active(), so the session key alone resolved to None. Re-verified by reverting the gate in place: both tests fail (True is not false, 200 != 401) and pass with it.
- frontend/src/routes/{BuilderJourney,CommunityJourney}.svelte, frontend/src/components/social-tasks/SocialTaskCard.svelte: Add ?.catch(() => {}) to the six non-awaited refreshes, matching the existing convention in RoleLanding and ValidatorWaitlist. The three awaited calls are inside try/catch blocks and are already handled, so they are left alone.
Validators can now connect Telegram groups to their validator from the
portal. A new "Telegram Support" page in the validator area issues
one-time group bind codes: the validator creates a Telegram group, adds
the Deckard support bot, and runs /bindcode <code> there; the bot
redeems the code server-to-server and the group is bound to the
validator. A validator can bind multiple groups (one code per group),
codes expire after 48 hours, and unredeemed codes can be revoked.
The bind code follows the service-account token design: only a SHA-256
digest is stored, lookup is by a non-secret identifier embedded in the
code, and redemption compares digests in constant time. Redemption is
gated by a new telegram_bind:redeem service-account scope reserved for
the Deckard bot and is atomic and single-use: it records the bound
group chat id and the redeeming Telegram uid, and upserts a new
TelegramConnection (social connection pattern, bot-confirmed instead of
OAuth) so the portal account carries a verifiable Telegram identity.
Issuance is throttled per user; the plaintext code is returned exactly
once and never listed again.
## Claude Implementation Notes
- backend/validators/models.py: TelegramGroupBindCode model — identifier
+ digest storage (hash_code/identifier_from_plaintext/issue mirroring
ServiceAccountToken), issued/redeemed/expired/revoked statuses, 48h
default TTL, lazy effective_status, is_redeemable helper
- backend/validators/migrations/0018_telegramgroupbindcode.py: schema
- backend/validators/views.py: TelegramBindCodeViewSet (create issues +
returns plaintext once, mine lists metadata, revoke with row lock and
409 for redeemed codes) and TelegramBindCodeRedeemView (service
account auth, required_scopes {'*': telegram_bind:redeem}, payload
validation, select_for_update single-use redemption, constant-time
digest compare, TelegramConnection upsert that never blanks an
existing username)
- backend/validators/serializers.py: TelegramBindCodeSerializer —
metadata only, status is the lazy-expiry effective status
- backend/validators/urls.py: explicit redeem path before the router so
the viewset detail route never swallows it; telegram-bind-codes
registered before the catch-all validator route
- backend/social_connections/models.py: TelegramConnection subclass of
the SocialConnection abstract base (numeric uid identity, display-only
username, no OAuth), db_table social_connections_telegram
- backend/social_connections/migrations/0008_telegramconnection.py: schema
- backend/social_connections/admin.py: TelegramConnection admin
- backend/service_accounts/scopes.py: TELEGRAM_BIND_REDEEM_SCOPE added
to ALLOWED_SERVICE_ACCOUNT_SCOPES so admin/command token issuance
accepts it
- backend/utils/throttling.py + backend/tally/settings.py:
TelegramBindCodeIssueRateThrottle, telegram_bind_issue 10/hour
- backend/validators/tests/test_telegram_bind_codes.py: 22 tests —
issuance auth + validator gate + single-shot plaintext, mine
isolation and lazy expiry, revoke rules, redeem scope gate,
single-use, expiry, invalid/tampered codes, connection upsert
- frontend/src/routes/ValidatorTelegram.svelte: Link-a-Telegram-group
page (Svelte 5 runes) — generate code, show once with /bindcode copy
command, list codes with status chips and revoke
- frontend/src/App.svelte: /validators/telegram role-gated route
- frontend/src/components/Sidebar.svelte: Telegram Support link in both
validator sections (role-locked pattern)
- frontend/src/lib/api.js: issue/list/revoke bind-code helpers
- frontend/src/lib/config.js: DECKARD_BOT_USERNAME from
VITE_DECKARD_BOT_USERNAME with generic-wording fallback
- backend/CLAUDE.md + frontend/CLAUDE.md: documented the new model,
endpoints, scope, route, and API helpers
* Stop re-scanning every user account on each authenticated request
Wallet session authentication resolved the signed-in account by
case-insensitive wallet address on every single API request. PostgreSQL cannot
answer that with the account table's only address index, which is
case-sensitive, so each request scanned the whole table. During the 27 July
degradation this one statement was 61.6% of database CPU.
The authenticator now resolves the account through Django's own session
machinery, which validates the session's auth backend, its auth hash and the
active flag, and costs a single primary-key lookup. The session's wallet
address must still bind to the resolved account, compared case-insensitively
because sign-in stores a lowercased address while email confirmation stores
database casing and production holds mixed-case rows. A mismatch raises rather
than returning nothing, so the next authenticator in the chain cannot grant the
request off the same session. CSRF enforcement stays at exactly the same point,
before any account resolution.
Three latent defects disappear with the address lookup: authenticating as
whoever later took over a changed address, an uncaught multiple-objects error
turning into a 500 for accounts differing only by case, and deactivated
accounts staying signed in. Sessions predating the introduction of Django login
carry no account id and are rejected; those wallets sign in again once, which
is preferable to keeping the scan alive indefinitely for them.
## Claude Implementation Notes
- backend/ethereum_auth/authentication.py: Replace User.objects.get(address__iexact=...) with request._request.user (DRF SessionAuthentication's own pattern). Add same_wallet() case-insensitive binding check that raises AuthenticationFailed on mismatch. Drop the now-unused get_user_model and timezone imports.
- backend/ethereum_auth/views.py: verify_auth reuses request.user instead of repeating the address query (the endpoint cost two scans per call). refresh_session gates on request.user.is_authenticated so a rejected session stops rolling its own expiry forward. Response contracts unchanged.
- backend/ethereum_auth/testing.py: New login_wallet_session() helper building a real session via force_login; mirrors service_accounts/testing.py.
- backend/ethereum_auth/test_authentication.py: New. Covers inactive/deleted/password-rotated/address-changed/null-address, the old-address-takeover regression, mixed-case binding both directions, admin-only sessions, wallet switch, CSRF enforcement ordering and exempt views, and the verify/refresh contracts. WalletSessionQueryShapeTests asserts no address literal reaches SQL, paired with a test proving the assertion is live on whichever backend runs (iexact compiles to UPPER on PostgreSQL but LIKE on SQLite).
- backend/ethereum_auth/tests.py, backend/poaps/tests/test_poaps.py: Switch the two hand-seeded sessions to the helper so they exercise the real flow.
* Index case-insensitive wallet address lookups
Public profile and leaderboard reads accept a wallet address as a dual key and
resolve it case-insensitively. On PostgreSQL that compiles to a comparison on
the uppercased column, which the account table's only address index cannot
serve because it is case-sensitive, so those reads scan the table.
Adds a non-unique functional index on the uppercased address. Address
uniqueness semantics are untouched: the existing exact-match conditional unique
constraint stays exactly as it is, and accepted address formats do not change.
The index is built concurrently on PostgreSQL. Migrations run at container
start under an advisory lock with no lock timeout, so a plain index build would
take an exclusive lock and could wait indefinitely behind a long-running query,
stalling the deploy at precisely the moment a fix is most wanted. Other
backends get the ordinary index, since concurrent builds are PostgreSQL-only.
## Claude Implementation Notes
- backend/users/models.py: Add Meta.indexes with models.Index(Upper('address'), name='users_user_address_upper_idx'). Required in model state for two reasons: CI runs makemigrations --check, and tally/test_settings.py disables migrations so the test database is built from model state.
- backend/users/migrations/0022_user_address_upper_index.py: New, atomic = False. SeparateDatabaseAndState pairs the shared AddIndex state operation with a RunPython that emits CREATE INDEX CONCURRENTLY on PostgreSQL and a plain CREATE INDEX elsewhere, so SQLite dev and CI still migrate. Verified against a scratch PostgreSQL database with 60k rows: index is valid, and the planner switches from Seq Scan (cost 2113) to Index Scan (cost 8.43).
* Cut redundant community leaderboard scoring work
Community score and ranking queries were the second largest source of database
load during the 27 July degradation. The ranking query annotates every visible
account with correlated subqueries and then filters and orders on the computed
total, so no index can serve it and each evaluation scans the whole population.
Dashboard statistics evaluated that scan twice: once for the points total, and
once more to build a set of account ids that nothing ever read. Removing the
dead half is exactly behaviour-preserving and halves the cost of every
statistics request. A second dead query object in the same function is gone too.
The remaining scan is now shared. Both the ranking snapshot and the statistics
summary take no request input and contain no per-account data, so each is
computed once per minute rather than once per request. Everything personalized
stays live per request: search, a caller's own rank, profile context, and the
bounded detail hydration. Ranks may lag by up to a minute, which sits well
inside the existing freshness envelope given chat XP already syncs hourly.
Worth knowing: no cache backend is configured, so this is per-process local
memory. Each worker on each container keeps its own copy, which means the
relief scales with worker count and is strongest at steady state rather than at
maximum fan-out. Making it independent of container count needs a shared cache
tier, which is deliberately out of scope here.
## Claude Implementation Notes
- backend/community_xp/cache.py: New. cached_or_compute() with 60s TTL, versioned keys, miss detected via `is None` so an empty ranking is a valid cached value, and nothing cached when compute raises. clear_community_caches() for tests. Module docstring records the LocMemCache scaling caveat.
- backend/leaderboard/views.py: Delete the unused `'user_ids': set(score_queryset.values_list(...))` and the dead community_contribs queryset. Route the ranking snapshot and the stats summary through cached_or_compute. Deliberately does NOT reuse the memoized member set for the since= membership call: that call filters Creator rows by the eligible set WITHOUT POAP claimants, so substituting would silently widen "new community members".
- backend/leaderboard/tests/test_community_query_counts.py: New. First query-count guards in the backend. Markers match on the users_user FROM clause plus the correlated MEE6 subquery, because values_list() inlines expressions and drops annotation aliases; a companion test proves the marker still detects a real scan.
- backend/community_xp/tests/test_ranking_cache.py: New. Hit, miss, expiry, empty result, zero TTL, exception isolation, explicit clear, and cache-alias isolation.
- backend/leaderboard/tests/test_community_search.py, test_stats.py, community_xp/tests/test_mee6_sync.py: clear_community_caches() in setUp. LocMemCache is not reset between tests.
* Stop the portal shell from multiplying requests per page view
The global shell issued several requests per navigation where one would do. In
the 27 July window that meant roughly six unread-count calls, three CSRF calls
and two profile reads for every app load, all landing on an already saturated
database.
The notification bell fetched the full notification list on every route change
and on every auth-store emission, even while closed, and each of those also
dragged a redundant unread-count along. Route changes now refresh only the
badge; the list loads when the panel is actually opened. Polling stays at sixty
seconds and keeps its shared timer and hidden-tab skip, but failed polls now
back off so a struggling backend is not hammered at a fixed rate by every open
tab.
The CSRF token could never be read from the cookie in production, because the
API and the portal are on different hosts, so every state-changing request
fetched a fresh one first. The token is now held in memory for the session and
cleared on sign-in, sign-out, wallet switch and email confirmation, the paths
where the server rotates it. A genuine CSRF rejection clears it too, told apart
from an ordinary permission rejection by the error detail. Nothing is retried:
POAP claims share the same client and must never be replayed.
The current profile was re-read on every role-gated navigation. Route guards
are a convenience gate rather than the security boundary, since the backend
authorizes every request independently, so successful reads are now cached for
thirty seconds. Every other caller asks for a fresh read explicitly, so sign-in,
wallet switch, profile edits and journey completion behave exactly as before; a
role change can take up to thirty seconds to show in navigation while the
backend refuses the data immediately.
Session refresh on tab focus is throttled to once a minute, and a verify call
that fails because the backend is unreachable now waits thirty seconds before
retrying instead of repeating on every subsequent trigger. Transient failures
still never sign anyone out.
## Claude Implementation Notes
- frontend/src/components/NotificationCenter.svelte: The $effect now only calls loadUnreadCount(); loadLatest() runs solely from toggleOpen. Keeps $location tracked so the badge refreshes on navigation.
- frontend/src/lib/notificationStore.js: Add skipPollUntil backoff (60s/180s/240s, reset on first success) checked by the poll tick and reset by reset(). markRead deliberately still refetches the count rather than decrementing locally: a count request can observe the server-side mark-read before the POST resolves, so a blind decrement double-subtracts (pinned by notificationPolling.test.js).
- frontend/src/lib/csrf.js: Add in-memory cachedCsrfToken (never persistent storage), plus exported clearCsrfToken() and isCsrfFailure(). Cookie still wins when readable, which keeps same-origin dev on Django's rotation.
- frontend/src/lib/api.js: Response interceptor clears the cached token on a real CSRF failure and does not retry. The 401/403 re-verify branch is kept intact: DRF answers 403, not 401, for an expired session because the first authenticator supplies no authenticate_header.
- frontend/src/lib/auth.js: clearCsrfToken() on login, logout and signup email confirm. verifyCooldownUntil (30s) absorbs repeat verifies after a 5xx, cleared on success or definitive rejection. performVerification takes force and forwards it to loadUser. visibilitychange refresh throttled to 60s via lastRefreshAt.
- frontend/src/lib/userStore.js: 30s success-cache TTL on loadUser({ force }); only successful loads set the timestamp, clearUser resets it, setUser starts it.
- frontend/src/components/*, frontend/src/routes/*: Pass { force: true } at every state-change call site so only the route guard uses the TTL.
- frontend/src/tests/csrf.test.js: New. csrf.js previously had zero coverage; all three files referencing it mocked it away. Covers reuse, coalescing, clearing, no persistent storage, and permission-403 vs CSRF-403.
- frontend/src/tests/notificationCenterRequests.test.js: New. Renders the component, which nothing did before, which is why the route-change bug shipped.
- frontend/src/tests/userStore.test.js, authSession.test.js: TTL, force, 5xx-preserves-user, 401-clears; visibility throttle and verify cooldown. Visibility assertions measure growth rather than absolute counts, since resetModules leaves listeners on the shared document.
* Make slow requests visible in production logs
Production logged only server errors, so the 27 July degradation left almost no
application trace: tens of thousands of successful requests averaging nearly
five seconds produced no log lines at all. The middleware already measured
duration and built the message, then discarded both for anything that was not a
5xx.
Requests at or above a configurable threshold, one second by default, now emit
a single warning. Everything needed was already in place and is reused rather
than rebuilt: the existing duration measurement, the existing path redaction,
the existing skip list for static and health paths, and the request path rather
than the full URL, so query strings were never logged and still are not. Server
errors keep their existing error-level line and are not logged twice.
## Claude Implementation Notes
- backend/tally/middleware/api_logging.py: Add an `elif duration_ms >= settings.SLOW_REQUEST_LOG_MS` warning branch after the existing 5xx and DEBUG branches, so a slow 5xx logs once as an error. Update the module docstring.
- backend/tally/settings.py: SLOW_REQUEST_LOG_MS, env-overridable, default 1000, so the threshold is tunable without a deploy. The tally.api logger is already at WARNING in production, so the record emits through the existing JSON handler.
- backend/tally/tests/test_api_logging.py: New SlowRequestLoggingTest covering the threshold, fast requests logging nothing, a slow 5xx logging only the error, slow 4xx, claim-link redaction in the warning, query strings never appearing, and skipped paths staying silent.
* Silence the Gunicorn control socket warning at startup
Gunicorn 26 opens a control socket in the working directory, which the
container owns as root while the application process runs unprivileged, so
every boot logs a permission warning. Nothing in the application uses the
control socket. Unrelated to the 27 July degradation.
## Claude Implementation Notes
- backend/Dockerfile, backend/deploy-apprunner.sh, backend/deploy-apprunner-dev.sh: Add --no-control-socket to all five copies of the Gunicorn start command. Worker count, bind, timeout and logging flags unchanged. Flag confirmed present in the pinned gunicorn 26.0.0.
* Document the request-volume and query-shape constraints
## Claude Implementation Notes
- backend/CLAUDE.md: Record that EthereumAuthentication runs on every DRF request and must never look a user up by address; document the case-insensitive wallet binding, the login_wallet_session test helper, the Upper(address) index, the community aggregate cache and its clear_community_caches() test requirement, and SLOW_REQUEST_LOG_MS.
- frontend/CLAUDE.md: Document the notification route-change rule and poll backoff, why markRead refetches rather than decrementing, the new CSRF section (cross-host cookie, in-memory-only cache, clear paths, why the 403 re-verify branch must stay), the userStore TTL and its force call sites, and the verify cooldown plus refresh throttle.
* Repair an interrupted address index build and honour forced profile reads
Addresses review of the outage fixes.
An interrupted concurrent index build leaves the index in the catalog marked
invalid. Retrying with "if not exists" then silently skips it rather than
rebuilding, so a deploy that was interrupted once would leave the
case-insensitive address lookups permanently unindexed while appearing to
succeed. Verified against PostgreSQL: the retry logs "already exists, skipping"
and the index stays invalid. The migration now detects that state and clears
the leftover before rebuilding.
Forcing a profile refresh could also settle for a response that predated the
change it was meant to observe. Callers force a refresh precisely because they
just altered server state, so a forced read now queues behind any request
already in flight instead of joining it, and the newer response is the one that
lands in the store. Unforced callers still share in-flight work.
Notification poll backoff no longer reacts to superseded responses, so a stale
failure cannot re-arm a delay that a newer success already cleared, or
reintroduce one after sign-out.
## Claude Implementation Notes
- backend/users/migrations/0022_user_address_upper_index.py: Add _index_is_invalid() checking pg_index.indisvalid via to_regclass (search-path aware), and DROP INDEX CONCURRENTLY the leftover before recreating. Non-PostgreSQL path unchanged. Verified end to end against a scratch database: forced an invalid index via the catalog, confirmed CREATE INDEX CONCURRENTLY IF NOT EXISTS no-ops on it, then confirmed create_index() takes it INVALID to VALID and that drop/recreate still works. Not unit-tested: migrations are disabled under test settings and CONCURRENTLY cannot run inside the test transaction.
- frontend/src/lib/userStore.js: loadUser({ force: true }) no longer returns an in-flight promise; it awaits the previous load, then issues its own request. A token guard stops a superseded load from clearing the newer in-flight handle, and clearUser() resets both.
- frontend/src/lib/notificationStore.js: Move notePollResult() after the epoch/version guard in both handlers.
- frontend/src/tests/userStore.test.js: The 5xx TTL test now advances mocked time past the original TTL and reads UNFORCED, which is the only way to catch a failure wrongly refreshing the timestamp; the previous forced read hit the network regardless and proved nothing. Add a test that a forced load started during an in-flight load resolves with the newer response.
- backend/tally/settings.py: String default for os.environ.get (Ruff PLW1508).
- frontend/src/components/NotificationCenter.svelte: Reword comment to "unread count" per the project terminology rule.
* Discard profile responses that arrive after sign-out
A profile request already in flight when the account changes would still write
its result into the store when it resolved, restoring the previous account
after a sign-out or a wallet switch and starting a freshness window for data
that no longer applied. The notification store already guards against exactly
this with its epoch counter; the user store did not.
Every write is now gated on the load still being the current one, and clearing
the user invalidates whatever is in flight.
## Claude Implementation Notes
- frontend/src/lib/userStore.js: Gate the success write, lastLoadedAt, and the error write on currentLoadToken === token, and bail before the loading flag if the load was superseded while queued. clearUser() already drops the token, which is what invalidates in-flight work. The request still resolves or rejects for its caller; only the shared store state is protected.
- frontend/src/tests/userStore.test.js: Two regression tests covering a deferred request, clearUser(), then resolution: the store must stay empty, and the discarded response must not seed the TTL. Both verified to fail without the guard (the first asserts the logged-out account was being restored), so they are not vacuous.
- frontend/CLAUDE.md: Document the token guard and why forced loads queue rather than join.
* Stop a late profile read from reverting a just-saved profile
Writing the user store directly always carries state that is newer than any
read already in flight: the server's own user from a profile save, a role join
or a claim. Those writes could still be overwritten when an earlier request
resolved, reverting a saved profile or dropping a merged field.
Clearing the session already invalidated in-flight work. The two direct writes
now do the same, through one shared helper, so all three behave alike. A read
started after such a write takes a fresh token and is unaffected.
## Claude Implementation Notes
- frontend/src/lib/userStore.js: Extract invalidateInFlightLoad() and call it from setUser, updateUser and clearUser. updateUser was not in the review finding but carries the identical defect across ~10 call sites, and its partial merge is fully replaced by a stale response; fixing only setUser would have left the bug next door and an asymmetry that reads as intentional.
- frontend/src/tests/userStore.test.js: Regression tests for a pending load followed by setUser, and by updateUser, then a late response. Both verified to fail without the fix ('Stale' beating 'Saved' and 'Merged').
- frontend/CLAUDE.md: Note that all three direct writes invalidate, and why.
* Keep signup-in-progress sessions from reading as signed in
Starting a wallet signup with an unregistered address marks the session as not
authenticated but deliberately leaves the previous sign-in and address in
place. The verification and refresh endpoints had been narrowed to the resolved
user alone, so that session read as fully signed in as the previous account:
verification never reported the pending signup the client was waiting for, and
refresh kept extending a session the user was in the middle of leaving.
Journey and waitlist flows also refresh the profile after acting, and several of
those refreshes could be served from the new short-lived cache instead of
rereading, leaving waitlist, journey and points state a step behind. Those call
sites now ask for a fresh read; the navigation guard, which is where the saved
requests actually come from, still uses the cache.
Auth requests additionally had no way to notice a rejected security token. They
are the only users of their own client, so a token rotated in another tab was
never cleared and session refresh would keep failing every five minutes until
the tab reloaded.
## Claude Implementation Notes
- backend/ethereum_auth/views.py: Restore the session 'authenticated' flag alongside request.user.is_authenticated in verify_auth and refresh_session. The pending-signup branch of login() sets it False while leaving _auth_user_id and ethereum_address, so DRF's SessionAuthentication still resolves the old user; the flag is what distinguishes the two.
- backend/ethereum_auth/test_authentication.py: Two regression tests for that session shape, verified to fail against the previous gate.
- frontend/src/lib/auth.js: Add the authAxios response interceptor mirroring api.js (clear on a real CSRF failure, never retry). Force the post-login and signup-confirm profile reads, both session boundaries.
- frontend/src/routes/{BuilderJourney,CommunityJourney,ValidatorWaitlist}.svelte, frontend/src/components/funnel/RoleLanding.svelte, frontend/src/components/social-tasks/SocialTaskCard.svelte: Pass { force: true } on post-mutation refreshes. These call loadUser?.() with optional chaining, which an earlier literal search for loadUser() missed. ValidatorWaitlist's read straight after setUser stays cached on purpose: setUser already wrote the authoritative user.
- frontend/src/tests/authSession.test.js: Mock interceptors.response, capture the handler at registration since importAuth clears mock calls, and cover CSRF-403 clearing versus permission-403 not clearing.
* Assert the pending-signup response and catch best-effort profile refreshes
The pending-signup regression test only set the session key, without the
signup record it points at, so the endpoint took the "no pending signup"
path and the test never checked the response the client actually depends on.
It now creates the record and asserts the pending signup and its address,
which is the half of the contract the fix was really about.
The fire-and-forget profile refreshes in the journey and task flows also had
no rejection handler, so a failed refresh surfaced as an unhandled rejection
rather than being quietly ignored as intended.
## Claude Implementation Notes
- backend/ethereum_auth/test_authentication.py: Create an active PendingWalletSignup and set pending_wallet_signup_id, then assert pending_signup is True and address is the pending one. get_pending_signup_from_session() filters on STATUS_PENDING and is_active(), so the session key alone resolved to None. Re-verified by reverting the gate in place: both tests fail (True is not false, 200 != 401) and pass with it.
- frontend/src/routes/{BuilderJourney,CommunityJourney}.svelte, frontend/src/components/social-tasks/SocialTaskCard.svelte: Add ?.catch(() => {}) to the six non-awaited refreshes, matching the existing convention in RoleLanding and ValidatorWaitlist. The three awaited calls are inside try/catch blocks and are already handled, so they are left alone.
Getting a copy of production into a local environment now runs as a single command: it dumps production with pg_dump, restores that into a throwaway local Postgres container, brings the copy up to the current schema, and only then converts it to SQLite. The previous approach serialized production over the network row by row and never completed against the current data volume. The two overlapping sync commands are consolidated into one, and the documentation now records why each stage of the pipeline exists — the schema drift, identifier mismatches, seeded rows and unguarded model signals that each caused a failed restore before the shape settled. ## Claude Implementation Notes - backend/scripts/sync_prod_to_sqlite.py: New end-to-end sync. Stages are pg_dump, restore into a local Postgres 17 container, migrate that copy, dumpdata locally, loaddata into a fresh SQLite, rebuild the leaderboard. Pins the Docker platform (cached amd64 images fail on Apple Silicon), keeps contenttypes/auth.permission in the export so permission ids match, clears all tables but django_migrations, and suppresses model signals during the load. Resume flags for each stage. - .claude/commands/sync-db.md: Rewritten around the new script; records the rationale per stage, the unguarded post_save receivers it works around, and why the other two scripts are not the local path. - .claude/commands/migrate-to-sqlite.md: Deleted; folded into sync-db. - backend/scripts/README.md: Adds a which-script-do-I-want table and the same rationale and known-bug notes. - backend/CLAUDE.md: Replaces the RDS-to-SQLite section with the new script.
The sync now recreates the local Postgres container before every restore, so tables created by a previous run's migrations can no longer survive the dump's --clean pass and corrupt the copy. The restore aborts on the first failed statement instead of tolerating errors, the production URL is handed to pg_dump whole instead of being hand-parsed (percent-encoded passwords and sslmode query params now work), and --reuse-postgres exits with an error when there is no container to reuse rather than exporting a freshly created empty database over the local SQLite. ## Claude Implementation Notes - backend/scripts/sync_prod_to_sqlite.py: dump_production passes the SSM value directly as pg_dump --dbname and drops the URL parsing plus the PGPASSWORD env; start_postgres gains fresh=True (docker rm -f before starting) used by the restore path, with pg_container_exists extracted as a helper; restore runs psql with ON_ERROR_STOP=1 and an updated rationale comment; main fails fast on --reuse-postgres when pg_container_exists() is false.
…v var The bind code issuance log now identifies the issuing user by portal id only, keeping full wallet addresses out of application log sinks in line with the portal's address-privacy stance. The Deckard bot username setting is now part of the documented frontend environment so deployments configure it instead of silently falling back to generic wording. ## Claude Implementation Notes - backend/validators/views.py: issuance log drops request.user.address; user_id + code_id remain as identifiers - frontend/CLAUDE.md: add VITE_DECKARD_BOT_USERNAME to the Environment Variables section - frontend/.env.example: add VITE_DECKARD_BOT_USERNAME with a note that unset falls back to generic wording
The pg_dump step now hands the production password to libpq through the environment and passes a password-free URI on the command line, so the credential no longer appears in process listings while the dump runs. A restore that fails or is interrupted removes the staging container before the script exits, so a later --reuse-postgres resume can only ever see a completely restored copy. ## Claude Implementation Notes - backend/scripts/sync_prod_to_sqlite.py: dump_production splits userinfo off the URI (rpartition on @, partition on : — correct per RFC 3986, query params and encoding untouched), passes the password via bare `-e PGPASSWORD` with a subprocess env, and unquotes it for libpq; main() wraps restore() so any BaseException (incl. KeyboardInterrupt) triggers docker rm -f of the container before re-raising. Two CodeRabbit findings were skipped deliberately: `docker start` on a running container exits 0 (no already-running failure exists), and the --reuse-postgres existence check has only a millisecond TOCTOU window on a single-user laptop.
…up-bind-codes Let validators link Telegram support groups through the Deckard bot
The SQLite steps now pass an explicit empty DATABASE_URL instead of removing the variable. Removing it let settings reload the value from backend/.env, which pointed the migrate and destructive table-clearing steps at whatever database that file names. An empty value stays present, survives dotenv, and resolves to SQLite. The staging Postgres now binds to loopback only, since it holds unredacted production data behind a fixed password. The production snapshot moved into the gitignored backups/ directory so it can never be staged, and pg_dump writes to a .partial name that is renamed only on success, so --reuse-dump can never resume from an interrupted download. ## Claude Implementation Notes - backend/scripts/sync_prod_to_sqlite.py: manage() and load_into_sqlite() set DATABASE_URL='' instead of popping it (load_dotenv only fills ABSENT vars; verified on this machine that '' survives .env and settings picks SQLite); SNAPSHOT constant moved to BACKUP_DIR with a mkdir in export_snapshot() for --reuse-postgres runs; container publish changed to 127.0.0.1:5434 and LOCAL_DB_URL to 127.0.0.1 to skip localhost/::1 resolution; dump_production() writes tally_prod_*.sql.partial and renames after pg_dump succeeds, which latest_dump()'s *.sql glob never matches. - .claude/commands/sync-db.md: --keep-json flag doc now names backups/prod_snapshot.json.
The backups directory, which holds the unredacted SQL dumps and the intermediate JSON snapshot, is now forced to owner-only permissions every time the sync touches it, so a permissive umask or a pre-existing loose directory can no longer expose production data to other local users. The sync documentation now states that --keep-json retains a full production fixture that must be deleted manually. ## Claude Implementation Notes - backend/scripts/sync_prod_to_sqlite.py: ensure_backup_dir() helper does mkdir + chmod 0o700 (chmod unconditionally, since mkdir's mode is umask-masked and skipped for existing dirs); replaces the two bare BACKUP_DIR.mkdir sites in dump_production and export_snapshot. Directory perms cover every file inside, so CodeRabbit's per-file chmod and run-container-as-host-user suggestions were skipped as redundant. Also skipped: UUID dump names (concurrent runs are structurally unsupported -- shared container name/port/snapshot; the second run's docker rm -f kills the first mid-restore) and a generated PG_PASSWORD (loopback-only bind; any same-user process can read backups/*.sql directly, and an owner-only password file is readable by the same principal). - .claude/commands/sync-db.md: --keep-json flag now notes the snapshot is a full production fixture to delete manually.
Make syncing the production database to local development reliable
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.