Festival person-link reimport rollout (waves) — execute the #873 repair

[holden]

Executes the reimport repair from #873 (the empty-payload festival person-links). Tracked separately so we can run it in monitored waves and log progress. Parent context: #873 (root cause + diagnosis), #1101 §3 (the code fix), #1090 Phase 3a (surface-area).

Why this exists

#873's gap is ~100% empty-payload nominations (details: {nominee_names: nil, person_imdb_ids: []}) written before the scraper↔worker shape fix. The resolver recovers 0 from them — they need re-import (re-scrape → repopulate details → resolve). The code fix (#1101 WS3: create_nomination now always patches empty details even when person resolution misses) is deployed and validated. This issue drives the re-imports.

✅ Validated — Wave 0 (2026-06-09)

Golden Globes 2016 / 2017 / 2018 re-imported → HFPA resolved 179 → 277 (+98), empty-payload 2,785 → 2,682. (2018 hit a transient Crawlbase HTTP 520, recovered on retry.) Confirms the path: re-scrape repopulates details + resolves person_ids (incl. the Christian Slater Golden Globe links that spawned #873).

Full scope — empty-payload person-tracked nominations by org

Org	source_key	empty noms	years
Golden Globes (HFPA)	golden_globes	2,682	80
BAFTA	tbd	1,786	74
Critics Choice (CCA)	tbd	1,165	30
SAG	tbd	689	28
NYFF	tbd	603	52
Cannes (CFF)	tbd	385	73
Venice (VIFF)	tbd	259	79
Berlin (BIFF)	tbd	94	53
Locarno (LFF)	tbd	69	31
Sundance (SFF)	tbd	35	…

(~7.7k+ total — the remainder of the #873 ~10k after Wave 0.)

Mechanism

• Enqueue Cinegraph.Workers.UnifiedFestivalWorker per (festival_key, year) (scraping queue, max_attempts: 3 → transient Crawlbase 520s retry; 404/403 cancel cleanly). On the eval node, insert oban_jobs rows directly (Oban is stripped there); the web node runs them.
• Each year re-scrapes IMDb via Crawlbase (JS mode — 403/520-prone), upserts ceremony.data, repopulates empty details, and resolves person_ids.
• After each wave: re-run Cinegraph.Maintenance.ResolvePersons.run if needed, then Cinegraph.Health.AwardPersonLinkage.audit(org:) to measure the delta.

Wave plan

1. Golden Globes (validated org) by year-batches: Wave 0 ✅ (2016–18) · Wave 1 (2001–2015) · then 1944–2000.
2. BAFTA → CCA → SAG → NYFF → Cannes → Venice → Berlin → Locarno → Sundance → … (resolve each org's source_key at wave time).

• Batch size ~15 years/wave to monitor Crawlbase success rate + cost before committing the next.

Closure

Closes the reimport portion of #873 when: every org's empty-payload nominations are repopulated, the resolver has run, and person_required_nomination_missing_person drift → green. #873 stays open until then.

Progress log

• Wave 0 ✅ 2026-06-09 — GG 2016/2017/2018 → +98 resolved.

[holden]

Wave 1 enqueued (2026-06-09): Golden Globes 2001–2015 (15 years) on the scraping queue. Draining now with retries for transient Crawlbase 520s. Will re-audit HFPA after and log the delta, then continue with GG 1944–2000.

[holden]

Progress (2026-06-09 eve):

• Wave 1 result ✅ — GG 2001–2015 all 15 jobs completed; HFPA resolved 277 → 774 (+497), empty-payload 2,682 → 2,159. Path scales cleanly.
• Wave 2 enqueued — GG 1944–2000 (57 years) → Golden Globes now fully queued (all 80 years).
• Wave 3 enqueued — BAFTA (74) · Cannes (73) · Venice (79) · Berlin (53) = 279 years (the scraper's documented core festival orgs).
• ~336 scrape jobs draining on :scraping (concurrency 3) — ~2h with Crawlbase 520 retries.

Remaining orgs (uncertain IMDb scrapeability — will validate one year each before bulk-enqueuing): CCA (1,165), SAG (689), NYFF (603), Locarno (69), Sundance (35), Telluride (31), SXSW (18), Toronto (2).

[holden]

Wave 4 enqueued — rollout fully queued (2026-06-09 night):

• Wave 4: CCA (30) · SAG (28) · NYFF (52) · Locarno (31) · Sundance (29) · Telluride (9) · SXSW (9) · Toronto (2) = 190 jobs.
• All 13 orgs now queued (~544 year-jobs total). These orgs all have working scraper configs (they were imported before), so re-import should behave like Golden Globes; the worker cancels cleanly on any 404/403.
• Drain progress: person-link % 20.14% → 38.1% and climbing; ~410 jobs remaining on :scraping (~5h at ~1.4/min with Crawlbase retries).

Next: re-audit in ~2h for per-org deltas; final audit at drain completion. #873 closes when person_required_nomination_missing_person drift → green.

[holden]

Bulk rollout drained (2026-06-10): all 544 jobs ran — 529 completed, 15 discarded (Crawlbase 520×3). Festival person-link 20.14% → 65.3%, empty-payload 7,888 → 816. Re-enqueued the 15 discarded years (Crawlbase load eased). Remaining ~800 empties shrink as those re-run + the daily FestivalPersonResolverSweeper (2k/day) resolves repopulated-but-unlinked noms; a residual is irreducible (IMDb has no nominee data), so it asymptotes below 100%. Per-org tail: HFPA 466 · CCA 168 · NYFF 82 · BAFTA 64 · rest <20.