A GitHub Actions job runs daily, pulls hackathons / game jams / CTFs from ~10 sources, dedupes them, and publishes:
| Output | What for |
|---|---|
index.html (GitHub Pages) |
browse; grouped In Kraków → Near Kraków → Poland → Expected → Online |
krakow-hackathons.ics |
subscribe once in Google/Apple/Outlook calendar; events appear on their own |
feed.xml |
RSS of newly discovered events |
events.json, health.json |
machine-readable; per-source health |
| Telegram digest (optional) | push message when something new is found |
Krakow has only ~25–35 public hackathons a year, and global platforms (Devpost, Luma, MLH…) list almost none of them. So the feed is layered, and each layer covers for the others:
- Structured sources (Crossweb, Devpost, Luma, MLH, Devfolio, CTFtime, GDG, Meetup iCal, JSON-LD pages).
- Series watchlist (
data/series.yml): ~22 annual events (HackYeah, BITEhack, Hacknarök, Kościuszkon, Global Game Jam…). Shown as Expected with a month window until real dates appear anywhere, then upgraded automatically. Their own sites are scanned for schema.org dates. - Web-search discovery: ~9 queries + a targeted query for every series about to happen → fetch → JSON-LD, else LLM extraction that must quote the date sentence verbatim from the page (otherwise dropped).
- Manual file (
data/manual.yml) for Facebook/Discord-only events: 5 lines per event.
- Every source is isolated; one failing never breaks the build.
- Events that vanish from sources are kept for 14 days (flaky parsers, paging).
- Plain HTTP blocked by Cloudflare → transparently retried with Chrome TLS impersonation (
curl_cffi) — works from a home IP. - Crossweb: live HTML → live RSS → last snapshot; if the HTML parser matches 0 rows it raises a flag (markup changed).
data/health.jsontracks each source's item-count history; a source returning 0 / <25% of its median for 3 runs turnsdegraded, and the workflow opens (and later auto-closes) a GitHub issue "Hackathon sources need attention".- Committing
data/every run stops GitHub from disabling the schedule after 60 inactive days.
GitHub's datacenter IPs are blocked by several things we'd like to use, so:
| Piece | Runs in the cloud? | Notes |
|---|---|---|
| Devpost, Luma, MLH, Devfolio, CTFtime, GDG, Meetup iCal, JSON-LD pages, series, manual | ✅ fully automatic | zero upkeep |
| Crossweb (biggest Polish source) | cloud route: with TAVILY_API_KEY, Crossweb-restricted searches are read through Tavily's crawler. The repo ships a one-off snapshot (data/crossweb_snapshot.json, 3 Oct 2026) as a fallback; it goes stale after ~8 days and the build then opens an issue until you add the key. (Optional alternative: scripts/install_task.ps1 refreshes the snapshot from a home PC.) |
|
| Web-search discovery | needs the free TAVILY_API_KEY secret (1000 searches/mo, no card; the config uses ~15/day). Without it the build flags it in the health table / issue. |
|
| LLM date extraction | best effort | uses LLM_API_KEY if set, else tries GitHub's free Models endpoint with the built-in token (untested; failure is non-fatal) |
- Repo is public (unlimited free Actions + Pages). Settings → Pages → Source: GitHub Actions.
- Actions → update → Run workflow (then it runs daily at 05:17 UTC).
- Needed for full coverage: add secret
TAVILY_API_KEY(tavily.com, free) — this single key powers web discovery and the cloud route to Crossweb. Add an LLM key too (below) so discovered pages can be dated. - Optional:
LLM_API_KEY(+LLM_BASE_URL, variableLLM_MODEL; e.g. a free Gemini key),TELEGRAM_BOT_TOKEN+TELEGRAM_CHAT_IDfor push digests.
- Just open the page / subscribe to the
.ics. Nothing to do. - Spotted something only on Facebook? Add a block to
data/manual.yml. - New recurring event? Add a block to
data/series.yml. New Meetup group? Add its iCal URL toconfig.yml → sources.ics.feeds.
pip install -r requirements.txt
python -m krkhack --out site --no-notify # build
python -m pytest -q # tests
Research notes that led to these choices: research/ (platforms, organizers, Polish sources).