Add codebase review plans: simplification, LOC reduction, and bug fixes

[1aeo]

Summary

Thorough review of the codebase (all allium/lib/ modules, allium/allium.py, templates, and tests), delivered as three separate execution plans under docs/development/plans/:

• simplification-plan.md — 9 phases reducing architectural complexity (single escaping strategy, one error-handling layer, one API registry, progress-stack collapse, Relays facade removal, page-writer unification, utils consolidation, template dedup, de-layering), each gated on a byte-identical output diff.
• loc-reduction-plan.md — 6 batches of mechanical deletion (~1,200–1,400 production Python lines + ~500–650 template lines), every symbol grep-verified as dead or verbatim-duplicated before removal.
• bug-fix-plan.md — 9 phases fixing verified and suspected bugs, ordered by risk and expected HTML impact.

Explicitly out of scope: the consensus-health pipeline (fetch_consensus_health, get_consensus_health_data, the consensus_health_data attach, and the reader stub in relays.py) and the fetch_collector_data wrapper are kept. The simplification plan folds consensus-health into the unified API registry as a disabled/optional entry instead of deleting it.

Recorded decisions (options weighed in the bug-fix plan):

• Escaping (Phase 1): keep escaping logic in Python for render performance; wrap pre-escaped strings in markupsafe.Markup so autoescape stops double-escaping them while remaining the safety net for raw fields. No |safe in templates.
• Downtime alerts (Phase 3): exclusive role buckets using the codebase-standard Exit > Guard > Middle priority; total_offline computed in Python, template arithmetic removed (Jinja2 is slower — templates render precomputed values).
• v3 migration percentage (Phase 5): Option C — two explicitly-named fields: v3_pct_of_total (leaderboards, tier, adopter flag, contact sentence) and v3_migration_progress_pct (mixed-migration pill); the ambiguous v3_relay_percentage is deleted so every consumer picks semantics explicitly.
• Leaderboard ties (Phase 6): break by metric, then relay count (more relays wins), then operator key purely for byte-stable determinism.
• Progress steps (Phase 6): derive total_steps dynamically from the API worker registry and site_generator page definitions instead of hard-coded literals.
• Veteran scoring: datetime.now(timezone.utc) + parse_onionoo_timestamp() for host-timezone-independent output.
• IPv6 authority parsing: bracket-aware parsing via ip_utils.safe_parse_ip_address, preferring the IPv4 entry for latency probes (latent bug — zero expected output change).
• burst-limit: format as a data volume (bytes), not a rate — per proposal 328/dir-spec, BandwidthBurst is a token-bucket size, not bytes/sec.
• Rare-country loop (Phase 9): iterate the union of the GEOPOLITICAL_CLASSIFICATIONS sets so zero-relay countries are scored as intended; diff on rare-country surfaces reviewed rather than assumed noise-floor.

Every plan mandates the output comparison workflow: generate a full-API baseline (python3 allium/allium.py --out allium/www_baseline --apis all --progress) before changes, regenerate after, and diff with compare_outputs.py.

Key verified findings baked into the plans

• Double-escaped HTML entities in production output — autoescape=True plus Python pre-escaped *_escaped fields double-escapes; verified in generated output (misc/all.html: 1,275 occurrences; index.html: 71; top500.html: 61 — e.g. <admin AT my-mail dot rocks> renders as literal <...> text).
• HTTP 304/error cache fallback can never find the cache — handle_http_errors receives display names ("onionoo details") but cache files are keyed onionoo_details.json.
• Contact-page downtime alerts triple-count Guard+Exit relays; reliability leaderboard drops 0%-uptime relays from averages; inconsistent v3 migration denominators between contact pages and leaderboards; ~130 unreachable contact-branch lines in page_writer.py; and more.

Workflow verification (in this environment)

• ✅ Full --apis all baseline run completes (~3.5 min, ~7 GB RSS, 61/61 steps)
• ✅ Second full run + python3 compare_outputs.py --quiet — 28,339 common files; measured back-to-back noise floor of ~25 content-diff files (relay uptime ticking, root timestamps), now documented as the acceptance gate in all three plans

  

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9169004c-c906-4b43-b939-7cb9cecd0104

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/review-plans-simplify-loc-bugs-e387

---

_{Comment @coderabbitai help to get the list of available commands.}