Skip to content

Multi-proxy support (Redis + shared MySQL) and cache/SQL race fixes - #1

Merged
Macaron27 merged 5 commits into
mainfrom
multi-proxy
Sep 28, 2026
Merged

Macaron27 merged 5 commits into
mainfrom
multi-proxy

Conversation

@Macaron27

Copy link
Copy Markdown
Owner

What

  • Multi-proxy networks: set storage.type: mysql on every proxy, then redis.enabled: true with a unique redis.proxy-id on each. Requests, accept/deny, join/leave messages, status, current server, /f list and tab completion then work across proxies.
  • Single proxy: unchanged. Network.LOCAL is a no-op.

How

  • One code path: every cross-proxy change is an Event (a sealed interface of records). The proxy where it happens applies it, then publishes it; every other proxy applies the same event in Friends.apply.
  • RedisNetwork (Jedis 8):
    • presence hash of who is online where;
    • one key per pending request, with a TTL;
    • a heartbeat per proxy, so a dead proxy's players are shown offline after about 30–40 s;
    • pub/sub fan-out, with a resync after every reconnect;
    • a SET NX claim so crossing accepts on two proxies can't both go through.
  • Publish order: events go out through the SQL queue, so other proxies only see a change once it is stored.

Fixes found by the new tests

  • Cache vs database drift under concurrency: accept racing remove, and two concurrent /f best toggles. Fixed with one lock around decide, cache update and queued write.
  • Stale profile load: a friendship change while a player's profile was still loading was missed.
  • Two cross-proxy ordering races: see .claude/known_errors.md.

Tests

86 tests pass locally (SQLite, MySQL 8.0.46, Redis 8.10.2):

  • an in-memory two/three-proxy hub;
  • randomized cache-vs-database checks with a deliberately slow database;
  • a concurrency stress test;
  • real-Redis tests.

Each race guard has a test that fails when the guard is removed. The exception is the lock: its random stress test only fails occasionally without it.

Also tested with two real Velocity 4.2.0 proxies sharing MySQL and Redis: logins on both proxies, correct events, a kill -9 on one proxy reaped by the other after about 40 s, and clean shutdown.

Not covered: gameplay through a real client on a backend server (the local test client can log in but not play).

🤖 Generated with Claude Code

Macaron27 and others added 5 commits September 28, 2026 18:11
Multi-proxy
- Every cross-proxy change is an Event (sealed records) applied by Friends.apply
  on the proxy where it happens and on every other proxy; Network.LOCAL keeps
  single-proxy setups unchanged.
- RedisNetwork (Jedis 8): presence hash, one TTL key per request, per-proxy
  heartbeat with reaping of dead proxies, pub/sub fan-out, resync after
  reconnects, SET NX claim for crossing accepts.
- Events are published through the SQL queue, so other proxies only see a
  change once it is stored.
- Velocity: redis config section (requires MySQL), server switches announced,
  shutdown flushes writes then leaves the network.

Correctness and optimization
- One lock around decide + cache update + queued write: fixes cache/database
  drift found by the new randomized and concurrent consistency tests.
- Re-read a profile whose friendships changed while it was loading.
- HikariCP-recommended MySQL statement/metadata caching.
- Sorted, network-wide tab completion (hides players appearing offline).

Tests: 86 total (SQLite, MySQL, real Redis, in-memory multi-proxy hub,
randomized cache-vs-database checks with a lagging database, concurrency).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Macaron27
Macaron27 merged commit c04171b into main Sep 28, 2026
1 check passed
@Macaron27
Macaron27 deleted the multi-proxy branch September 28, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant