A deterministic rollback-vs-naive netcode desync benchmark — proof that AI's localhost-only netcode desyncs the instant the network is real — with an interactive in-browser demo.
· ▶ Live demo · code MIT · data CC BY 4.0
Results at a glance (3-scenario slice) — the
rollbackstrategy converges in 100% of scenarios,naivein 0%, and every verdict is a simulation run under latency + packet loss. Drag the sliders in the live demo to watch the naive client drift off the server in real time.
Networking is where AI fails most invisibly: it produces multiplayer code with no client-side prediction or reconciliation, gets the authority model wrong, and it all works on localhost — then desyncs the moment there's latency or packet loss. "Localhost lies to you." A file-level assistant can't see this; you only catch it by running the code under adverse network conditions.
netfox-rollback-lab turns that into a machine-graded benchmark. A deterministic two-peer simulation runs two strategies under injected latency + packet loss:
- rollback — reliable input + client-side prediction + reconciliation to the authoritative snapshot (the technique Netfox brings to Godot),
- naive — unreliable input, no reconciliation (what an LLM tends to emit),
and measures the max divergence between the client's displayed state and the server's authoritative state. Rollback converges; naive desyncs.
Why this is hard to fake: the verdict comes from running the netcode under packet loss and measuring client-vs-server divergence per tick — exactly the failure that's invisible on localhost. Rollback's divergence returns to 0 each snapshot; naive's grows and never recovers.
Engine-agnostic by design: the rollback technique is implemented as a deterministic simulator so it's verifiable in CI and playable in the browser without an engine. The same approach is what Netfox implements for Godot 4; a Godot web-export demo is on the roadmap.
rollbacklab (pure stdlib)
sim ── deterministic 2-peer netcode sim (latency + seeded packet loss)
verifier ── convergence (max client-vs-server divergence within tolerance?)
runner ── grade a strategy on a scenario
score ── aggregate -> results/results.json (+ per-tick divergence traces)
export ── AI-training data (records + rollback/naive preference pairs)
dashboard (Next.js)
INTERACTIVE in-browser sim with latency + loss sliders — drag them and watch the
naive client desync from the server in real time while rollback stays locked —
plus per-scenario divergence charts from the Python run.
| Scenario | Conditions |
|---|---|
01-combined |
6-tick latency + 20% loss |
02-packet-loss |
2-tick latency + 30% loss |
03-high-latency |
10-tick latency + 15% loss, long session |
In every scenario the rollback strategy converges (max divergence ≤ tolerance) and the naive strategy desyncs.
make build # worker image (python)
make run # simulate every scenario -> results/results.json (rollback converges, naive desyncs)
make test # sim unit tests + invariants
make export # dataset/{records,preference_pairs}.jsonl
make dashboard # static site: interactive sim + divergence chartsPure Python + JSON to grade; the dashboard re-implements the same deterministic sim in JavaScript for the interactive demo. No game engine required.
- Proven, not asserted. Every verdict is a simulation run; the divergence trace is the evidence.
- Deterministic. Fixed input pattern + seeded loss → reproducible runs (the dashboard JS sim matches the Python one).
- Contrastive. Each scenario pairs rollback vs naive — clean chosen/rejected training pairs about authority + reconciliation.
No credentials are needed or committed; the harness runs a pure simulation (no real networking, no code execution of inputs). See SECURITY.md.