fair-play: offline mirror-detection tooling - #802
Open
QuintenB-2261189 wants to merge 2 commits into
Open
Conversation
Read-only, offline fair-play analysis under scripts/fair-play/. Nothing runs on the server, touches the database, or can sanction an account; the durable label ceiling is `suspicious`, and `normal`/`proven_cheater` require a named human and a written reason. - mirror_scan.py: relay, linked-account and colour-swapped self-mirror shapes over canonical position hashes; multiplicity-corrected floor (default alpha 1e-3), permutation-null and epoch controls. Seat inference opt-in (--enable-seat), off by default; P1 gates on a strictly contiguous run; case files state when seat is off. - registry.py: durable record; human-only append-only per-finding `dismiss` and per-account `label` (trim + reject empty/`detector`); stable-code de-dup; legacy migration. - suspects.py: ranked queue sharing the attribution helpers (enforced by test), honours --enable-seat. book_follow.py: rating-banded explorer mark, never in the registry. engine_check.py: optional GPU second stage. pull_archive.py: keyset-paginated puller. 134 hermetic tests (stdlib only; faked engine; scrubbed real-shape fixture). No live account name in any tracked file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…limits Every validation number measured on the current code at the shipped default (alpha 1e-3: 21 pairs, floor 23.3 bits above the 20-bit benign ceiling; K=500 null 0.24 vs 21). States plainly that the null rejects random concurrency but does NOT measure precision or recall; documents the off-switches, keyset pagination, and the weak-by-schema account linkage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fair-play: offline mirror-detection tooling
What this adds
Offline, read-only fair-play tooling under
scripts/fair-play/. Nothing in this PRruns on the server, touches the database, or can sanction an account.
mirror_scan.py— finds pairs of games that traversed the same positions atthe same time: a rated game relayed into a concurrent bot game (the attack that
actually occurred), linked accounts (two bot games, two humans, one line), and
colour-swapped self-mirrors (a rating distortion, explicitly not treated as
cheating). Multiplicity-corrected evidence floor, permutation-null and
epoch-control built in (
--null-permutations,--epoch-control).registry.py— the durable record. Three labels; automation can only everreach
suspicious;normal/proven_cheaterrequire a named human and a writtenreason (enforced). Append-only log. Per-finding
dismiss. Idempotent acrossrescans.
suspects.py— the ranked review queue, sharing the same attribution helpersas the case files (by test, so the two entry points cannot drift apart).
book_follow.py— opening-explorer use measured as a consequence(rating-banded, leave-one-player-out book). A mark for a human; never enters the
registry.
engine_check.py— optional second stage against the engine's policy head;requires a local engine + eval server; documented as unusable as a screen (the
measured human/engine overlap is in the README).
stdlib-only at runtime except
pull_archive.py(requests, cbor2).What it found on the real archive
21 pairs across 2.3 years of games (at the shipped alpha=1e-3), 20 of which belong to
one already-removed operation. With the opt-in seat inference enabled it names the
removed account 15/15 without being told which account was removed; off by default it
withholds both names. Details are in
logs/output,which is gitignored — review files contain usernames and must never be committed.
No account names appear in this PR.
Safety posture
suspiciouslabel in a local JSONfile.
registry.py dismiss), everyaccount clearable (
label normal); the seat inference is off by default(
--enable-seatopts in) and the linked/self-mirror detectors take--disable linked/--disable self-mirror. All logged append-only.benefited, and state whom the evidence excludes.
0.24 ± 0.48 admissions per scan versus 21 observed; it rejects random concurrency as
the cause but does not bound overall precision or the false-positive rate — those
are unmeasured until human-reviewed labels accumulate.
Not in this PR
admin_fairplay.rspage is sketched but needscargo checkagainst the workspace; it should be its own PR.
(written at move time) would give per-move causality for future games. It is NOT a
redefinition of
game_hashes.played_at— that column already means "finish time",and historical per-move times are not retained anywhere. Its own schema PR.
🤖 Generated with Claude Code