Skip to content

fair-play: offline mirror-detection tooling - #802

Open
QuintenB-2261189 wants to merge 2 commits into
hiveboardgame:mainfrom
QuintenB-2261189:fair-play/mirror-detector
Open

fair-play: offline mirror-detection tooling#802
QuintenB-2261189 wants to merge 2 commits into
hiveboardgame:mainfrom
QuintenB-2261189:fair-play/mirror-detector

Conversation

@QuintenB-2261189

Copy link
Copy Markdown

fair-play: offline mirror-detection tooling

What this adds

Offline, read-only fair-play tooling under scripts/fair-play/. Nothing in this PR
runs on the server, touches the database, or can sanction an account.

  • mirror_scan.py — finds pairs of games that traversed the same positions at
    the same time: a rated game relayed into a concurrent bot game (the attack that
    actually occurred), linked accounts (two bot games, two humans, one line), and
    colour-swapped self-mirrors (a rating distortion, explicitly not treated as
    cheating). Multiplicity-corrected evidence floor, permutation-null and
    epoch-control built in (--null-permutations, --epoch-control).
  • registry.py — the durable record. Three labels; automation can only ever
    reach suspicious; normal/proven_cheater require a named human and a written
    reason (enforced). Append-only log. Per-finding dismiss. Idempotent across
    rescans.
  • suspects.py — the ranked review queue, sharing the same attribution helpers
    as the case files (by test, so the two entry points cannot drift apart).
  • book_follow.py — opening-explorer use measured as a consequence
    (rating-banded, leave-one-player-out book). A mark for a human; never enters the
    registry.
  • engine_check.py — optional second stage against the engine's policy head;
    requires a local engine + eval server; documented as unusable as a screen (the
    measured human/engine overlap is in the README).
  • 134 hermetic tests (no network/GPU/archive; scrubbed real-shape fixture),
    stdlib-only at runtime except pull_archive.py (requests, cbor2).

What it found on the real archive

21 pairs across 2.3 years of games (at the shipped alpha=1e-3), 20 of which belong to
one already-removed operation. With the opt-in seat inference enabled it names the
removed account 15/15 without being told which account was removed; off by default it
withholds both names. Details are in logs/ output,
which is gitignored — review files contain usernames and must never be committed.
No account names appear in this PR.

Safety posture

  • No auto-actions, structurally: the ceiling is a suspicious label in a local JSON
    file.
  • Every finding is dismissible by a named human (registry.py dismiss), every
    account clearable (label normal); the seat inference is off by default
    (--enable-seat opts in) and the linked/self-mirror detectors take
    --disable linked / --disable self-mirror. All logged append-only.
  • Case files withhold names wherever the analysis cannot say which of two players
    benefited, and state whom the evidence excludes.
  • The K=500 permutation control estimates the random-concurrency baseline at
    0.24 ± 0.48 admissions per scan versus 21 observed; it rejects random concurrency as
    the cause but does not bound overall precision or the false-positive rate — those
    are unmeasured until human-reviewed labels accumulate.

Not in this PR

  • Any UI. A read-only admin_fairplay.rs page is sketched but needs cargo check
    against the workspace; it should be its own PR.
  • The server-side change this work motivates: a new nullable per-move timestamp field
    (written at move time) would give per-move causality for future games. It is NOT a
    redefinition of game_hashes.played_at — that column already means "finish time",
    and historical per-move times are not retained anywhere. Its own schema PR.

🤖 Generated with Claude Code

QuintenB-2261189 and others added 2 commits August 10, 2026 21:26
Read-only, offline fair-play analysis under scripts/fair-play/. Nothing runs on the
server, touches the database, or can sanction an account; the durable label ceiling is
`suspicious`, and `normal`/`proven_cheater` require a named human and a written reason.

- mirror_scan.py: relay, linked-account and colour-swapped self-mirror shapes over
  canonical position hashes; multiplicity-corrected floor (default alpha 1e-3),
  permutation-null and epoch controls. Seat inference opt-in (--enable-seat), off by
  default; P1 gates on a strictly contiguous run; case files state when seat is off.
- registry.py: durable record; human-only append-only per-finding `dismiss` and
  per-account `label` (trim + reject empty/`detector`); stable-code de-dup; legacy
  migration.
- suspects.py: ranked queue sharing the attribution helpers (enforced by test), honours
  --enable-seat. book_follow.py: rating-banded explorer mark, never in the registry.
  engine_check.py: optional GPU second stage. pull_archive.py: keyset-paginated puller.

134 hermetic tests (stdlib only; faked engine; scrubbed real-shape fixture). No live
account name in any tracked file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…limits

Every validation number measured on the current code at the shipped default (alpha 1e-3:
21 pairs, floor 23.3 bits above the 20-bit benign ceiling; K=500 null 0.24 vs 21). States
plainly that the null rejects random concurrency but does NOT measure precision or recall;
documents the off-switches, keyset pagination, and the weak-by-schema account linkage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant