Skip to content

James/fix hotkey ban - #16

Open
James-4u wants to merge 5 commits into
0xsigurd:devfrom
James-4u:james/fix-hotkey-ban
Open

James-4u wants to merge 5 commits into
0xsigurd:devfrom
James-4u:james/fix-hotkey-ban

Conversation

@James-4u

@James-4u James-4u commented May 18, 2026

Copy link
Copy Markdown

PR: Validator Fairness, Query Rotation & Log Hardening

Summary

This PR addresses four issues in neurons/validator.py related to miner selection fairness, query ordering, prompt leakage in logs, and round observability.


Changes

### 1. Random UID Selection Within Coldkey/IP Groups

_available_miner_uids()

Previously, after grouping miners by shared coldkey or axon IP via union-find, the validator always kept the minimum UID from each group as the sole representative. This permanently excluded all other hotkeys in the group from ever being queried.

The fix collects all UIDs per group and uses random.SystemRandom().choice() to pick one UID at random each round. Every hotkey in a group now rotates through the candidate pool, giving each a fair chance of being selected and scored.

[Rejected/Reverted]


2. Shuffle Miners Before Querying

run() loop

After miner selection, the UID list is now shuffled with self.system_random.shuffle() before being passed to the dendrite. This randomises the query order each round so no miner is consistently advantaged or disadvantaged by position.

miner_uids = self._select_random_miners(available_uids, seed=self._seed_from_block(block))
self.system_random.shuffle(miner_uids)  # added

3. Redact Challenge Prompt in All Log Output

generate_challenge() and run() loop

The challenge prompt was previously logged in plaintext at three points, allowing miners with access to W&B or validator logs to scrape the prompt and pre-process adversarial examples before responding.

All three log sites now emit "***" instead of the raw prompt value. The actual prompt is still passed correctly to API calls and miners via the synapse.

Affected log calls:

  • _log_summary("challenge_attempt", ..., prompt=chosen_prompt)prompt="***"
  • _log_step_start("challenge_fetch_image", prompt=chosen_prompt)prompt="***"
  • _log_summary("challenge_summary", ..., prompt=challenge.prompt)prompt="***"

4. Log the Round Winner's UID

run() loop — loop_summary

Added best_uid to the per-round summary log, identifying the UID of the miner with the highest score in that round. This makes it easy to track top performers and debug scoring from logs.

best_uid = results_by_uid[rewards.index(max_score)][0] if rewards else None
self._log_summary(
    "loop_summary",
    ...
    max_score=f"{max_score:.4f}",
    best_uid=best_uid,   # added
    ...
)

Example log output:

loop_summary block=4210 selected=8 success=3/8 avg_score=0.1843 min_score=0.0000 max_score=0.7612 best_uid=42 ...

Files Changed

  • neurons/validator.py

Closes #15

@James-4u

Copy link
Copy Markdown
Author

@0xsigurd feel free to take a look - reverted first commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant