Conversation
…irness" This reverts commit 6d5e320.
Revert "feat: select random ui from the same coldkey, ip group for fa…
Author
|
@0xsigurd feel free to take a look - reverted first commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: Validator Fairness, Query Rotation & Log Hardening
Summary
This PR addresses four issues in
neurons/validator.pyrelated to miner selection fairness, query ordering, prompt leakage in logs, and round observability.Changes
### 1. Random UID Selection Within Coldkey/IP Groups_available_miner_uids()Previously, after grouping miners by shared coldkey or axon IP via union-find, the validator always kept the minimum UID from each group as the sole representative. This permanently excluded all other hotkeys in the group from ever being queried.The fix collects all UIDs per group and usesrandom.SystemRandom().choice()to pick one UID at random each round. Every hotkey in a group now rotates through the candidate pool, giving each a fair chance of being selected and scored.[Rejected/Reverted]
2. Shuffle Miners Before Querying
run()loopAfter miner selection, the UID list is now shuffled with
self.system_random.shuffle()before being passed to the dendrite. This randomises the query order each round so no miner is consistently advantaged or disadvantaged by position.3. Redact Challenge Prompt in All Log Output
generate_challenge()andrun()loopThe challenge prompt was previously logged in plaintext at three points, allowing miners with access to W&B or validator logs to scrape the prompt and pre-process adversarial examples before responding.
All three log sites now emit
"***"instead of the raw prompt value. The actual prompt is still passed correctly to API calls and miners via the synapse.Affected log calls:
_log_summary("challenge_attempt", ..., prompt=chosen_prompt)→prompt="***"_log_step_start("challenge_fetch_image", prompt=chosen_prompt)→prompt="***"_log_summary("challenge_summary", ..., prompt=challenge.prompt)→prompt="***"4. Log the Round Winner's UID
run()loop —loop_summaryAdded
best_uidto the per-round summary log, identifying the UID of the miner with the highest score in that round. This makes it easy to track top performers and debug scoring from logs.Example log output:
Files Changed
neurons/validator.pyCloses #15