Skip to content

Gate every node reputation penalty behind REPUTATION_PENALTY_SCALE (default 0) - #507

Merged
jehanazad merged 6 commits into
mainfrom
feat/reputation-penalty-scale
Sep 19, 2026
Merged

jehanazad merged 6 commits into
mainfrom
feat/reputation-penalty-scale

Conversation

@jehanazad

Copy link
Copy Markdown
Contributor

Why

On the test droplet the mirrored real node ndebvzgeoij5t2l was received but discarded: reputation 0.0, blocked: true, 100 stored penalties of 0.15 reading Trust score critically low: 0.000, one per minute, from a trust entry with one claim residual. evaluate_reputations() acted on any node with at least one sample, one out-of-threshold sample scores 0.0, evaluate_trust charges 0.15 per 60 s pass, so 1.0 crossed the 0.2 block threshold in six minutes. Once blocked, record_detection_frame drops every frame, apply_reward is a no-op, and state_snapshot.py persists the block across restarts. The backend's own trust reader (node_bias.get_node_trust) already used a 0.5 prior below 3 samples; the evaluator bypassed it.

What

Pairs with retina-analytics #34 (merge that first; the pin here is its head).

  • REPUTATION_PENALTY_SCALE, default 0: one switch every reputation penalty routes through (trust 0.15/0.05, heartbeat 0.1, detection rate 0.05, neighbour inconsistency 0.08, ADS-B cross-validation 0.1). At 0 nothing is recorded and no node can be blocked; rewards untouched; trust still computed and published. Set from the environment in core/state.py before the snapshot restore and the first evaluator pass; logged at startup. Documented in .env.example, the runbook and the architecture gate table as a temporary stance while the trust input is a single residual.
  • Min-sample bar shared: node_bias now imports TRUST_MIN_SAMPLES from retina-analytics, where the evaluator applies the same bar, so a single residual cannot start a block even when penalties are turned back on.
  • backend/scripts/unblock_nodes.py: stdlib-only, edits the schema-2 snapshot (checksum verified and rewritten atomically) to reset selected or all blocked entries to reputation 1.0 / unblocked / no penalties. A script rather than a route: there is no unblock endpoint today and adding one is a separate decision. Procedure in the runbook (server stopped while editing).
  • Tests: penalties_on fixture for the tests that assert a penalty lands; new tests for the switch, the parser, restored blocks staying as restored, and the script.

Full backend suite: 4195 passed, 1 skipped, 2 failed — both pre-existing on main in untouched files (test_no_real_identities, test_health_returns_ok). Pre-commit gate green.

Deployed to the test droplet and verified there (details in the session report).

🤖 Generated with Claude Code

jehanazad and others added 3 commits September 18, 2026 23:50
…efault 0

A real mirrored node on the test droplet (node_ref ndebvzgeoij5t2l, node_id
retce36dbb4) was permanently blocked off ONE trust sample.  The chain:
NodeAnalyticsManager.evaluate_reputations() acted on any node with at least
one sample, a single out-of-threshold sample scores 0.0, NodeReputation
.evaluate_trust charges 0.15 for that, and the evaluator runs every
REPUTATION_INTERVAL_S = 60 s — so 1.0 crosses the 0.2 block threshold in six
minutes.  Once blocked, record_detection_frame returns False and every
mirrored frame is dropped, apply_reward is a no-op so the node cannot climb
back, and services/state_snapshot.py persists the block, so a restart brings
it straight back.  There is no admin unblock route, and NodeReputation
.unblock() only resets to 0.3 — one penalty above re-blocking.

Operator decision: for now, trust must never lower a node's reputation.  One
switch, default off, and a temporary stance rather than a change of intent —
the only trust input today is a single claim residual from the identity-first
lane, which is not enough evidence to act on.  Trust is still computed and
still reported; rewards are untouched.

REPUTATION_PENALTY_SCALE (config/constants.py, default 0) multiplies every
reputation penalty in the estate: trust 0.15/0.05, stale heartbeat 0.1, high
detection rate 0.05, neighbour inconsistency 0.08, and the ADS-B
cross-validation 0.1 charged from services/tasks/periodic.py.  0 records
nothing at all, so no node can be blocked by any of those paths; 1 restores
the historical behaviour.  A negative or non-finite value logs a warning and
reads as 0 — an unparseable gate must not read as "penalties on".  core/state
.py pushes it into the library at import, before restore_snapshot() rebuilds
the reputations and before the evaluator's first pass, and logs it at INFO so
a deploy log says which way it went.

The scaling itself and the evaluator's new min-sample bar live in
retina-analytics (separate submodule commit; the pin bump follows).
services/node_bias.py now imports TRUST_MIN_SAMPLES from there instead of
keeping its own literal 3, so the solver's reading of a young node's trust and
the evaluator's willingness to act on it cannot drift apart.

The switch deliberately does not unblock anything a snapshot already carries —
that would hide from the operator which nodes had been blocked and why.
backend/scripts/unblock_nodes.py does that instead: stdlib-only, operating on
the schema-2 snapshot envelope with the server stopped.  A script rather than a
route because there is no unblock endpoint today and adding one is a separate
decision (it would need an authz story and an audit trail, and this is a
one-off cleanup after a bug, not an operation the product needs).

Tests: penalties_on fixture in tests/conftest.py, applied to exactly the tests
that assert a penalty lands, so the suite's ambient configuration is the
deployed one.  New coverage for the default recording nothing on a 100 km
mismatch, the env parsing, that a restored block survives an evaluator pass
untouched, and the unblock script's round trip through a verifying snapshot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Picks up retina-analytics PR #34: NodeReputation.penalty_scale +
set_penalty_scale(), which core/state.py now sets from
REPUTATION_PENALTY_SCALE, and TRUST_MIN_SAMPLES, which node_bias imports.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
id_for_ref answers from the registry, and a mirrored node has no row
there: on the test droplet it printed None for ndebvzgeoij5t2l. The key
was found by hand (the one ret* id among the synth-* blocked entries,
whose single trust sample reproduces the published rms_delay_error_us).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@claude

This comment has been minimized.

core/state.py runs at import, before main.py's logging.basicConfig, so
its INFO line never reached the container log on the test droplet. The
scale is still set at import (it must precede the snapshot restore); the
line now prints from the lifespan, just before restore_snapshot().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@claude

This comment has been minimized.

jehanazad and others added 2 commits September 19, 2026 00:19
test_no_real_identities caught retce36dbb4 in the docstring, the argparse
epilog and the test fixture. This repo is public; a placeholder says the
same thing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Unrelated to the penalty scale; here so this PR's CI can go green.

- test_health_returns_ok: #506 added synthetic_fleet to /api/health
  (routes/health.py) and the test still expected {"status": "ok"} alone.
  Assert the contract instead: status ok, synthetic_fleet a bool, no
  other keys.
- test_no_real_identities: #506's mapShell.test.tsx rendered
  /nodes/<real node id>; any path segment does for that assertion.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@claude

claude Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Claude finished @jehanazad's task in 1m 25s —— View job


Code Review: Gate node reputation penalties behind REPUTATION_PENALTY_SCALE

Todos

  • Read CLAUDE.md / repo conventions
  • Review diff against origin/main (PR synchronized with 2 new commits: b5bf9d9, a205806)
  • Read key changed files
  • Read new/changed tests
  • Post review feedback

Re-reviewed at the current head. The two commits added since the last pass (b5bf9d9, a205806) are both good: b5bf9d9 replaces the real node id retce36dbb4 with placeholders in unblock_nodes.py's docstring/epilog and in test_unblock_nodes.py's fixtures, and a205806 fixes the two pre-existing-on-main test failures (test_health_returns_ok, test_no_real_identities) unrelated to this PR's core change, so CI can go green — I checked routes/health.py and confirmed the new assertion (status + boolean synthetic_fleet, no other keys) matches the endpoint's actual contract exactly.

New finding

  • backend/config/constants.py:500 still has the real node_ref from the incident, ndebvzgeoij5t2l, inline in the REPUTATION_PENALTY_SCALE comment: # (node_ref ndebvzgeoij5t2l) on the test droplet from ONE sample — once. b5bf9d9's own commit message says the point was to keep the real node id out of the tree because "this repo is public" — but that pass only covered unblock_nodes.py's ret<hex> node_id shape (retce36dbb4); this is a node_ref literal in a different file, and it's still there. It isn't caught by test_no_real_identities.py either: that test's REAL regex (ret[0-9a-f]{8}|...retnode...) only matches the node_id/hostname shapes, not the node_ref shape, so this specific leak has no automated gate. Since the whole point of the sibling fix was to stop shipping this incident's real identifier in a public repo, this occurrence should get the same placeholder treatment (e.g. <node_ref>, matching the style already used for --node <node_id> in the same script). Fix this →

Still open from the prior review pass (minor, non-blocking)

  • backend/scripts/unblock_nodes.py:230-233 — the --node path in main() still does entry = reps[node_id] then _reset(entry) without an isinstance(entry, dict) check, unlike the --all-blocked path which does check it. Only fires against an already-corrupt snapshot entry, so low severity, but worth a one-line guard for a consistent operator-facing error.

Not verified (environment limitation)

backend/.venv and libs/retina-analytics aren't checked out in this sandbox, so I couldn't run the backend suite or pre-commit; read the diff and tests statically. The PR description's claim (4195 passed / 1 skipped / 2 pre-existing failures, now presumably 0 after a205806) is consistent with what I read, but unconfirmed independently.

@jehanazad
jehanazad merged commit 61e2f18 into main Sep 19, 2026
15 checks passed
@Babissimo
Babissimo deleted the feat/reputation-penalty-scale branch September 21, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant