Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/library.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ the following fields:
|-------|------|-------------|
| `user_login` | `str` | GitHub username that was scored |
| `context_repo` | `str` | Repository used as scoring context |
| `raw_score` | `float` | Raw graph score before normalization |
| `raw_score` | `float` | Pre-normalization score: graph score (v1) or logit (v2) |
| `normalized_score` | `float` | Normalized score (0.0 - 1.0) |
| `trust_level` | `TrustLevel` | HIGH, MEDIUM, LOW, UNKNOWN, or BOT |
| `percentile` | `float` | Percentile rank (0.0 - 1.0) |
Expand Down
2 changes: 1 addition & 1 deletion docs/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ Returns an expanded breakdown with contributions, flags, and metadata.
"context_repo": "octocat/Hello-World",
"trust_level": "HIGH",
"normalized_score": 0.82,
"raw_score": 0.0045,
"raw_score": 0.2871,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example output for raw_score in the get_trust_details tool has been updated to reflect the new logit value for v2. This ensures consistency between the documentation and the actual API response.

"account_age_days": 3650,
"total_merged_prs": 47,
"unique_repos_contributed": 12,
Expand Down
14 changes: 13 additions & 1 deletion docs/methodology.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ These raw weights are normalized to sum to 1.0, so actual values in the random w

The directed graph is scored using personalized graph-based ranking with a damping factor (alpha) of 0.85. This produces a raw score for the user node.

Normalization converts the raw score to a 0-1 range:
**v1:** Normalization converts the raw graph score to a 0-1 range:

```
baseline = 1 / num_nodes
Expand All @@ -111,6 +111,18 @@ normalized = ratio / (ratio + 1)

This sigmoid-like mapping means a score equal to the uniform baseline maps to 0.5, with diminishing returns above.

**v2:** The normalized graph score is combined with merge rate and account age into a logit, then passed through a sigmoid:

```
logit = intercept + graph_score_weight * graph_score
+ merge_rate_weight * merge_rate
+ account_age_weight * log(account_age_days + 1)

normalized = 1 / (1 + e^(-logit))
```

`raw_score` in the v2 output contains the pre-sigmoid logit value.
Comment on lines +114 to +124

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The new section clearly explains the v2 normalization process and explicitly states that raw_score in v2 output contains the pre-sigmoid logit value. This is crucial for users to correctly interpret the v2 results.


### Classification

| Level | Threshold | Meaning |
Expand Down
2 changes: 1 addition & 1 deletion src/good_egg/scorer.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ def _score_v2(
return TrustScore(
user_login=login,
context_repo=context_repo,
raw_score=raw_score,
raw_score=logit,
normalized_score=normalized,
trust_level=trust_level,
account_age_days=user_data.profile.account_age_days,
Expand Down
5 changes: 3 additions & 2 deletions tests/test_scorer.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from __future__ import annotations

import math
from datetime import UTC, datetime, timedelta

from good_egg.config import GoodEggConfig
Expand Down Expand Up @@ -426,8 +427,8 @@ def test_v2_scoring_produces_nonzero_score(self) -> None:
data = _make_contribution_data(merged_prs=prs, repos=repos, closed_pr_count=5)
result = scorer.score(data, "my-org/my-elixir-app")

assert result.raw_score > 0.0
assert 0.0 <= result.normalized_score <= 1.0
expected = 1.0 / (1.0 + math.exp(-result.raw_score))
assert abs(result.normalized_score - expected) < 1e-9
assert result.scoring_model == "v2"

def test_v2_component_scores_populated(self) -> None:
Expand Down