Skip to content

Skip scoring for existing contributors#40

Merged
jeffreyksmithjr merged 4 commits intomainfrom
skip-known-contributors
Feb 26, 2026
Merged

Skip scoring for existing contributors#40
jeffreyksmithjr merged 4 commits intomainfrom
skip-known-contributors

Conversation

@jeffreyksmithjr
Copy link
Contributor

Summary

  • Add a lightweight GraphQL pre-check that short-circuits the full scoring pipeline when the PR author already has merged PRs in the target repo
  • New EXISTING_CONTRIBUTOR trust level, skip_known_contributors config toggle (default: true), --force-score CLI flag, and force_score MCP parameter
  • Handles the new trust level in all output formatters (markdown, CLI, check run, JSON)

Closes #39

Test plan

  • uv run ruff check src/ tests/ scripts/ passes
  • uv run mypy src/good_egg/ passes
  • uv run pytest --cov=good_egg -v — 293 tests pass, 94% coverage

Add a lightweight pre-check that short-circuits the full scoring
pipeline when the PR author already has merged PRs in the target repo.
This avoids fetching all PRs and building a contribution graph for
established contributors.

- Add EXISTING_CONTRIBUTOR trust level to TrustLevel enum
- Add skip_known_contributors config field (default: true) with
  GOOD_EGG_SKIP_KNOWN_CONTRIBUTORS env var override
- Add check_existing_contributor() GraphQL method on GitHubClient
- Short-circuit in score_pr_author() and action entry point
- Add --force-score CLI flag and force_score MCP parameter
- Handle new trust level in all formatters (markdown, CLI, check run)
- Add skip-known-contributors action input and skipped output
@github-actions
Copy link

github-actions bot commented Feb 26, 2026

🥚 Better Egg: HIGH Trust

Score: 80%

Score Breakdown

Component Value
Graph Score 82%
Merge Rate 86% (42/49 PRs)
Account Age 4,928 days

Top Contributions

Repository PRs Language Stars
2ndSetAI/good-egg 17 Python 16
jeffreyksmithjr/verskyt 9 Python 2
jeffreyksmithjr/galapagos_nao 7 Elixir 21
aws-samples/aws-big-data-blog 3 Java 895
pytorch/pytorch.github.io 2 HTML 276
melissawm/open-source-ai-contribution-policies 1 N/A 89
nerves-project/nerves_examples 1 Elixir 402
kilimchoi/engineering-blogs 1 Ruby 37173
kdeldycke/plumage 1 CSS 55

@gemini-code-assist
Copy link

Summary of Changes

Hello @jeffreyksmithjr, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant optimization to the trust scoring system by recognizing and handling existing contributors more efficiently. By performing a quick check for prior merged pull requests, the system can now bypass the extensive scoring process for trusted users, leading to faster results and reduced resource consumption. This enhancement improves the overall user experience for established contributors while maintaining the integrity of the trust assessment for new or less familiar users.

Highlights

  • New Trust Level for Existing Contributors: Introduced a new EXISTING_CONTRIBUTOR trust level to recognize users who have previously merged pull requests in the target repository, indicating a level of established trust.
  • Optimized Scoring for Known Contributors: Implemented a lightweight GraphQL pre-check that short-circuits the full scoring pipeline for existing contributors, significantly reducing API calls and improving efficiency for trusted users.
  • Configurable Skipping Behavior: Added a skip_known_contributors configuration toggle (defaulting to true) to control whether scoring is skipped for existing contributors. This can be overridden via environment variables or configuration files.
  • Force Scoring Option: Provided a --force-score CLI flag and a force_score parameter for the MCP server, allowing users to explicitly force a full trust score calculation even for known contributors, bypassing the skipping mechanism.
  • Enhanced Output Formatting: Updated all output formatters (Markdown, CLI, Check Run, JSON) to correctly display and communicate the EXISTING_CONTRIBUTOR trust level and indicate when scoring has been skipped.
Changelog
  • action.yml
    • Added skip-known-contributors as an input parameter.
    • Updated the trust-level output to include EXISTING_CONTRIBUTOR.
    • Added a new skipped output to indicate if scoring was bypassed.
  • src/good_egg/action.py
    • Imported TrustScore for use in the action.
    • Implemented logic to read the skip_known_contributors input.
    • Added a pre-check using client.check_existing_contributor to determine if scoring should be skipped.
    • Created a TrustScore object with EXISTING_CONTRIBUTOR trust level if scoring is skipped.
    • Ensured the skipped output is correctly set based on whether scoring was bypassed.
  • src/good_egg/cli.py
    • Added a --force-score CLI option to bypass skipping for known contributors.
    • Modified the score function to accept and process the force_score argument.
    • Updated the configuration to set skip_known_contributors to False when --force-score is used.
  • src/good_egg/config.py
    • Added skip_known_contributors boolean field to GoodEggConfig with a default value of True.
    • Implemented logic to load skip_known_contributors from the GOOD_EGG_SKIP_KNOWN_CONTRIBUTORS environment variable.
  • src/good_egg/formatter.py
    • Added EXISTING_CONTRIBUTOR to the _TRUST_LEVEL_COLORS and _TRUST_LEVEL_EMOJI mappings.
    • Implemented specific Markdown formatting for EXISTING_CONTRIBUTOR trust levels, including merged PR count.
    • Added dedicated CLI output formatting for EXISTING_CONTRIBUTOR trust levels.
    • Provided distinct check run summary formatting for EXISTING_CONTRIBUTOR trust levels.
  • src/good_egg/github_client.py
    • Defined a new GraphQL query _EXISTING_CONTRIBUTOR_QUERY to efficiently check for merged PRs.
    • Implemented the check_existing_contributor asynchronous method to query GitHub for a user's merged PR count in a given repository.
  • src/good_egg/mcp_server.py
    • Added force_score parameter to _scoring_resources, score_user, check_pr_author, and get_trust_details functions.
    • Updated the configuration within _scoring_resources to disable skip_known_contributors if force_score is set to True.
  • src/good_egg/models.py
    • Extended the TrustLevel enumeration with a new EXISTING_CONTRIBUTOR member.
  • src/good_egg/scorer.py
    • Integrated a check for config.skip_known_contributors within score_pr_author.
    • If skipping is enabled and the user is an existing contributor, a TrustScore with EXISTING_CONTRIBUTOR level is immediately returned, bypassing further scoring.
  • tests/test_action.py
    • Updated mock GitHubClient instances to include check_existing_contributor.
    • Added TestExistingContributorAction class with tests for skipping scoring for existing contributors.
    • Included a test to verify that skip_known_contributors=false correctly triggers full scoring.
    • Ensured get_user_contribution_data is not called when scoring is skipped.
  • tests/test_cli.py
    • Added a test case to confirm that the --force-score CLI option correctly disables skip_known_contributors in the configuration.
  • tests/test_config.py
    • Added tests for the default value of skip_known_contributors.
    • Included tests for explicitly setting skip_known_contributors to False.
    • Verified that skip_known_contributors can be overridden via YAML configuration files.
    • Added tests for overriding skip_known_contributors using environment variables (true, false, 1).
  • tests/test_formatter.py
    • Added TestExistingContributorFormatting class to test the new trust level's output.
    • Verified correct Markdown formatting for EXISTING_CONTRIBUTOR scores, including singular/plural PR counts.
    • Confirmed accurate CLI output formatting for EXISTING_CONTRIBUTOR scores.
    • Ensured proper check run summary and JSON output for the new trust level.
  • tests/test_github_client.py
    • Added TestCheckExistingContributor class to thoroughly test the new check_existing_contributor method.
    • Tested scenarios where merged PRs exist, no merged PRs exist, for bot logins, inaccessible repositories, and API errors.
  • tests/test_mcp_server.py
    • Added TestForceScoreParameter class to test the force_score parameter in MCP server functions.
    • Verified that force_score=True correctly disables skip_known_contributors for score_user, check_pr_author, and get_trust_details.
  • tests/test_scorer.py
    • Added TestExistingContributorSkip class to test the skipping logic in score_pr_author.
    • Verified that scoring is skipped when a contributor exists and skip_known_contributors is True.
    • Confirmed that full scoring proceeds when no merged PRs are found.
    • Ensured that skip_known_contributors=False bypasses the existing contributor check entirely.
Activity
  • The author has implemented a comprehensive test plan, including ruff check, mypy, and pytest with coverage, all passing successfully.
  • The pull request addresses and closes issue Skip scoring for existing contributors #39, indicating a resolution to a previously identified problem or feature request.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable optimization by skipping the full scoring pipeline for existing contributors. The implementation is comprehensive, adding new configuration options, API parameters, and updating all output formats. The changes are also well-covered by new tests.

I've identified a couple of areas for improvement:

  • There's some code duplication for the scoring logic that could be refactored for better maintainability.
  • There's an opportunity to improve type safety when parsing GraphQL responses to make the code more robust.

Overall, this is a solid contribution that improves the efficiency of the tool. My comments are aimed at further improving the code's long-term quality.

Good Egg eats its own dogfood, so always run full scoring to
exercise the complete pipeline on every PR.
- Eliminate scoring logic duplication: action.py now calls
  score_pr_author() instead of reimplementing the short-circuit
  inline. The skipped flag is derived from score.flags.
- Replace search-based GraphQL query with repository.pullRequests
  filterBy, which is a single-field query and more efficient.
- Use isinstance checks instead of type:ignore for GraphQL response
  parsing in check_existing_contributor.
- Extract _existing_contributor_context() helper in formatter.py to
  share PR count / plural logic across all three format functions.
- Set scoring_model on the skipped TrustScore so it reflects the
  configured model rather than defaulting to v1.
- Add boundary test for count == 1 in check_existing_contributor.
- Update docs: github-action.md, configuration.md, library.md,
  mcp-server.md, README.md, CLAUDE.md, examples/.good-egg.yml.
- Action "Acting on Trust Level" example now treats
  EXISTING_CONTRIBUTOR the same as HIGH for auto-approve
- Library usage example and docs check scoring_skipped flag before
  printing score/PR fields that would be zero when skipped
@jeffreyksmithjr jeffreyksmithjr merged commit 712da70 into main Feb 26, 2026
2 checks passed
@jeffreyksmithjr jeffreyksmithjr deleted the skip-known-contributors branch February 26, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip scoring for existing contributors

1 participant