Skip to content

fix(judge): make weighted criterion evidence authoritative - #1239

Closed
seonghobae wants to merge 3 commits into
fix-assert-validations-7577214956643354730from
fix/judge-weighted-score-boundary-1235-1238
Closed

seonghobae wants to merge 3 commits into
fix-assert-validations-7577214956643354730from
fix/judge-weighted-score-boundary-1235-1238

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Superseded by live parent implementation

This successor originally carried #1235/#1238 on top of fix-assert-validations-7577214956643354730@4163bbb42763282c0b7729c37e8b919895141a5f.

During concurrent development, that exact parent branch advanced to e4d170040ed2230bef8133855826fbdfaec29b86 and independently implemented the same two bounded defects in llm_judge.py: aggregate criterion-weight overflow now fails before transport, and the direct/plain result derives its authoritative score/decision from validated weighted criterion evidence while retaining top-level score validation. Fresh tests on the parent cover both regressions, and its repository-local CI/Security/SAST/CodeQL are exact-head green.

Because this branch is now 3 commits ahead / 1 behind the live parent with overlapping implementations of the same mutable source and issue scope, retaining it would create a competing writer rather than add distinct product value. No rebase, force-push, or duplicate merge is performed. The live implementation is adopted in PR #1019, whose body now closes #1235 and #1238 and documents the expanded scope.

Historical lineage retained for audit only:

  • RED 4ee40e6b139da203194dafaeabed6a9d2550ecfa
  • GREEN e52c79be5126eb00eecfe59addf98f099afdaff1
  • evidence 74da6ae89b9e2b787a85e72b0df091c1f84bec3b

Closed without merge as superseded; no product requirement is being dropped.

@opencode-agent opencode-agent Bot added area: auth Authentication, authorization, identity, or tenant isolation priority: medium Normal-priority or P2 work status: draft Draft pull request type: bug Defect or incorrect behavior labels Aug 22, 2026
@seonghobae
seonghobae marked this pull request as ready for review August 22, 2026 09:38
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9a592ba0-5089-42f9-bf33-206ed6e45291

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

An error occurred during the review process. Please try again later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@opencode-agent opencode-agent Bot added status: needs-review Open pull request requiring current-head review or checks and removed status: draft Draft pull request labels Aug 22, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread python/fast_mlsirm/llm_judge.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: auth Authentication, authorization, identity, or tenant isolation priority: medium Normal-priority or P2 work status: needs-review Open pull request requiring current-head review or checks type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant