[ankit27755]

Adds DeputyDev v1 to the 50-PR offline code-review benchmark, including GPT-5.5 judged results.

What is included

• one deputydev-v1 review entry for each of the 50 benchmark tasks
• 267 raw review comments
• 224 extracted candidates
• GPT-5.5 candidates, dedup groups, and evaluations
• DeputyDev registration in the evaluated-tools list and dashboard
• support for collecting reviews posted through a human GitHub account
• a focused downloader regression test

GPT-5.5 result

• TP: 74
• FP: 145
• FN: 63
• micro precision: 33.79%
• micro recall: 54.01%
• skipped evaluations: 0
• judge errors: 0

Validation

• exactly 50 DeputyDev v1 benchmark reviews
• exactly 50 candidate and evaluation entries
• 43 dedup-group entries
• only deputydev-v1 is present as the tool key in the GPT-5.5 result files
• downloader tests: 8 passed
• focused Ruff checks passed
• generated dashboard includes the gpt-5.5 model and DeputyDev v1

The source clone repository names and PR URLs retain their original internal run identity for provenance, while the submitted public benchmark tool identity is deputydev-v1.

[ashleyzhang01]

hey @ankit27755, can you provide the Github org of the repos/PRs used for your run of the benchmark?

[ankit27755]

hey @ankit27755, can you provide the Github org of the repos/PRs used for your run of the benchmark?

[ankit27755]

hey @ankit27755, can you provide the Github org of the repos/PRs used for your run of the benchmark?

Hey @ashleyzhang01 :
GitHub organization: https://github.com/dd-benchmarking
The organization also contains repositories from earlier internal benchmark runs. The 50 repositories used for the submitted DeputyDev V5 result are specifically those matching:

__deputydev-v5__PR__20260819
You can filter them here:
https://github.com/orgs/dd-benchmarking/repositories?q=deputydev-v5

Each recreated benchmark PR is PR #1 in its corresponding repository.
Example:
dd-benchmarking/grafana__grafana__deputydev-v5__PR80329__20260819#1

Repositories containing identifiers such as deputydev-v2, deputydev-v2-gpt-5-6, or other run dates were from previous experiments and were not used for the submitted V5 score. I can also provide the exact list of 50 PR URLs if needed.