Skip to content

Add LLM Benchmark Costco for benchmark discovery and protocol inspection - #95

Open
joe1chief wants to merge 1 commit into
benchflow-ai:mainfrom
joe1chief:add-llm-benchmark-costco
Open

joe1chief wants to merge 1 commit into
benchflow-ai:mainfrom
joe1chief:add-llm-benchmark-costco

Conversation

@joe1chief

Copy link
Copy Markdown

Adds LLM Benchmark Costco to §6 as a resource for inspecting task-construction and grading protocols before choosing a benchmark to reproduce.

I maintain the project. The browser brings together benchmark metadata, English/Chinese summaries, original-source links, and separate construction/evaluation flowcharts. The entry identifies it as a benchmark browser and includes a source-verification caveat.

Try the AutomationBench example, then open Build Process. It separates task construction from evaluation and distinguishes public-task local results from the official private leaderboard. The browser helps readers locate protocol details; it does not run evaluations or produce model rankings.

For the inclusion-evidence requirement: as of September 7, 2026, the repository has 6 stars, from five accounts other than mine plus my own. The repository was created on March 2, 2026. This is an early external-interest signal under the contribution guide's star criterion, not a claim of verified production adoption. I do not yet have an independently documented deployment or user case study to cite.

Validation: this changes only one README entry; checked the project, live browser, and example links, and ran git diff --check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant