Skip to content

Clarify local benchmarking and the notebook/resource directory roles - #22

Merged
kevinhyj merged 1 commit into
mainfrom
benchmark-documentation-20260909
Sep 9, 2026
Merged

kevinhyj merged 1 commit into
mainfrom
benchmark-documentation-20260909

Conversation

@kevinhyj

@kevinhyj kevinhyj commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

The comparison-model guides now start with locally prepared checkpoints and prediction CSVs from upstream scorers. Repeated connectivity observations are removed from the usage path; checkpoint identity, required environments and scoring protocols remain documented where they matter for an exact paper comparison.

Add entry READMEs for notebooks/ and reproduction/, showing how interactive notebooks, command-line scripts and fixed reproduction resources work together. The root repository map links these guides. Existing files and scientific workflows remain in place, and the documentation checker now includes both guides.

Validation: 142 maintained relative links checked and git diff --check passed. The existing standard-library evaluator processed all 135 stored Milena predictions through the documented CSV interface and returned Spearman 0.8360456283218484 without loading any model. Python 3.10/3.11 repository CI passed on this revision.

@kevinhyj
kevinhyj merged commit de2cf7a into main Sep 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant