Skip to content

[DON'T MERGE] Testing router submission via file modification - #47

Closed
yl231 wants to merge 6 commits into
mainfrom
trigger-evaluation
Closed

yl231 wants to merge 6 commits into
mainfrom
trigger-evaluation

Conversation

@yl231

@yl231 yl231 commented Nov 29, 2025

Copy link
Copy Markdown
Contributor

No description provided.

…ion option

* Updated the PR evaluation workflow to detect changed prediction files more accurately by comparing against the fork's base branch.
* Added a `--force` option to the evaluation script to allow re-evaluation of all entries, even if they have already been evaluated.
* Minor adjustments to the GLM-4-air-router prediction JSON to test the above functionalities.
@github-actions

Copy link
Copy Markdown

📊 Router Evaluation Results

Router: glm-4-air-router
Dataset Split: full

Metrics

Metric Value
RouterArena Score 0.5617
Accuracy 54.65%
Total Cost $0.396728
Avg Cost per Query $0.000047
Avg Cost per 1K Queries $0.0472
Number of Queries 8400

Evaluation completed by RouterArena automated workflow

@yl231

yl231 commented Nov 30, 2025

Copy link
Copy Markdown
Contributor Author

Good to go. See you in the next official PR.

@yl231 yl231 closed this Nov 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant