Skip to content

fix: add LoRA support and batch generation to gsm8k eval script - #19

Open
yuxizhe wants to merge 1 commit into
NVlabs:mainfrom
yuxizhe:feat-eval
Open

fix: add LoRA support and batch generation to gsm8k eval script#19
yuxizhe wants to merge 1 commit into
NVlabs:mainfrom
yuxizhe:feat-eval

Conversation

@yuxizhe

@yuxizhe yuxizhe commented Nov 21, 2025

Copy link
Copy Markdown

This commit addresses two issues in the evaluation script:

  1. Fix LoRA Application: I found out the evaluation script wasn't applying the LoRARequest properly. This meant we were just evaluating the base model every time, which is why the results looked almost the same as pre-training. Inquiry about the detailed config of reproduction #14
  2. Implement Batch Evaluation: Added batched inference to the evaluation process. This optimization drastically improves performance, reducing the total evaluation time from several hours down to a few minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant