Thanks for your valuable work in MoE!
When I try to evaluate the openmoe-8b-chat model by MT-bench with this command:
python gen_model_answer.py --model-path OrionZheng/openmoe-8b-chat --model-id moe-8b-chat --dtype bfloat16.
It throws this error: AssertionError: assert max_length <= self.sin.shape[0]
It can be solved by setting the max_new_token to 512 (The default is 1024). But I can only get a 2.0 score on MT-bench.
Would you happen to have any suggestions?
Thanks for your valuable work in MoE!
When I try to evaluate the openmoe-8b-chat model by MT-bench with this command:
python gen_model_answer.py --model-path OrionZheng/openmoe-8b-chat --model-id moe-8b-chat --dtype bfloat16.
It throws this error: AssertionError: assert max_length <= self.sin.shape[0]
It can be solved by setting the max_new_token to 512 (The default is 1024). But I can only get a 2.0 score on MT-bench.
Would you happen to have any suggestions?