model evaluation

[mars203030]

Hello,

do you have teh code for model evaluation

Regards,

[jingyeyang95]

Hello, for model evaluation, we randomly hold out 20% of the Biolark-gcs dataset, which you can get from https://data.mendeley.com/datasets/v4t59p8w4z/2
We manually compared the generated results with the dataset's labels for better accuracy. Specifically, we load our pre-trained model and give it text data from biolark as a prompt. The PhenoGPT will generate (predict) phenotypes and HPO IDs. You can compare these directly with the true results for evaluation.

[mars203030]

Thank you so much this is what I am intended too
I tried the code you posted for llama 2 but it seems I am not getting any answer.
is there a step I am missing

[mars203030]

here is wandb results

[kaichop]

are there any update on the evaluation? Please note that the phenogpt has been updated a few times and you may try the new version as well.