In another research project I was on, instead of averaging StrongREJECT scores across responses, we measured the percentage of responses that achieved the maximum StrongREJECT score of 1.0. We found that oftentimes responses with StrongREJECT scores of, say, 0.5, did not look very harmful.
This looks easy to incorporate into our codebase by adding another field to the output of compute_results().
This is somewhat blocked by adding the rubric-based StrongREJECT evaluator into the codebase, since the rubric-based evaluator gives scores that are multiples of 1/16th, whereas the classifier-based evaluator gives floating point values and will never achieve the max score of 1.0. Though for the classifier-based evaluator we could instead define a threshold (e.g., measure the percentage of responses whose score is at least 1 - 1/32.)
In another research project I was on, instead of averaging StrongREJECT scores across responses, we measured the percentage of responses that achieved the maximum StrongREJECT score of 1.0. We found that oftentimes responses with StrongREJECT scores of, say, 0.5, did not look very harmful.
This looks easy to incorporate into our codebase by adding another field to the output of
compute_results().This is somewhat blocked by adding the rubric-based StrongREJECT evaluator into the codebase, since the rubric-based evaluator gives scores that are multiples of 1/16th, whereas the classifier-based evaluator gives floating point values and will never achieve the max score of 1.0. Though for the classifier-based evaluator we could instead define a threshold (e.g., measure the percentage of responses whose score is at least
1 - 1/32.)