Helpfulness preference labeling accuracy on Ultra-Creative
73.7Accuracy (Bench)Few-Shot
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Few-ShotJudge=Deepseek-V32026.02 | 73.7 | 58.6 | 15.1 | |
| Few-ShotJudge=Qwen3-14B2026.02 | 73.5 | 57.8 | 15.7 | |
| Biased Rubric SearchJudge=Deepseek-V32026.02 | 73.5 | 54.7 | 18.8 | |
| RandomJudge=Qwen3-14B2026.02 | 73.4 | 56.2 | 17.2 | |
| SeedJudge=Deepseek-V32026.02 | 73.4 | 59.6 | 13.8 | |
| RandomJudge=Deepseek-V32026.02 | 73.4 | 58.5 | 14.9 | |
| TextGradJudge=Deepseek-V32026.02 | 73.4 | 60.5 | 12.9 | |
| SeedJudge=Qwen3-14B2026.02 | 72.8 | 59.1 | 13.7 | |
| Few-ShotJudge=Gemma-3-27b-it2026.02 | 72.7 | 59.3 | 13.4 | |
| Biased Rubric SearchJudge=Qwen3-14B2026.02 | 72.1 | 54.5 | 17.6 | |
| TextGradJudge=Qwen3-14B2026.02 | 72 | 57.3 | 14.7 | |
| RandomJudge=Gemma-3-27b-it2026.02 | 71.5 | 58 | 13.5 | |
| TextGradJudge=Gemma-3-27b-it2026.02 | 71.5 | 58.2 | 13.3 | |
| Biased Rubric SearchJudge=Gemma-3-27b-it2026.02 | 71 | 58 | 13 | |
| SeedJudge=Gemma-3-27b-it2026.02 | 70.3 | 60.1 | 10.2 |