Rubric Generation on OSS EVAL 300
1Spearman's RhoGold
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GoldRubric Source=Gold2026.03 | 1 | 1 | 0.366 | 0 | |
| RubricRAG (nothink)Rubric Source=RubricRAG (nothink), reasoning=disabled2026.03 | 0.545 | 0.478 | 0.408 | 0.042 | |
| RubricRAG (think)Rubric Source=RubricRAG (think), reasoning=enabled2026.03 | 0.495 | 0.415 | 0.374 | 0.008 | |
| Few-ShotRubric Source=Few-Shot2026.03 | 0.466 | 0.368 | 0.397 | 0.031 | |
| SFTRubric Source=SFT2026.03 | 0.457 | 0.309 | 0.287 | -0.079 | |
| Zero-ShotRubric Source=Zero-Shot2026.03 | 0.426 | 0.33 | -0.035 | -0.401 | |
| GRPORubric Source=GRPO2026.03 | 0.331 | 0.223 | 0.134 | -0.232 |