Rubric Discovery on Arena-Expert-5K, HelpSteer3, HH-RLHF, UltraFeedback cross-source mean
4.45St. ScorePREMISE (+ VFR unconstrained)
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PREMISE (+ VFR unconstrained)K=18.02026.05 | 4.45 | 48.8 | 31.3 | 58.5 | 29.9 | 55.2 | 0.09 | 14 | 64.8 | 66.4 | 89.9 | |
| CRITIQK=5.82026.05 | 4.4 | 35.8 | 34.6 | 59.2 | 43.9 | 75.3 | 0.01 | 7 | 68.1 | 69.5 | 87.8 | |
| PREMISE (+ VFR constrained)K=18.02026.05 | 4.39 | 51.9 | 27.4 | 60.9 | 36 | 57.7 | 0.09 | 14 | 64.9 | 67 | 90.9 | |
| PREMISE (discovered)K=20.02026.05 | 4.21 | 53.1 | 26.3 | 59.7 | 46.4 | 57.7 | 0.08 | 15 | 65 | 66.5 | 91.9 | |
| AUTO-RUBRICK=20.02026.05 | 3.94 | 31.9 | 13.9 | 36.5 | 29.7 | 3 | 0.02 | 10 | 68.7 | 63.7 | 90.8 | |
| PREMISE (+ pref repair)K=20.02026.05 | 3.83 | 49.9 | 24.1 | 54.8 | 43.2 | 34 | 0.08 | 11 | 68.6 | 68.5 | 91.6 | |
| INVERSE CAIK=2.82026.05 | 3.44 | 55.8 | 37.8 | 54.4 | 51.7 | 63 | 0.05 | 4 | 67.5 | 61.4 | 93.2 | |
| AGENTEVALK=4.02026.05 | 3.38 | 58.2 | 28.1 | 70.2 | 33 | 82.4 | 0.06 | 3 | 68.2 | 66.5 | 87.3 | |
| AUTORULEK=26.22026.05 | 2.12 | 51.3 | 25.9 | 58.5 | 50.2 | 40.3 | 0.04 | 18 | 68.2 | 65.7 | 88.6 |