Preference Evaluation on UltraFeedback core250 (test)
80Win RateTEA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TEABaseline=GRPO, Evaluation Budget (N)=642026.05 | 80 | 0.8 | 19.2 | |
| TEABaseline=GRPO, Evaluation Budget (N)=1282026.05 | 78.4 | 0.8 | 20.8 | |
| Prefix-TEABaseline=GRPO, Evaluation Budget (N)=1282026.05 | 76.4 | 0.8 | 22.8 | |
| TEABaseline=Best TTA, Evaluation Budget (N)=642026.05 | 75.6 | 0.8 | 23.6 | |
| TEABaseline=GRPO, Evaluation Budget (N)=2562026.05 | 75.2 | 1.6 | 23.2 | |
| Prefix-TEABaseline=GRPO, Evaluation Budget (N)=642026.05 | 74.8 | 0.8 | 24.4 | |
| TEABaseline=Best TTA, Evaluation Budget (N)=1282026.05 | 70.4 | 0.8 | 28.8 | |
| Prefix-TEABaseline=GRPO, Evaluation Budget (N)=2562026.05 | 69.2 | 2 | 28.8 | |
| TEABaseline=Best TTA, Evaluation Budget (N)=2562026.05 | 67.2 | 1.6 | 31.2 | |
| TEABaseline=GRPO-Z, Evaluation Budget (N)=642026.05 | 63.2 | 0 | 36.8 | |
| TEABaseline=GRPO-Z, Evaluation Budget (N)=1282026.05 | 62.8 | 1.2 | 36 | |
| TEABaseline=GRPO-Z, Evaluation Budget (N)=2562026.05 | 58.4 | 1.2 | 40.4 |