Model Ranking Agreement on MT-Bench Arena-derived reference ranking 1.0 (test)
121Coding ScoreSample25→20
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Sample25→20Judge=GPT-52026.06 | 121 | 128 | 121 | 122 | 132 | 80 | 95 | 168 | 121 | |
| Single→20Judge=GPT-52026.06 | 83 | 74 | 54 | 39 | 55 | 57 | 54 | 168 | 73 | |
| Single→20Judge=Prometheus2026.06 | 83 | 111 | 86 | 49 | 98 | 87 | 113 | 132 | 95 | |
| ScoreWin40Judge=GPT-52026.06 | 79 | 75 | 36 | 25 | 50 | 59 | 52 | 141 | 65 | |
| Sample25→20Judge=Prometheus2026.06 | 78 | 182 | 126 | 99 | 111 | 99 | 102 | 174 | 122 | |
| Random50→20×10Judge=GPT-52026.06 | 76 | 104 | 46 | 30 | 54 | 59 | 48 | 145 | 70 | |
| NoTrunc bootJudge=GPT-52026.06 | 75 | 99 | 44 | 28 | 51 | 57 | 46 | 142 | 68 | |
| BlockTop2Judge=GPT-52026.06 | 73 | 105 | 53 | 32 | 37 | 65 | 51 | 142 | 70 | |
| Trunc25→20Judge=GPT-52026.06 | 70 | 49 | 61 | 24 | 54 | 44 | 47 | 102 | 56 | |
| Random50→20×10Judge=Prometheus2026.06 | 69 | 132 | 90 | 43 | 62 | 80 | 60 | 162 | 87 | |
| NoTrunc bootJudge=Prometheus2026.06 | 66 | 118 | 88 | 41 | 57 | 79 | 59 | 159 | 83 | |
| BlockTop1Judge=GPT-52026.06 | 65 | 114 | 59 | 29 | 44 | 43 | 83 | 139 | 72 | |
| ScoreWin40Judge=Prometheus2026.06 | 60 | 84 | 85 | 35 | 51 | 69 | 55 | 152 | 74 | |
| BlockTop1Judge=Prometheus2026.06 | 54 | 137 | 91 | 30 | 70 | 93 | 56 | 159 | 86 | |
| BlockTop2Judge=Prometheus2026.06 | 53 | 94 | 86 | 20 | 54 | 92 | 53 | 156 | 76 | |
| Trunc25→20Judge=Prometheus2026.06 | 52 | 55 | 90 | 20 | 56 | 53 | 72 | 159 | 70 |