JudgeBench
Benchmarks
TaskDatasetSOTA resultTrendResults
JudgeBench
93.3Accuracy
117
JudgeBench (test)
71.53AUROC
77
JudgeBench (test)
82Overall
40
JudgeBench
74.8Positional Consistent Accuracy
30
JudgeBench
84.19Accuracy
29
JudgeBench
0.037Kuiper
24
JudgeBench (test)
83.4Score
22
JudgeBench
74.6Knowledge
22
JudgeBench
85.2Reward Modeling Score
16
JudgeBench (test)
79.9Knowledge
16
JudgeBench
75.7Accuracy
16
JudgeBench Knowledge
74.4Accuracy
16
JudgeBench response pairs generated by GPT-4o 1.0
68.18Knowledge
11
JudgeBench
77.3Knowledge Accuracy
8
JudgeBench
56.3Positional Consistency Score
8
JudgeBench (Merged GPT Claude)
87.38Direct Baseline Score
8
JudgeBench (test)
5.63Kuiper
8