GPQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GPQA (held-out)
31.5Accuracy (GPQA held-out)
12
GPQA
50Accuracy
12
GPQA
36.98Wall-clock Time (s)
12
GPQA-D
68.5Accuracy
12
GPQA
48.1pass@1
12
GPQA-D
30.13GPQA-D Score
12
GPQA Diamond
67.93Accuracy
12
GPQA Diamond
53Accuracy
12
GPQA
82.4Accuracy
12
GPQA
0.703WP-AUC
12
GPQA Diamond
98pass@k
12
GPQA-D
49.5Accuracy
12
GPQA
0.546Accuracy (GPQA)
12
GPQA
35.66Avg@k
12
GPQA Diamond
64Accuracy
12
GPQA D
66.2Accuracy
12
GPQA Diamond
65.8Pass@4
12
GPQA Diamond
44.4Avg@8
12
GPQA DIAMOND
65.15EM
12
GPQA
38.22Accuracy
12
GPQA Diamond
55Accuracy (avg@4)
12
GPQA
48.13Accuracy
12
GPQA Diamond (test)
65.22Accuracy
12
GPQA
3.11Speedup
12
GPQA
50Avg@16
12