GPQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GPQA Diamond
61.62Avg@1 Score
19
GPQA Diamond
69.5pass@1
19
GPQA D
85.4Score
18
GPQA Diamond
43.43Pass@1 Accuracy
18
GPQA
53.1Accuracy (ACC)
18
GPQA Diamond
80.3Score
18
GPQA D
43.2Pass@1 Accuracy
18
GPQA (test)
55.1Accuracy
18
GPQA Diamond
52.02Avg@4 Accuracy
18
GPQA Diamond
95Accuracy
18
GPQA
69.2Accuracy
18
GPQA
45.46Strict Match Accuracy
18
GPQA Diamond
52.5Accuracy (ACC)
18
GPQA
2,341.3Throughput
18
GPQA
0.117Brier Score
18
GPQA
37.9Mean@16
18
GPQA
79.42Accuracy
18
GPQA
18.2pass@1
18
GPQA Diamond
58.13Accuracy (avg@32)
18
GPQA 1.0 (test)
53.8Accuracy
18
GPQA diamond
0.696AUROC
18
GPQA main
0.672AUROC
18
GPQA
56.94Accuracy
17
GPQA
63.1Strict Accuracy
17
GPQA
31.92Accuracy
17