GPQA
Benchmarks
TaskDatasetSOTA resultTrendResults
GPQA Diamond
76.7Accuracy
22
GPQA
68.54Accuracy
22
GPQA
44.1Accuracy
22
GPQA
79.2Pass@1
22
GPQA
68.3Mean@1
22
GPQA
79.29Pass@8
21
GPQA D
38.5Accuracy (mean@4)
21
GPQA
32.8Accuracy
21
GPQA-D
61.6Accuracy
21
GPQA
59.1Score
21
GPQA
29.87Performance (%)
21
GPQA Diamond
91.94P@16
21
GPQA Diamond
50.2Pass@1
21
GPQA
204.55AUPC
20
GPQA Diamond
58.8Accuracy
20
GPQA Main (test)
23.04P@1
20
GPQA Diamond
84.3Pass@1 Score
20
GPQA Main
73.2Accuracy
20
GPQA D
73.18Accuracy
20
GPQA-D
14.43Accuracy (GPQA-D)
20
GPQA
2,513# Traces
20
GPQA full dataset
66.29Accuracy
20
GPQA Diamond set (test)
82.29Accuracy (Overall)
20
GPQA Diamond
67Accuracy
19
GPQA Biology, Chemistry, Physics
35.7Biology Score
19