GPQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GPQA
94.44Accuracy
33
GPQA
0.83AUROC
32
GPQA Diamond v1 (test)
86.7Avg@5
32
GPQA Diamond
64.58Pass@1
32
GPQA Diamond
72.6Accuracy
31
GPQA
70.18Pass@1
31
GPQA
60.9Accuracy
30
GPQA
60Accuracy
30
GPQA
1.58Average Inference Time (s)
30
GPQA
61.19Accuracy
29
GPQA
66Pass@1
29
GPQA (test)
78.18Accuracy
29
GPQA main
69.64Accuracy
28
GPQA
70Exact Match
28
GPQA
33.48Score
28
GPQA
44.1avg@16
28
GPQA
66.2Accuracy
28
GPQA-Diamond
99.3Final Accuracy
28
GPQA D
65.2Accuracy
28
GPQA Natural Setting (test)
90Accuracy
28
GPQA Diamond
1,440.8Throughput (tokens/s)
27
GPQA-D
75.8Accuracy
27
GPQA
32.89Accuracy (Multi-choice)
27
GPQA Diam
58.9Accuracy
27
GPQA
79.29Accuracy
27