GPQA
Benchmarks
TaskDatasetSOTA resultTrendResults
GPQA
35.3Accuracy
5
GPQA Diamond (out-of-domain)
0.864Accuracy
5
GPQA Main
0.181Accuracy
5
GPQA
0.511Accuracy
5
GPQA
49.7Score
5
GPQA with tools
81.4Accuracy
4
GPQA no tools
80.5Accuracy
4
GPQA-Chem
20.4Accuracy
4
GPQA Bio
70Accuracy (GPQA Bio)
4
GPQA
50Utility
4
GPQA
41.94Average Accuracy
4
GPQA
6,763.8Avg Length
4
GPQA-D
59.1Pass@1 Accuracy
4
GPQA
28.01TPS
4
GPQA
30.95Throughput (TPS)
4
GPQA
34.15Score
4
GPQA
30.95TPS
4
GPQA Diamond
56.67Avg@5
4
GPQA-D
8.7Full Length
4
GPQA
2.73SGLang Acceptance Length
4
GPQA
4.04Acceptance Length
4
GPQA D
34.7Avg@4
4
GPQA D
72.55Avg@4
4
GPQA
151.7TPS (tok/s)
4
GPQA Diamond
71.5Pass@1 Score
4