GPQA
Benchmarks
TaskDatasetSOTA resultTrendResults
GPQA Diamond (out-of-distribution)
48.5Accuracy
6
GPQA (test)
11.44Error Rate (%)
6
GPQA
10.82Error Rate (%)
6
GPQA Diamond
92.4Score
5
GPQA
50Exact Match (EM)
5
GPQA
1,158Latency (NoKV, ms)
5
GPQA Main
40.2Accuracy
5
GPQA
56.4Score
5
GPQA-Diamond (generalization)
41.6Accuracy
5
GPQA
17.27LLMcritic Calls
5
GPQA
18.81LLMcritic Calls
5
GPQA
69.2Pass@1
5
GPQA
38.25Pass@1 Accuracy
5
GPQA diamond en
32.32Accuracy
5
GPQA en
32.83Accuracy
5
GPQA diamond en
33.87Accuracy
5
GPQA
37.1Accuracy
5
GPQA (test)
34.3Accuracy
5
GPQA Phys (test)
45.9Accuracy
5
GPQA-Chem (test)
30.2Accuracy
5
GPQA
0.922Correlation
5
GPQA Protocol A (test)
87.3Accuracy
5
GPQA (OOD)
28.88Accuracy
5
GPQA Med
63.16Accuracy
5
GPQA Diamond
70.83Score
5