GPQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GPQA
18,851.29Avg Context Length (tokens)
16
GPQA (test)
0.609Accuracy
15
GPQA
33.84P@1
15
GPQA D
5,318Tokens Used for Thinking
15
GPQA
35.66Avg@k
15
GPQA Diamond
55.19Average@32
15
GPQA
33.3Accuracy
15
GPQA D
38.72Accuracy
15
GPQA Diamond
66.8Pass@1 Accuracy
15
Super GPQA
2.096Speedup Ratio
15
GPQA
50.6Accuracy
15
GPQA (val)
84GPQA Score
15
GPQA Diamond
55Pass@1
15
GPQA Extended
57.9Overall Accuracy
15
GPQA
84.85Accuracy
14
GPQA
47.98Accuracy
14
GPQA Diamond
69.89pass@3
14
GPQA
77.06Pass@1 Accuracy
14
GPQA
65.8Biology Domain Score
14
GPQA Diamond
68.94Accuracy
14
GPQA Diamond
52Accuracy
14
GPQA Diamond
65.2pass@1
14
GPQA
48.5Accuracy
14
GPQA Diamond
58.33Accuracy (avg@k)
14
GPQA
36.4Accuracy
14