GPQA
Benchmarks
TaskDatasetSOTA resultTrendResults
GPQA Diamond
53.13Accuracy
9
GPQA Diamond
28.79Accuracy (Pass@1)
9
GPQA Diamond
29.96Pass@1 (Reasoning)
9
GPQA
62.9FRS Score
9
GPQA 0-shot
27.5GPQA Accuracy (0-shot)
9
GPQA
45.46Score
9
GPQA
36.36GPQA Score
9
GPQA Diamond
58.6Biology Score
9
GPQA Diamond
41.41Accuracy
9
GPQA Diamond Expert-level OOD
37.3pass@1
9
GPQA
57.5GPQA Score
9
GPQA
34.5Accuracy
9
GPQA
43.81Mean@8
9
GPQA
180.31OTPS
9
GPQA
61.5Score
9
GPQA (test)
0.279Accuracy
9
GPQA
65.43Avg@8 Accuracy
9
GPQA
198Calls
9
GPQA
67.42Accuracy
8
GPQA Diamond
57.32Pass@1
8
GPQA-D
68.2Pass@1 Accuracy
8
GPQA (198)
73.2Accuracy
8
GPQA
91.41P@8
8
GPQA Diamond
34.3Accuracy
8
GPQA
51.52Accuracy
8