GPQA
Benchmarks
TaskDatasetSOTA resultTrendResults
GPQA Diamond
84Accuracy
65
GPQA (test)
70.4Accuracy
65
GPQA Diamond (test)
65.9Accuracy
63
GPQA main (test)
40.85Exact Match Accuracy
60
GPQA
86.4Accuracy
59
GPQA Diamond
91.9Pass@1
57
GPQA Diamond
86.4Pass@1 Accuracy
57
GPQA (test)
64.44Accuracy
56
GPQA
83.8Accuracy
55
GPQA Diamond (test)
67.34Accuracy@5
54
GPQA Diamond
65.25Accuracy
54
GPQA diamond
4.43Latency
54
GPQA-D
59.47Accuracy
53
GPQA-D
76.3Accuracy
52
GPQA
54.44Accuracy
51
GPQA
90.4Accuracy
51
GPQA
70.2Pass@1
50
GPQA Diamond
81.3Accuracy (GPQA Knowledge)
49
GPQA
85.7Accuracy
49
GPQA Diamond
75.7Pass@1
49
GPQA Diamond
91.17Throughput
48
GPQA Diamond
91.9Accuracy
48
GPQA
67.3Accuracy
48
GPQA Diamond
87.5Accuracy
48
GPQA Diamond
77Accuracy
47