GPQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GPQA Diamond
84Accuracy
65
GPQA (test)
70.4Accuracy
65
GPQA Diamond (test)
65.9Accuracy
63
GPQA main (test)
40.85Exact Match Accuracy
60
GPQA
86.4Accuracy
59
GPQA Diamond
91.9Pass@1
57
GPQA Diamond
86.4Pass@1 Accuracy
57
GPQA (test)
64.44Accuracy
56
GPQA
83.8Accuracy
55
GPQA Diamond (test)
67.34Accuracy@5
54
GPQA Diamond
65.25Accuracy
54
GPQA diamond
4.43Latency
54
GPQA-D
59.47Accuracy
53
GPQA-D
76.3Accuracy
52
GPQA
54.44Accuracy
51
GPQA
90.4Accuracy
51
GPQA
70.2Pass@1
50
GPQA Diamond
81.3Accuracy (GPQA Knowledge)
49
GPQA
85.7Accuracy
49
GPQA Diamond
75.7Pass@1
49
GPQA Diamond
91.17Throughput
48
GPQA Diamond
91.9Accuracy
48
GPQA
67.3Accuracy
48
GPQA Diamond
87.5Accuracy
48
GPQA Diamond
77Accuracy
47