Olympiad Bench
Benchmarks
TaskDatasetSOTA resultTrendResults
Olympiad Bench
76.71Accuracy
254
Olympiad Bench
68.13Pass@1 Accuracy
115
Olympiad Bench
85.2Accuracy
73
Olympiad Bench
55.4Accuracy
36
Olympiad Bench
62.1Pass@1
35
Olympiad Bench
48.5Pass@8
30
Olympiad-Bench
46.88Accuracy Pass@1
27
Olympiad Bench
68.9Score
26
Olympiad Bench (test)
83.53Pass@1 Accuracy
24
Olympiad bench
57.2Pass@1 Accuracy (@16)
17
Olympiad Bench
47.8Average@16 Accuracy
15
Olympiad-Bench
58Avg@32
14
Olympiad Bench
73.67PAQ
14
Olympiad Bench
59.3Peak Accuracy
12
Olympiad Bench
0.57Peak Accuracy
12
Olympiad-Bench
51.94Pass@1
12
Olympiad Bench
49.17Average Accuracy
10
Olympiad Bench
49.1Pass@8 Accuracy
10
Olympiad Bench
44Accuracy
9
Olympiad Bench
70.36Pass@1 Accuracy
6
Olympiad Bench
52.75Test Accuracy
6
Olympiad Bench
69.04Pass@32
6
Olympiad Bench
41.25Avg@32
6
Olympiad Bench N=112
89.3Top-1 Accuracy
5
Olympiad Bench
55.8Accuracy
4