F2F
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MiniF2F (test)
99.6Success Rate
100
MiniF2F (val)
63.9Success Rate
59
miniF2F Isabelle (val)
57Success Rate
41
miniF2F Isabelle (test)
51.2Success Rate
39
miniF2F rw (test)
75Pass@8
24
miniF2F rw (val)
81.1Pass@8
24
miniF2F Lean (test)
52Pass@64
24
miniF2F
31.97Accuracy
18
miniF2F (test)
96TC@1
16
miniF2F (val)
42.2Pass@1
15
MiniF2F (test)
100Pass@8
13
miniF2F
66.31Proof Success Rate
12
miniF2F
228.64Average Token Cost
12
miniF2F (val)
25.8Proven Theorems Rate
11
miniF2F Lean (val)
60.2Cumulative Pass Rate
10
MINIF2F 244 problems
84.02Pass@8
9
miniF2F (test)
24.6Accuracy
6
miniF2F Easy Mode (test)
215Solved Problems (Pass@32)
5
miniF2F full n=244
40.6Total Solve Rate
4
miniF2F Lean (curriculum)
32.1Pass@64
3
miniF2F Hard Mode (test)
204Total Solved (Pass@32)
2
F2F mini
3.14Objects Score
1