List your company

ResearchFormerly WizWand

Lean

Benchmarks

TaskDatasetSOTA resultTrendResults

Formal theorem proving

Lean (test)

73Pass@1

14

Formal Proof Generation

Lean 4 (val)

49.8Pass@1

6

Theorem Retrieval

Lean Augmented Statement

82.7R@1

4

Theorem Retrieval

Lean Synthetic User Query (Sec. 3.1)

54.4Recall@1

4

Theorem Retrieval

Lean Informalized Statement

64.2R@1

4

Theorem Retrieval

Lean Augmented Statement cleaned (test)

86.8R@1

2

Theorem Retrieval

Lean Synthetic User Query cleaned (test)

57.3Recall@1

2

Theorem Retrieval

Lean Informalized Statement cleaned (test)

65.7R@1

2