Theorem Proving on PutnamBench (test)
72AccuracyHilbert
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HilbertCompute=avg. pass@1840, Open-source=false2025.10 | 72 | 462 | |
| Seed-ProverCompute=medium, Open-source=false2025.10 | 51 | 329 | |
| Ax-ProverCompute=pass@1‡, Open-source=true2025.10 | 14 | 92 | |
| Goedel-Prover-V2Compute=pass@184, Open-source=true2025.10 | 13 | 86 | |
| DeepSeek-Prover-V2Compute=pass@1024, Open-source=true2025.10 | 7 | 47 | |
| DSP+Compute=pass@128, Open-source=true2025.10 | 4 | 23 | |
| BourbakiCompute=pass@512, Open-source=true2025.10 | 2 | 14 | |
| Kimina-Prover-7B-DistillCompute=pass@192, Open-source=true2025.10 | 2 | 10 | |
| Self-play Theorem ProverCompute=pass@3200, Open-source=true2025.10 | 1 | 8 | |
| Goedel-Prover-SFTCompute=pass@512, Open-source=true2025.10 | 1 | 7 | |
| Gemini-2.5-ProCompute=pass@1, Open-source=false2025.10 | 0.5 | 3 | |
| GPT-4oCompute=pass@10, Open-source=false2025.10 | 0.2 | 1 | |
| Claude-3.7-SonnetCompute=pass@1, Open-source=false2025.10 | 0 | 0 |