Formal Theorem Proving on Number Theory
2.51PutnamBenchHilbert
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HilbertBackbone=Gemini 2.5 Pro, System=Agentic System2026.04 | 2.51 | 2.19 | |
| Goedel-Prover-V2-8BType=Open-source LLM2026.04 | 2.1 | 1.72 | |
| HilbertBackbone=Gemini 3.1 Pro, System=Agentic System2026.04 | 1.87 | 1.52 | |
| Goedel-Prover-V2-32BType=Open-source LLM2026.04 | 1.62 | 1.27 | |
| HilbertBackbone=GPT-5.3-Codex, System=Agentic System2026.04 | 1.2 | 1.07 | |
| DreamProverBackbone=Gemini 3.1 Pro, System=Lemma Learning System2026.04 | 1.15 | 0.64 | |
| DreamProverBackbone=Gemini 2.5 Pro, System=Lemma Learning System2026.04 | 1.13 | 0.54 | |
| Gemini 2.5 ProType=Proprietary LLM2026.04 | 0.96 | 0.61 | |
| DreamProverBackbone=GPT-5.3-Codex, System=Lemma Learning System2026.04 | 0.89 | 0.27 | |
| Gemini 3.1 ProType=Proprietary LLM2026.04 | 0.59 | 0.6 | |
| DeepSeek-Prover-V2-7BType=Open-source LLM2026.04 | 0.58 | 0.5 | |
| Claude 4.6 OpusType=Proprietary LLM2026.04 | 0.51 | 0.4 | |
| GPT-5.3-CodexType=Proprietary LLM2026.04 | 0.18 | 0.13 |