Mathematical Reasoning on FormInv v1 (50 theorems, 366 items)
96.4AccuracyDeepSeek V3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek V3Tier=Flagship2026.05 | 96.4 | 6.9 | 82 | |
| o4-miniTier=Reasoning2026.05 | 96.4 | 7.6 | 80 | |
| Gemini 2.5 Flash†Tier=Flagship2026.05 | 95 | 8.4 | 80 | |
| GPT-4o-miniTier=Efficient2026.05 | 94.3 | 8.7 | 78 | |
| GPT-4oTier=Flagship2026.05 | 94.2 | 7 | 82 | |
| Claude Sonnet 4.6Tier=Flagship2026.05 | 93.2 | 10.7 | 74 | |
| DeepSeek R1Tier=Reasoning2026.05 | 93 | 8.8 | 78 | |
| Llama 3.3 70BTier=Open2026.05 | 90.5 | 13.5 | 66 | |
| Claude Haiku 4.5Tier=Efficient2026.05 | 86 | 19.4 | 50 |