Mathematical Reasoning on FALSE Controls 5 theorems, 25 items (pilot)
100Accuracyo4-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| o4-miniType=Reasoning model2026.05 | 100 | 0 | 100 | |
| DeepSeek R1Type=Reasoning model, Model name=deepseek-reasoner2026.05 | 100 | 0 | 100 | |
| Gemini 2.5 Flash2026.05 | 96 | 4 | 80 | |
| Claude Sonnet2026.05 | 96 | 4 | 80 | |
| GPT-4o-mini2026.05 | 92 | 8 | 80 | |
| Claude Haiku2026.05 | 92 | 8 | 60 | |
| GPT-4o2026.05 | 84 | 16 | 60 | |
| DeepSeek V3Model name=deepseek-chat2026.05 | 68 | 32 | 60 | |
| Always-TRUEDescription=Oracle2026.05 | 0 | 100 | 0 |