Counterexample generation on VERI-REASON
213Pass@1Ours
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OursConfig=fine-tuned2026.03 | 213 | 260 | 295 | |
| Leanabell-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 144 | 210 | 231 | |
| Deepseek-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 144 | 203 | 234 | |
| STP-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 131 | 151 | 170 | |
| Goedel-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 88 | 147 | 200 | |
| Kimina-prover-distillCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 66 | 114 | 156 | |
| GPT-4.1-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 54 | 97 | 150 | |
| Deepseek-R1Category=PROPRIETARY REASONING MODELS2026.03 | 51 | 75 | 105 | |
| Gemini-2.5-FlashCategory=PROPRIETARY REASONING MODELS2026.03 | 5 | 9 | 15 | |
| Grok-3-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 4 | 10 | 19 |