Counterexample generation on FOR-COUNTER
222Pass@1Ours
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OursConfig=fine-tuned2026.03 | 222 | 274 | 302 | |
| Deepseek-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 127 | 200 | 224 | |
| Leanabell-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 106 | 159 | 181 | |
| STP-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 101 | 157 | 179 | |
| Goedel-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 89 | 177 | 215 | |
| Deepseek-R1Category=PROPRIETARY REASONING MODELS2026.03 | 61 | 135 | 158 | |
| Kimina-prover-distillCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 31 | 109 | 165 | |
| Grok-3-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 30 | 74 | 101 | |
| Gemini-2.5-FlashCategory=PROPRIETARY REASONING MODELS2026.03 | 21 | 66 | 82 | |
| GPT-4.1-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 19 | 65 | 103 |