Counterexample generation on VERI-FORMALIZE
174Pass@1Ours
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OursConfig=fine-tuned2026.03 | 174 | 255 | 313 | |
| Leanabell-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 111 | 198 | 228 | |
| STP-proverCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 99 | 150 | 171 | |
| Deepseek-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 69 | 135 | 186 | |
| Goedel-prover-v2Category=OPEN-SOURCED NEURAL PROVERS2026.03 | 63 | 165 | 201 | |
| GPT-4.1-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 31 | 87 | 137 | |
| Deepseek-R1Category=PROPRIETARY REASONING MODELS2026.03 | 27 | 72 | 102 | |
| Kimina-prover-distillCategory=OPEN-SOURCED NEURAL PROVERS2026.03 | 18 | 141 | 249 | |
| Gemini-2.5-FlashCategory=PROPRIETARY REASONING MODELS2026.03 | 2 | 8 | 13 | |
| Grok-3-miniCategory=PROPRIETARY REASONING MODELS2026.03 | 2 | 8 | 19 |