Autoformalization on MiniF2F
100Compilation Pass Rate@10Godel-V2-8B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Godel-V2-8BAugmentation=+CRAMF2025.08 | 100 | 0.4 | |
| Godel-V2-8BAugmentation=Base2025.08 | 99.6 | — | |
| Kimina-7BAugmentation=+CRAMF2025.08 | 99.2 | 0.4 | |
| Kimina-7BAugmentation=Base2025.08 | 98.8 | — | |
| Claude 4.1Augmentation=+CRAMF2025.08 | 98 | 3 | |
| Claude 4.1Augmentation=Base2025.08 | 95.1 | — | |
| Godel-V2-8BAugmentation=CRAMF2025.08 | 95.1 | 7.9 | |
| Herald-7BAugmentation=+CRAMF2025.08 | 93.6 | 18.3 | |
| Claude 4.1Augmentation=CRAMF2025.08 | 92.6 | 10.8 | |
| Gemini 2.5 FlashAugmentation=+CRAMF2025.08 | 89.3 | 8.4 | |
| Godel-V2-8BAugmentation=Base2025.08 | 88.1 | — | |
| Kimina-7BAugmentation=CRAMF2025.08 | 84.8 | 5.6 | |
| Claude 4.1Augmentation=Base2025.08 | 83.6 | — | |
| GPT-4oAugmentation=+CRAMF2025.08 | 82.7 | 18 | |
| Gemini 2.5 FlashAugmentation=Base2025.08 | 82.4 | — | |
| Gemini 2.5 FlashAugmentation=CRAMF2025.08 | 82 | 13.7 | |
| Kimina-7BAugmentation=Base2025.08 | 80.3 | — | |
| Herald-7BAugmentation=Base2025.08 | 79.1 | — | |
| Gemini 2.5 FlashAugmentation=Base2025.08 | 72.1 | — | |
| GPT-4oAugmentation=Base2025.08 | 70.1 | — | |
| Deepseek-V3Augmentation=+CRAMF2025.08 | 69.3 | 32.5 | |
| Herald-7BAugmentation=CRAMF2025.08 | 63.1 | 28.3 | |
| GPT-4oAugmentation=CRAMF2025.08 | 60.1 | 21.2 | |
| Deepseek-V3Augmentation=Base2025.08 | 52.3 | — | |
| GPT-4oAugmentation=Base2025.08 | 49.6 | — | |
| Herald-7BAugmentation=Base2025.08 | 49.2 | — | |
| Deepseek-V3Augmentation=CRAMF2025.08 | 47.1 | 27.6 | |
| Deepseek-V3Augmentation=Base2025.08 | 36.9 | — |