Autoformalization on ProofNet (test)
44.09πFVMonotonic Reference-Free Refinement
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=22026.01 | 44.09 | 32.93 | 59.14 | 47.58 | 28.41 | — | — | |
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=32026.01 | 44.09 | 32.93 | 58.96 | 49.46 | 28.97 | — | — | |
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=42026.01 | 44.09 | 33.33 | 59.5 | 49.73 | 29.38 | — | — | |
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=52026.01 | 44.09 | 33.87 | 59.68 | 50.27 | 29.79 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=22026.01 | 44.09 | 13.58 | 29.21 | 8.87 | 8.44 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=32026.01 | 44.09 | 12.9 | 28.85 | 8.6 | 8 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=42026.01 | 44.09 | 12.9 | 29.03 | 8.33 | 7.97 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=52026.01 | 44.09 | 12.9 | 29.21 | 8.33 | 8.03 | — | — | |
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=12026.01 | 43.55 | 30.78 | 58.24 | 45.16 | 26.4 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=12026.01 | 43.55 | 13.98 | 29.39 | 8.33 | 8.15 | — | — | |
| Monotonic Reference-Free RefinementJudges=GPT-4.1-mini, Time Step (t)=02026.01 | 29.57 | 26.34 | 55.91 | 37.63 | 16.52 | — | — | |
| Monotonic Reference-Free RefinementJudges=Qwen2.5-Coder-7B, Time Step (t)=02026.01 | 29.57 | 11.96 | 29.39 | 7.8 | 5.18 | — | — | |
| DRIFTRetrieval Strategy=DRIFT, Model=Claude-Opus-42025.10 | — | — | — | — | — | 78.61 | 19.79 | |
| Oracle*Retrieval Strategy=Oracle*, Model=Claude-Opus-42025.10 | — | — | — | — | — | 81.28 | 26.2 | |
| RAutoRetrieval Strategy=RAuto, Model=Claude-Opus-42025.10 | — | — | — | — | — | 75.67 | 19.25 | |
| Zero-shotRetrieval Strategy=Zero-shot, Model=Claude-Opus-42025.10 | — | — | — | — | — | 68.45 | 17.65 |