Verdict Prediction on EPLA
75.38AccuracyClaude-Sonnet-4
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-Sonnet-42026.07 | 75.38 | 66.14 | 43.19 | 52.26 | |
| LeanScorer2026.07 | 74.5 | 68.78 | 33.42 | 44.98 | |
| GPT-5-mini2026.07 | 74.26 | 62.69 | 43.19 | 51.14 | |
| GPT-4.12026.07 | 72.81 | 62.89 | 31.36 | 41.85 | |
| Claude-Sonnet-4.62026.07 | 70.01 | 53.46 | 29.82 | 38.28 | |
| FormalRx-8BSize=8B2026.07 | 69.61 | 51.34 | 49.1 | 50.2 | |
| Qwen3-8BSize=8B2026.07 | 68.08 | 48.93 | 52.7 | 50.74 | |
| GPT-5.3-Codex2026.07 | 67.36 | 47.49 | 43.7 | 45.52 | |
| FormalRx-4BSize=4B2026.07 | 66.96 | 47.4 | 53.98 | 50.48 | |
| Qwen3-4BSize=4B2026.07 | 66.88 | 46.74 | 44.22 | 45.44 | |
| FormalRx-1.7BSize=1.7B2026.07 | 66.24 | 46.97 | 63.75 | 54.09 | |
| GPT-5.22026.07 | 63.27 | 43.78 | 62.47 | 51.48 | |
| DeepSeek-V3.22026.07 | 56.38 | 39.37 | 73.78 | 51.34 | |
| Qwen3-1.7BSize=1.7B2026.07 | 56.3 | 36.07 | 51.93 | 42.57 | |
| BLEU (Best F1)2026.07 | 44.03 | 34.72 | 90.23 | 50.14 |