Correction on FormalRx (test)
4,551ExMatchFormalRx-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FormalRx-8B2026.07 | 4,551 | 571 | 72.9 | |
| Claude-Sonnet-4.6Model Type=Frontier Models2026.07 | 4,146 | 867 | 71.4 | |
| Claude-Sonnet-4Model Type=Frontier Models2026.07 | 3,222 | 962 | 59.5 | |
| GPT-5.3-CodexModel Type=Frontier Models2026.07 | 2,822 | 2,013 | 68.8 | |
| GPT-4.1Model Type=Frontier Models2026.07 | 2,797 | 1,275 | 57.9 | |
| DeepSeek-v3.2Model Type=Frontier Models2026.07 | 2,406 | 1,752 | 59.2 | |
| GPT-5-miniModel Type=Frontier Models2026.07 | 2,299 | 1,955 | 60.5 | |
| Qwen3-8BModel Type=Instruct Models2026.07 | 2,200 | 1,002 | 45.8 | |
| Qwen3-1.7BModel Type=Instruct Models2026.07 | 1,495 | 833 | 33.1 | |
| GPT-5.2Model Type=Frontier Models2026.07 | 977 | 3,084 | 57.8 | |
| Qwen3-4BModel Type=Instruct Models2026.07 | 547 | 1,941 | 35.4 |