Categorization on FormalRx (test)
64.4AccuracyFormalRx-8B
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FormalRx-8B2026.07 | 64.4 | 90.5 | 58.3 | 70.9 | |
| Claude-Sonnet-4.6Model Type=Frontier Models2026.07 | 44.8 | 81.4 | 33.5 | 47.5 | |
| GPT-5.3-CodexModel Type=Frontier Models2026.07 | 37.6 | 73.4 | 25.6 | 38 | |
| Claude-Sonnet-4Model Type=Frontier Models2026.07 | 33.8 | 63.3 | 26.7 | 37.6 | |
| GPT-5-miniModel Type=Frontier Models2026.07 | 33.5 | 61.2 | 29.4 | 39.7 | |
| GPT-4.1Model Type=Frontier Models2026.07 | 31.5 | 65.6 | 16.9 | 26.9 | |
| GPT-5.2Model Type=Frontier Models2026.07 | 27.3 | 52.9 | 22.1 | 31.1 | |
| Qwen3-8BModel Type=Instruct Models2026.07 | 16.5 | 31.6 | 10.3 | 15.5 | |
| DeepSeek-v3.2Model Type=Frontier Models2026.07 | 15.7 | 37.6 | 19.8 | 25.9 | |
| Qwen3-4BModel Type=Instruct Models2026.07 | 7.4 | 20.4 | 8.4 | 11.9 | |
| Qwen3-1.7BModel Type=Instruct Models2026.07 | 7.1 | 17 | 6.3 | 9.2 |