Physics Reasoning on arXiv human-adapted
60.4Accuracy (hep-th)Claude-Opus-4.5
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude-Opus-4.5Model Category=Proprietary Models2026.04 | 60.4 | 58.2 | 70 | 80 | 63.7 | 58.4 | 65.8 | 65.1 | |
| Gemini-2.5-flashModel Category=Proprietary Models2026.04 | 53.7 | 50.9 | 66.2 | 71.5 | 61.6 | 57.6 | 75 | 62.3 | |
| OSS-20bModel Category=Open-Weight Models2026.04 | 39.8 | 45.5 | 56.7 | 53.8 | 41.1 | 50.4 | 56.5 | 48.8 | |
| Qwen3-4B-Thinking-2507Model Category=Open-Weight Models2026.04 | 23.7 | 24.5 | 41.8 | 47.7 | 43.2 | 42.5 | 36.2 | 35.4 | |
| Qwen3-4B-Instruct-2507Model Category=Open-Weight Models2026.04 | 18.5 | 14.5 | 33.6 | 27.7 | 31.1 | 28 | 20 | 24.9 | |
| DeepSeek-7BModel Category=Open-Weight Models2026.04 | 12.4 | 9.1 | 23.8 | 10 | 22.6 | 16.7 | 15.4 | 16.6 |