Ranking LLM solutions on EXPERTMATH 1.0
85.71HumanWinConsequence-Based Utility
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Consequence-Based UtilityBackbone=Qwen3-30B-A3B2026.02 | 85.71 | 90 | 75.79 | 78.37 | 76.24 | |
| Consequence-Based UtilityBackbone=GPT-OSS-120B2026.02 | 82.86 | 90 | 76.27 | 83.04 | 79.63 | |
| Consequence-Based UtilityBackbone=Qwen3-235B-A22B2026.02 | 81.43 | 90 | 73.42 | 74.15 | 71.38 | |
| Consequence-Based UtilityBackbone=GPT-OSS-20B2026.02 | 74.29 | 75.71 | 74.59 | 82.46 | 79.18 | |
| Qwen3-235B-A22BProtocol=LLM-Judge2026.02 | 67.14 | 85.71 | 62.59 | 80.02 | 69.48 | |
| GPT-OSS-20BProtocol=LLM-Judge2026.02 | 52.86 | 82.86 | 72.13 | 72.06 | 69.03 | |
| GPT-OSS-120BProtocol=LLM-Judge2026.02 | 48.57 | 81.43 | 67.21 | 76.91 | 71.42 | |
| Qwen3-30B-A3BProtocol=LLM-Judge2026.02 | 47.14 | 75.71 | 61.3 | 72.4 | 65.81 | |
| Qwen3-235B-GenRMProtocol=Generative Reward Model2026.02 | 27.05 | 77.05 | 65.37 | 71.72 | 67.85 | |
| Llama3.3-Nemotron-49B-GenRMProtocol=Generative Reward Model2026.02 | 25.71 | 31.43 | 43.47 | 55.36 | 49.57 | |
| Qwen2.5-Math-RM-72BProtocol=Generative Reward Model2026.02 | 1.63 | 27.87 | 36.89 | 40.98 | 34.05 | |
| AceMath-72B-RMProtocol=Generative Reward Model2026.02 | 0 | 12.86 | 8.2 | 29.85 | 20.75 |