Mathematical Problem Solving on MATH (Accuracy & Cost)
95.7AccuracyOracle (Perfect Knowledge)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Oracle (Perfect Knowledge)Model=ensemble2026.02 | 95.7 | 6.37 | |
| Probe Routerlambda=0, Model=ensemble2026.02 | 93 | 28.37 | |
| SFTModel=DS-R1-Qwen-32B2026.04 | 92.65 | — | |
| D-QRELOModel=DS-R1-Qwen-32B2026.04 | 92.45 | — | |
| SFTModel=DS-R1-Qwen-14B2026.04 | 92.25 | — | |
| gpt-oss-20b-highModel=gpt-oss-20b (high)2026.02 | 92 | 40 | |
| Probe Routerlambda=0.2, Model=ensemble2026.02 | 91.7 | 10.35 | |
| gpt-oss-20b-mediumModel=gpt-oss-20b (medium)2026.02 | 91.4 | 11.78 | |
| R1-Qwen-7BModel=DeepSeek-R1-Distill-Qwen-7B2026.02 | 91.4 | 18.43 | |
| D-QRELOModel=DS-R1-Qwen-14B2026.04 | 91.4 | — | |
| Probe Routerlambda=0.4, Model=ensemble2026.02 | 90.9 | 8.29 | |
| Probe Routerlambda=0.6, Model=ensemble2026.02 | 90.2 | 7.02 | |
| SFTModel=DS-R1-LLaMA-8B2026.04 | 89.55 | — | |
| Probe Routerlambda=0.8, Model=ensemble2026.02 | 89.4 | 5.98 | |
| Probe Routerlambda=1, Model=ensemble2026.02 | 89 | 5.34 | |
| Random RoutingModel=ensemble2026.02 | 88.9 | 15.43 | |
| Delta-CoMeModel=DS-R1-Qwen-32B2026.04 | 88.1 | — | |
| Delta-CoMeModel=DS-R1-Qwen-14B2026.04 | 87.7 | — | |
| gpt-oss-20b-lowModel=gpt-oss-20b (low)2026.02 | 86.6 | 4 | |
| Math-7BModel=Qwen2.5-Math-7B-Instruct2026.02 | 82.7 | 3.28 | |
| D-QRELOModel=DS-R1-LLaMA-8B2026.04 | 79.05 | — | |
| MagnitudeModel=DS-R1-Qwen-32B2026.04 | 78.65 | — | |
| WandaModel=DS-R1-Qwen-32B2026.04 | 78.4 | — | |
| WandaModel=DS-R1-Qwen-14B2026.04 | 78.2 | — | |
| MagnitudeModel=DS-R1-Qwen-14B2026.04 | 76.35 | — | |
| BackboneModel=DS-R1-Qwen-14B2026.04 | 64.15 | — | |
| BackboneModel=DS-R1-Qwen-32B2026.04 | 62.4 | — | |
| SVDModel=DS-R1-Qwen-32B2026.04 | 60.3 | — | |
| RandomModel=DS-R1-Qwen-14B2026.04 | 58.45 | — | |
| RandomModel=DS-R1-Qwen-32B2026.04 | 57.3 | — | |
| SVDModel=DS-R1-Qwen-14B2026.04 | 53.9 | — | |
| Delta-CoMeModel=DS-R1-LLaMA-8B2026.04 | 45.4 | — | |
| SVDModel=DS-R1-LLaMA-8B2026.04 | 39.2 | — | |
| WandaModel=DS-R1-LLaMA-8B2026.04 | 24.7 | — | |
| BitDeltaModel=DS-R1-Qwen-32B2026.04 | 22.75 | — | |
| BitDeltaModel=DS-R1-Qwen-14B2026.04 | 20.95 | — | |
| MagnitudeModel=DS-R1-LLaMA-8B2026.04 | 18.4 | — | |
| BitDeltaModel=DS-R1-LLaMA-8B2026.04 | 11.8 | — | |
| RandomModel=DS-R1-LLaMA-8B2026.04 | 7.65 | — | |
| BackboneModel=DS-R1-LLaMA-8B2026.04 | 2.6 | — |