Mathematical Reasoning on GSM8K (test) (Accuracy, Cost)
97.8AccuracyOracle (Perfect Knowledge)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Oracle (Perfect Knowledge)Model=ensemble2026.02 | 97.8 | 0.4 | |
| Probe Routerlambda=0.2, Model=ensemble2026.02 | 94.6 | 0.55 | |
| Math-7BModel=Qwen2.5-Math-7B-Instruct2026.02 | 94.5 | 0.43 | |
| Probe Routerlambda=0.4, Model=ensemble2026.02 | 94.5 | 0.47 | |
| gpt-oss-20b-highModel=gpt-oss-20b (high)2026.02 | 94.4 | 2.14 | |
| Probe Routerlambda=0.6, Model=ensemble2026.02 | 94.2 | 0.43 | |
| Probe Routerlambda=0.8, Model=ensemble2026.02 | 94.1 | 0.42 | |
| Probe Routerlambda=0, Model=ensemble2026.02 | 94 | 0.95 | |
| Probe Routerlambda=1, Model=ensemble2026.02 | 93.9 | 0.41 | |
| gpt-oss-20b-mediumModel=gpt-oss-20b (medium)2026.02 | 93.3 | 0.82 | |
| Random RoutingModel=ensemble2026.02 | 91.8 | 0.99 | |
| gpt-oss-20b-lowModel=gpt-oss-20b (low)2026.02 | 90 | 0.39 | |
| R1-Qwen-7BModel=DeepSeek-R1-Distill-Qwen-7B2026.02 | 88.4 | 1.39 |