Reasoning on MuSR (test)
73.9AccuracyCorrectness Only
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Correctness OnlyModel=Phi4-reasoning-plus2026.02 | 73.9 | 62 | |
| MAT-SteerModel=Phi4-reasoning-plus2026.02 | 73.1 | 66.4 | |
| OriginalModel=Phi4-reasoning-plus2026.02 | 72.3 | 65.1 | |
| REMULModel=Phi4-reasoning-plus2026.02 | 71.6 | 67.7 | |
| Balanced RewardsModel=Phi4-reasoning-plus2026.02 | 69.1 | 63.5 | |
| Faithfulness OnlyModel=Phi4-reasoning-plus2026.02 | 67 | 68.2 | |
| MAT-SteerModel=Qwen3-14b2026.02 | 66.2 | 78.3 | |
| OriginalModel=Qwen3-14b2026.02 | 65.8 | 76.4 | |
| REMULModel=Qwen3-14b2026.02 | 65.8 | 79.5 | |
| Correctness OnlyModel=Qwen3-14b2026.02 | 65.4 | 72.8 | |
| Hint OptimizedModel=Phi4-reasoning-plus2026.02 | 65.3 | 73 | |
| Balanced RewardsModel=Qwen3-14b2026.02 | 62.2 | 74.5 | |
| Faithfulness OnlyModel=Qwen3-14b2026.02 | 60.5 | 80.1 | |
| Hint OptimizedModel=Qwen3-14b2026.02 | 58.7 | 87.6 | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=strict2025.06 | 48.94 | — | |
| Dense baselineModel Type=Dense, Compute=5.45e212025.06 | 35.98 | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=-2025.06 | 35.58 | — |