Multi-turn Reasoning on DRIFT-BENCH (test)
68.71Accuracy (%)MUS-Repair
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MUS-RepairModel=gpt-oss-20b2026.04 | 68.71 | 16.2 | 14.52 | 0.02 | 48.3 | |
| MUS-RepairModel=gpt-oss-120b2026.04 | 62.68 | 10.05 | 8.4 | 0.02 | 42.9 | |
| Chain-of-ThoughtModel=gpt-oss-120b2026.04 | 53.95 | 2.04 | 0.36 | 2.95 | 24.2 | |
| LedgerModel=gpt-oss-20b2026.04 | 53.7 | 1.91 | 0.25 | 3.27 | 29.1 | |
| DirectModel=gpt-oss-120b, is_baseline=true2026.04 | 52.12 | — | — | — | 30.2 | |
| DirectModel=gpt-oss-20b, is_baseline=true2026.04 | 51.8 | — | — | — | 23.1 | |
| Chain-of-ThoughtModel=gpt-oss-20b2026.04 | 50.35 | 1.39 | 3.05 | 11.83 | 19.4 | |
| LedgerModel=gpt-oss-120b2026.04 | 50.02 | 2.29 | 4.03 | 2.2 | 36.4 | |
| MUS-RepairModel=Qwen3-32B2026.04 | 38.22 | 9.03 | 6.98 | 0.02 | 15.7 | |
| Chain-of-ThoughtModel=Qwen3-32B2026.04 | 31.44 | 2.54 | 0.73 | 1.87 | 2.5 | |
| LedgerModel=Qwen3-32B2026.04 | 31.44 | 1.63 | 0.93 | 22.04 | 23.6 | |
| MUS-RepairModel=Qwen3-8B2026.04 | 30.01 | 2.03 | 0.32 | 2.95 | 8.8 | |
| DirectModel=Qwen3-32B, is_baseline=true2026.04 | 28.93 | — | — | — | 6.5 | |
| DirectModel=Qwen3-8B, is_baseline=true2026.04 | 28.19 | — | — | — | 5.1 | |
| Chain-of-ThoughtModel=Qwen3-8B2026.04 | 27.91 | 0.19 | 2.01 | 84.22 | 6.7 | |
| LedgerModel=Qwen3-8B2026.04 | 25.23 | 3.14 | 4.99 | 0.34 | 5.7 |