General Multitask Language Understanding on MMLU-Pro (Pass@1, # Tokens, Saved %)
989.8Avg TokensTRACES
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TRACESthreshold=0.9, Backbone=DS-Qwen14B, Seed=422026.04 | 989.8 | 63.4 | 60.44 | |
| TRACESthreshold=0.8, Backbone=DS-Qwen14B, Seed=422026.04 | 1,286.9 | 65 | 48.56 | |
| TRACESthreshold=0.7, Backbone=DS-Qwen14B, Seed=422026.04 | 1,614.7 | 65.2 | 35.46 | |
| Basel. P^{(1)}_{system}Backbone=DS-Qwen14B, Seed=422026.04 | 1,705.6 | 63.7 | 31.83 | |
| TRACESthreshold=0.6, Backbone=DS-Qwen14B, Seed=422026.04 | 1,766.3 | 66.5 | 29.4 | |
| TRACESthreshold=0.5, Backbone=DS-Qwen14B, Seed=422026.04 | 1,880.6 | 66.4 | 24.83 | |
| TRACESthreshold=0.4, Backbone=DS-Qwen14B, Seed=422026.04 | 1,923.1 | 66.9 | 23.13 | |
| Basel. P^{(3)}_{system}Backbone=DS-Qwen14B, Seed=422026.04 | 1,935.2 | 61.3 | 22.65 | |
| Basel. P^{(0)}_{user}Backbone=DS-Qwen14B, Seed=422026.04 | 2,194.1 | 66.5 | 12.3 | |
| Basel. P^{(0)}_{system}Backbone=DS-Qwen14B, Seed=422026.04 | 2,292.9 | 67.5 | 8.35 | |
| StandardBackbone=DS-Qwen14B, Seed=422026.04 | 2,501.9 | 65.2 | — |