Language Understanding on MMLU Pro (Accuracy and Token consumption)
76.4AccuracyCoT-SC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT-SCBackbone=QwQ-32B2026.03 | 76.4 | 0.63 | |
| MAD-M2 (S)Backbone=QwQ-32B, Masking Strategy=Subjective2026.03 | 75.8 | 1.22 | |
| MADBackbone=QwQ-32B2026.03 | 75.4 | 1 | |
| CoTBackbone=QwQ-32B2026.03 | 75.2 | 0.11 | |
| MAD-M2 (O)Backbone=QwQ-32B, Masking Strategy=Objective2026.03 | 75.2 | 0.69 | |
| MAD-M2 (S)Backbone=Qwen2.5-7B-Instruct, Masking Strategy=Subjective2026.03 | 43.6 | 1.17 | |
| MADBackbone=Qwen2.5-7B-Instruct2026.03 | 43 | 1 | |
| MAD-M2 (O)Backbone=Qwen2.5-7B-Instruct, Masking Strategy=Objective2026.03 | 42.4 | 0.69 | |
| CoT-SCBackbone=Qwen2.5-Math-7B-Instruct2026.03 | 41.4 | 0.47 | |
| CoTBackbone=Qwen2.5-Math-7B-Instruct2026.03 | 39.6 | 0.08 | |
| CoT-SCBackbone=Qwen2.5-7B-Instruct2026.03 | 39.2 | 0.51 | |
| MAD-M2 (O)Backbone=Qwen2.5-Math-7B-Instruct, Masking Strategy=Objective2026.03 | 37 | 0.62 | |
| CoTBackbone=Qwen2.5-7B-Instruct2026.03 | 36 | 0.08 | |
| MAD-M2 (S)Backbone=Qwen2.5-Math-7B-Instruct, Masking Strategy=Subjective2026.03 | 35 | 1.36 | |
| MADBackbone=Qwen2.5-Math-7B-Instruct2026.03 | 34.2 | 1 | |
| CoT-SCBackbone=DeepSeek-Math-7B-Instruct2026.03 | 32.2 | 0.99 | |
| MADBackbone=DeepSeek-Math-7B-Instruct2026.03 | 31.2 | 1 | |
| MAD-M2 (S)Backbone=DeepSeek-Math-7B-Instruct, Masking Strategy=Subjective2026.03 | 30.8 | 1.66 | |
| MAD-M2 (O)Backbone=DeepSeek-Math-7B-Instruct, Masking Strategy=Objective2026.03 | 30.8 | 0.75 | |
| CoTBackbone=DeepSeek-Math-7B-Instruct2026.03 | 27.8 | 0.17 |