Knowledge-based reasoning on MMLU High School History 1.0 (test)
92.83AccuracyQwQ-32B (Full COT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| QwQ-32B (Full COT)Model=QwQ-32B, Pruning Level=None (Full COT)2025.08 | 92.83 | 1,703.9 | |
| QwQ-32B (80%)Model=QwQ-32B, Pruning Level=80% step-entropy pruning2025.08 | 92.83 | 1,683.9 | |
| QwQ-32B (90%)Model=QwQ-32B, Pruning Level=90% step-entropy pruning2025.08 | 92.83 | 1,683.9 | |
| QwQ-32B (No Thinking)Model=QwQ-32B, Pruning Level=No Thinking (Complete elimination)2025.08 | 91.14 | — | |
| DeepSeek-R1-7B (80%)Pruning level=80%2025.08 | 64.32 | 1,907.5 | |
| DeepSeek-R1-7BPruning level=Full COT2025.08 | 61.74 | 2,054.3 | |
| DeepSeek-R1-7B (90%)Pruning level=90%2025.08 | 61.74 | 1,936.2 | |
| DeepSeek-R1-7BPruning level=No Thinking2025.08 | 47.82 | — |