List your company
ResearchFormerly WizWand

Knowledge-based reasoning on MMLU College Medicine 1.0 (test)

86.13AccuracyQwQ-32B (Full COT)
51.113260.204169.29578.3859Aug 5, 2025
Updated 8mo ago

Evaluation results

MethodLinks
QwQ-32B (Full COT)Model=QwQ-32B, Pruning Level=None (Full COT)2025.0886.132,912.3
QwQ-32B (80%)Model=QwQ-32B, Pruning Level=80% step-entropy pruning2025.0884.972,475.9
QwQ-32B (90%)Model=QwQ-32B, Pruning Level=90% step-entropy pruning2025.0884.972,326.4
QwQ-32B (No Thinking)Model=QwQ-32B, Pruning Level=No Thinking (Complete elimination)2025.0884.3—
DeepSeek-R1-7B (80%)Pruning level=80%2025.0862.342,127.9
DeepSeek-R1-7B (90%)Pruning level=90%2025.0862.342,069.4
DeepSeek-R1-7BPruning level=Full COT2025.0861.732,612.7
DeepSeek-R1-7BPruning level=No Thinking2025.0852.46—