List your company
ResearchFormerly WizWand

General Knowledge (STEM) on MMLU STEM

74.1AccuracyBase Model
15.8630.9846.161.22Feb 9, 2026Feb 11, 2026Feb 14, 2026Feb 17, 2026Feb 19, 2026Feb 22, 2026Feb 25, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Base ModelBackbone=Qwen3-1.7B2026.0274.11,081—
Extra-CoTBackbone=Qwen3-1.7B, Ratio=0.82026.0271.31,2671.17
Extra-CoTBackbone=Qwen3-1.7B, Ratio=1.02026.0270.51,2521.16
Extra-CoTBackbone=Qwen3-1.7B, Ratio=<POLICY>2026.0267.51,0260.95
Extra-CoT (CHRPO)Backbone=Qwen3-1.7B, Ratio=<POLICY>2026.0267.13300.31
Extra-CoTBackbone=Qwen3-1.7B, Ratio=0.42026.0261.37300.67
Extra-CoTBackbone=Qwen3-1.7B, Ratio=0.62026.0260.61,0120.94
TokenSkipBackbone=Qwen3-1.7B, Ratio=1.02026.02561,5561.44
TokenSkipBackbone=Qwen3-1.7B, Ratio=0.82026.0254.51,4811.37
Extra-CoTBackbone=Qwen3-1.7B, Ratio=0.22026.0254.34580.42
Gemma2-9BRole=Teacher2026.0250.2——
TokenSkipBackbone=Qwen3-1.7B, Ratio=0.62026.0247.51,3101.21
Qwen3-8B (teacher)Role=Teacher2026.0240.2——
TSD-KDTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0237.5——
Sequence KDTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0236.9——
DistiLLMTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0236.8——
GKDTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0236.7——
Supervised KDTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0236.5——
MiniLLMTeacher=Qwen3-8B, Student=Qwen3-1.7B2026.0236.5——
Qwen3-1.7B (student)Role=Student2026.0236.2——
TokenSkipBackbone=Qwen3-1.7B, Ratio=<POLICY>2026.0232.61,0350.96
TokenSkipBackbone=Qwen3-1.7B, Ratio=0.42026.0229.88770.81
TSD-KD2026.0227——
GKDβ=0.92026.0226.2——
Sequence-Level KD2026.0225.9——
MiniLLM2026.0225.9——
Gemma2-2BRole=Student2026.0225.5——
Supervised-KD2026.0225.5——
DistiLLM2026.0225.2——
TokenSkipBackbone=Qwen3-1.7B, Ratio=0.22026.0218.14460.41