Multi-task Language Understanding on MMLU (Accuracy, Time, Peak)
71.08AccuracyVanilla
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VanillaBackbone=Llama3.1-8B Series2026.06 | 71.08 | 42.86 | 2,800 | |
| CoTBackbone=Qwen2.5-7B Series2026.06 | 70.01 | 3.88 | 676 | |
| Vanilla + H2OBackbone=Llama3.1-8B Series2026.06 | 69.52 | 90.18 | 1,024 | |
| VanillaBackbone=Qwen2.5-7B Series2026.06 | 65.92 | 28.89 | 2,241 | |
| Vanilla + SepLLMBackbone=Llama3.1-8B Series2026.06 | 64.85 | 49.03 | 1,024 | |
| HybridThinkerBackbone=Qwen2.5-7B Series2026.06 | 63.39 | 23.07 | 1,009 | |
| HybridThinkerBackbone=Llama3.1-8B Series2026.06 | 62.71 | 31.54 | 1,053 | |
| Vanilla + H2OBackbone=Qwen2.5-7B Series2026.06 | 62.32 | 76.01 | 1,024 | |
| LightThinkerBackbone=Llama3.1-8B Series2026.06 | 60.86 | 33.45 | 932 | |
| LightThinkerBackbone=Qwen2.5-7B Series2026.06 | 60.66 | 22.52 | 827 | |
| CoTBackbone=Llama3.1-8B Series2026.06 | 60.18 | 5.13 | 698 | |
| Vanilla + SepLLMBackbone=Qwen2.5-7B Series2026.06 | 59.2 | 66.76 | 1,024 | |
| Distill-R1Backbone=Qwen2.5-7B Series2026.06 | 31.26 | 15.38 | 1,289 | |
| Distill-R1Backbone=Llama3.1-8B Series2026.06 | 20.93 | 23.64 | 1,611 |