Logical Reasoning on BBH (Accuracy, Time, Peak Memory)
81.62AccuracyVanilla + H2O
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Vanilla + H2OBackbone=Llama3.1-8B Series2026.06 | 81.62 | 23.81 | 1,024 | |
| VanillaBackbone=Qwen2.5-7B Series2026.06 | 81.01 | 10.59 | 2,212 | |
| HybridThinkerBackbone=Llama3.1-8B Series2026.06 | 80 | 11.73 | 1,031 | |
| VanillaBackbone=Llama3.1-8B Series2026.06 | 78.18 | 13.81 | 2,515 | |
| Vanilla + H2OBackbone=Qwen2.5-7B Series2026.06 | 75.76 | 25.01 | 1,024 | |
| HybridThinkerBackbone=Qwen2.5-7B Series2026.06 | 73.13 | 9.87 | 1,058 | |
| LightThinkerBackbone=Llama3.1-8B Series2026.06 | 72.93 | 13.58 | 999 | |
| Vanilla + SepLLMBackbone=Llama3.1-8B Series2026.06 | 72.32 | 15.48 | 1,024 | |
| CoTBackbone=Qwen2.5-7B Series2026.06 | 71.31 | 1.27 | 582 | |
| Vanilla + SepLLMBackbone=Qwen2.5-7B Series2026.06 | 69.7 | 28.36 | 1,024 | |
| LightThinkerBackbone=Qwen2.5-7B Series2026.06 | 66.46 | 10.91 | 980 | |
| CoTBackbone=Llama3.1-8B Series2026.06 | 53.13 | 1.39 | 596 | |
| Distill-R1Backbone=Qwen2.5-7B Series2026.06 | 51.31 | 3.71 | 905 | |
| Distill-R1Backbone=Llama3.1-8B Series2026.06 | 29.49 | 5.15 | 1,146 |