Complex Reasoning on BBH (Acc, Time, Peak, Dep)
83.03AccVanilla
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| VanillaBackbone=Llama3.1-8B2026.04 | 83.03 | 6.8 | 2,793 | 5.9 | |
| LThinker_thoBackbone=Llama3.1-8B2026.04 | 79.39 | 7.46 | 911 | 1.9 | |
| Vanilla + H2OBackbone=Llama3.1-8B2026.04 | 78.99 | 11.43 | 1,024 | 2.1 | |
| LThinker_tokBackbone=Llama3.1-8B2026.04 | 74.14 | 7.43 | 1,512 | 2.9 | |
| LThinker_thoBackbone=Qwen2.5-7B2026.04 | 70.3 | 7.71 | 1,151 | 2.7 | |
| Vanilla + H2OBackbone=Qwen2.5-7B2026.04 | 70.1 | 15.61 | 1,024 | 3.5 | |
| VanillaBackbone=Qwen2.5-7B2026.04 | 69.9 | 11.5 | 3,786 | 13 | |
| AnLLMBackbone=Llama3.1-8B2026.04 | 68.89 | 9.79 | 621 | 1.6 | |
| CoTBackbone=Llama3.1-8B2026.04 | 66.46 | 0.93 | 642 | 0.2 | |
| CoTBackbone=Qwen2.5-7B2026.04 | 65.45 | 0.68 | 570 | 0.1 | |
| LThinker_tokBackbone=Qwen2.5-7B2026.04 | 62.83 | 8.95 | 1,884 | 5.6 | |
| Distill-R1Backbone=Llama3.1-8B2026.04 | 61.21 | 0.83 | 380 | 0.2 | |
| Distill-R1Backbone=Qwen2.5-7B2026.04 | 57.78 | 5.53 | 1,967 | 6 | |
| AnLLMBackbone=Qwen2.5-7B2026.04 | 54.95 | 10.04 | 1,303 | 3.8 | |
| Vanilla + SepLLMBackbone=Llama3.1-8B2026.04 | 14.55 | 27.14 | 1,024 | 8.5 | |
| Vanilla + SepLLMBackbone=Qwen2.5-7B2026.04 | 8.08 | 26.64 | 1,024 | 9.4 |