Scientific Reasoning on GPQA (Acc, Time, Peak, Dep)
40.91AccuracyVanilla
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VanillaBackbone=Llama3.1-8B2026.04 | 40.91 | 7.98 | 6,622 | 26 | |
| LThinker_thoBackbone=Llama3.1-8B2026.04 | 36.36 | 6.38 | 1,796 | 6.4 | |
| LThinker_tokBackbone=Llama3.1-8B2026.04 | 31.82 | 6.94 | 3,150 | 12 | |
| Vanilla + H2OBackbone=Llama3.1-8B2026.04 | 31.81 | 12.55 | 1,536 | 7.9 | |
| AnLLMBackbone=Llama3.1-8B2026.04 | 31.31 | 7.19 | 838 | 3.7 | |
| VanillaBackbone=Qwen2.5-7B2026.04 | 30.81 | 10.76 | 8,055 | 39 | |
| LThinker_thoBackbone=Qwen2.5-7B2026.04 | 30.3 | 8.41 | 2,385 | 9.3 | |
| LThinker_tokBackbone=Qwen2.5-7B2026.04 | 28.28 | 8.26 | 3,940 | 18 | |
| CoTBackbone=Qwen2.5-7B2026.04 | 26.76 | 0.6 | 968 | 0.5 | |
| Distill-R1Backbone=Qwen2.5-7B2026.04 | 24.75 | 8.01 | 6,718 | 31 | |
| Vanilla + H2OBackbone=Qwen2.5-7B2026.04 | 24.75 | 15.61 | 1,200 | 9.8 | |
| CoTBackbone=Llama3.1-8B2026.04 | 24.75 | 0.96 | 1,231 | 0.9 | |
| Distill-R1Backbone=Llama3.1-8B2026.04 | 20.2 | 5.24 | 3,972 | 16 | |
| AnLLMBackbone=Qwen2.5-7B2026.04 | 19.7 | 9.14 | 3,401 | 11 | |
| Vanilla + SepLLMBackbone=Llama3.1-8B2026.04 | 2.53 | 12.62 | 1,024 | 10 | |
| Vanilla + SepLLMBackbone=Qwen2.5-7B2026.04 | 0 | 11.65 | 1,024 | 10 |