Scientific Reasoning on GPQA diamond (ACC, TOK, LAT)
4.43LatencyCoT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CoTModel=Qwen3-1.7B2025.07 | 4.43 | — | — | — | 9.91 | — | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=MUR2025.07 | 7.29 | — | — | — | 21.81 | -21.57 | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=SMART2025.07 | 8.14 | — | — | — | 26.66 | 11.2 | |
| CoTModel=Qwen3-4B2025.07 | 8.57 | — | — | — | 15.63 | — | |
| Guided SearchModel=Qwen3-1.7B, Variant=MUR2025.07 | 9.89 | — | — | — | 16.8 | -35.81 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 10.03 | — | — | — | 23.55 | -19.83 | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 10.63 | — | — | — | 15.46 | — | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=MUR2025.07 | 10.96 | — | — | — | 17.75 | 2.29 | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 12.11 | — | — | — | 16.94 | 1.48 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=SMART2025.07 | 13.61 | — | — | — | 34.45 | -8.93 | |
| Guided SearchModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 17.62 | — | — | — | 22.41 | -30.2 | |
| CoTModel=Qwen3-8B2025.07 | 20.13 | — | — | — | 45.38 | — | |
| LLM As a CriticModel=Qwen3-4B, Variant=MUR2025.07 | 20.98 | — | — | — | 38.41 | -20.81 | |
| Guided SearchModel=Qwen3-1.7B, Variant=SMART2025.07 | 21.11 | — | — | — | 37.37 | -15.24 | |
| LLM As a CriticModel=Qwen3-4B, Variant=SMART2025.07 | 21.98 | — | — | — | 37.26 | -21.96 | |
| ϕ-DecodingModel=Qwen3-4B, Variant=MUR2025.07 | 23.79 | — | — | — | 58.06 | -36.95 | |
| Guided SearchModel=Qwen3-4B, Variant=MUR2025.07 | 25.31 | — | — | — | 34.11 | -60.24 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 30.14 | — | — | — | 43.38 | — | |
| Guided SearchModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 31.82 | — | — | — | 52.61 | — | |
| LLM As a CriticModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 32.6 | — | — | — | 78.8 | -0.07 | |
| ϕ-DecodingModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 36.23 | — | — | — | 59.94 | -35.07 | |
| LLM As a CriticModel=Qwen3-8B, Variant=MUR2025.07 | 37.45 | — | — | — | 72.22 | -6.65 | |
| Guided SearchModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 37.66 | — | — | — | 62.78 | -31.57 | |
| LLM As a CriticModel=Qwen3-8B, Variant=SMART2025.07 | 40.23 | — | — | — | 77.09 | -1.78 | |
| LLM As a CriticModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 40.34 | — | — | — | 47.86 | -11.36 | |
| ϕ-DecodingModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 41.22 | — | — | — | 163.97 | -76.13 | |
| LLM As a CriticModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 42.78 | — | — | — | 59.22 | — | |
| LLM As a CriticModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 47.08 | — | — | — | 78.87 | — | |
| Guided SearchModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 47.31 | — | — | — | 114.81 | -92.39 | |
| ϕ-DecodingModel=Qwen3-8B, Variant=MUR2025.07 | 48.56 | — | — | — | 160.16 | -79.94 | |
| InftyThink+Base Model=DeepSeek-R1-Distill-Qwen-1.5B, RL Setting=Task + efficiency reward used2026.02 | 49.87 | 35.46 | 8.69 | — | — | — | |
| InftyThink+RL Setting=Task and efficiency reward (✓ T+E), Sampling Temperature=0.7, Number of Samples=322026.02 | 49.87 | 35.46 | 8.69 | — | — | — | |
| Guided SearchModel=Qwen3-4B, Variant=SMART2025.07 | 51.05 | — | — | — | 96.38 | 2.03 | |
| ϕ-DecodingModel=Qwen3-4B, Variant=SMART2025.07 | 52.68 | — | — | — | 91.93 | -3.08 | |
| Guided SearchModel=Qwen3-8B, Variant=MUR2025.07 | 60.86 | — | — | — | 113.52 | -93.68 | |
| Guided SearchModel=Qwen3-8B, Variant=SMART2025.07 | 70.48 | — | — | — | 137.16 | -70.04 | |
| InftyThink+Base Model=DeepSeek-R1-Distill-Qwen-1.5B, RL Setting=Cold start only (No RL)2026.02 | 74.31 | 32.31 | 11.77 | — | — | — | |
| InftyThink+RL Setting=Cold start only (✗), Sampling Temperature=0.7, Number of Samples=322026.02 | 74.31 | 32.31 | 11.77 | — | — | — | |
| Guided SearchModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 79.32 | — | — | — | 207.2 | — | |
| ϕ-DecodingModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 84.37 | — | — | — | 240.1 | — | |
| ϕ-DecodingModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 85.27 | — | — | — | 95.01 | — | |
| Guided SearchModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 86.06 | — | — | — | 94.35 | — | |
| ϕ-DecodingModel=Qwen3-8B, Variant=SMART2025.07 | 91.84 | — | — | — | 207.32 | -32.78 | |
| VanillaBase Model=DeepSeek-R1-Distill-Qwen-1.5B, RL Setting=Cold start only (No RL)2026.02 | 101.84 | 29.4 | 10.45 | — | — | — | |
| VanillaRL Setting=Cold start only (✗), Sampling Temperature=0.7, Number of Samples=322026.02 | 101.84 | 29.4 | 10.45 | — | — | — | |
| InftyThink+Base Model=DeepSeek-R1-Distill-Qwen-1.5B, RL Setting=Task reward only2026.02 | 149.93 | 37.5 | 24.27 | — | — | — | |
| InftyThink+RL Setting=Task reward only (✓ T), Sampling Temperature=0.7, Number of Samples=322026.02 | 149.93 | 37.5 | 24.27 | — | — | — | |
| InftyThink+Base Model=Qwen3-4B-Base, RL Setting=Task + efficiency reward used2026.02 | 156.09 | 48.17 | 7.58 | — | — | — | |
| InftyThink+Base Model=Qwen3-4B-Base, RL Setting=Cold start only (No RL)2026.02 | 166.54 | 44.65 | 8.05 | — | — | — | |
| VanillaBase Model=DeepSeek-R1-Distill-Qwen-1.5B, RL Setting=Task reward only2026.02 | 197.33 | 29.81 | 15.48 | — | — | — | |
| VanillaRL Setting=Task reward only (✓ T), Sampling Temperature=0.7, Number of Samples=322026.02 | 197.33 | 29.81 | 15.48 | — | — | — | |
| VanillaBase Model=Qwen3-4B-Base, RL Setting=Cold start only (No RL)2026.02 | 254.73 | 45.65 | 8.1 | — | — | — | |
| InftyThink+Base Model=Qwen3-4B-Base, RL Setting=Task reward only2026.02 | 272.24 | 48.99 | 11.89 | — | — | — | |
| VanillaBase Model=Qwen3-4B-Base, RL Setting=Task reward only2026.02 | 579.22 | 47.02 | 12.32 | — | — | — | |
| Avg uncertaintyBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | — | 39.39 | 5,819 | — | — | — | |
| Avg uncertaintyBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | — | 52.53 | 5,561 | — | — | — | |
| Avg uncertaintyBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | — | 55.05 | 6,579 | — | — | — | |
| DeepSeek-V3.2mode=Thinking2026.02 | — | 82.4 | — | 7,000 | — | — | |
| DS V3.2-Thinking2026.02 | — | 82.4 | — | — | — | — | |
| ERNIE 5.02026.02 | — | 86.36 | — | — | — | — | |
| Gemini 2.5-Pro2026.02 | — | 86.4 | — | — | — | — | |
| Gemini 3-Pro2026.02 | — | 91.9 | — | — | — | — | |
| Gemini-3.0version=Pro2026.02 | — | 91.9 | — | 8,000 | — | — | |
| GPT-5 (High)2026.02 | — | 85.7 | — | — | — | — | |
| Kimi K2mode=Thinking2026.02 | — | 84.5 | — | 13,000 | — | — | |
| Kimi K2.52026.02 | — | 87.6 | — | 14,000 | — | — | |
| MURBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | — | 39.9 | 5,231 | — | — | — | |
| MURBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | — | 54.04 | 4,801 | — | — | — | |
| MURBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | — | 57.58 | 6,147 | — | — | — | |
| MURBackbone=Qwen3-30B-A3B, Search Strategy=Guided search2025.07 | — | 49.49 | 855 | — | — | — | |
| MURBackbone=GLM-4-9B, Search Strategy=Guided search2025.07 | — | 32.83 | 1,574 | — | — | — | |
| Non-Thinking ModeBackbone=Qwen3-4B, Reasoning Mode=Non-Thinking Mode2025.07 | — | 39.9 | 612 | — | — | — | |
| Non-Thinking ModeBackbone=Qwen3-8B, Reasoning Mode=Non-Thinking Mode2025.07 | — | 39.9 | 859 | — | — | — | |
| Per-Step ScaleBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | — | 38.89 | 6,032 | — | — | — | |
| Per-Step ScaleBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | — | 51.01 | 6,547 | — | — | — | |
| Per-Step ScaleBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | — | 56.06 | 6,910 | — | — | — | |
| SMARTBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | — | 38.38 | 7,678 | — | — | — | |
| SMARTBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | — | 53.53 | 8,024 | — | — | — | |
| SMARTBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | — | 54.04 | 8,726 | — | — | — | |
| Vanilla CoTBackbone=Qwen3-1.7B, Reasoning Mode=Non-Thinking Mode2025.07 | — | 26.26 | 1,086 | — | — | — |