Scientific Reasoning on GPQA (Accuracy & Generation Length)
70.2AccuracyMCTS with Const-o-T
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MCTS with Const-o-TBackbone=GPT-4.12025.10 | 70.2 | — | |
| MCTS with CoTBackbone=GPT-4.12025.10 | 67.1 | — | |
| CoT + RSBackbone=GPT-4.12025.10 | 66.6 | — | |
| CoTBackbone=GPT-4.12025.10 | 65.9 | — | |
| CTPBackbone=GPT-4.12025.10 | 65.9 | — | |
| ToTBackbone=GPT-4.12025.10 | 65.6 | — | |
| CoVBackbone=GPT-4.12025.10 | 65.4 | — | |
| MCTSBackbone=GPT-4.12025.10 | 65.1 | — | |
| Const-o-TBackbone=GPT-4.12025.10 | 64.1 | — | |
| MCTS with Const-o-TBackbone=LLaMA-32025.10 | 52 | — | |
| MCTS with CoTBackbone=LLaMA-32025.10 | 50.5 | — | |
| MCTSBackbone=LLaMA-32025.10 | 48.4 | — | |
| Full-CoTBackbone=LLama3.2-3B-Instruction2026.01 | 45.5 | 179.3 | |
| L-ImgCoTBackbone=LLama3.2-3B-Instruction2026.01 | 45.5 | 115.7 | |
| BaseBackbone=Qwen3-4B2025.05 | 45 | — | |
| SHE-LoRABackbone=Qwen3-4B2025.05 | 43.94 | — | |
| L-ImgCoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 43.6 | 130.8 | |
| ImgCoTBackbone=LLama3.2-3B-Instruction2026.01 | 43.6 | 8 | |
| ImgCoT (w/o layout)Backbone=LLama3.2-3B-Instruction2026.01 | 43.6 | 8 | |
| CoVBackbone=LLaMA-32025.10 | 43 | — | |
| Const-o-TBackbone=LLaMA-32025.10 | 42.9 | — | |
| CODIBackbone=Qwen2.5-1.5B-Instruction2026.01 | 42.8 | 6 | |
| Vanilla LoRABackbone=Qwen3-4B2025.05 | 42.57 | — | |
| ToTBackbone=LLaMA-32025.10 | 41.9 | — | |
| ImgCoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 41.8 | 8 | |
| ImgCoT (w/ textual tokens)Backbone=LLama3.2-3B-Instruction2026.01 | 41.8 | 8 | |
| CoT + RSBackbone=LLaMA-32025.10 | 40.9 | — | |
| Full-CoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 40 | 229.5 | |
| ICoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 39.6 | 0 | |
| Evo 8BPost-training=SFT, Number of shots=52026.02 | 39.1 | — | |
| CTPBackbone=LLaMA-32025.10 | 38.5 | — | |
| L-ImgCoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 38.1 | 89.3 | |
| CoLaRBackbone=Qwen2.5-1.5B-Instruction2026.01 | 37.1 | 30.8 | |
| ImgCoT (w/o layout)Backbone=Qwen2.5-1.5B-Instruction2026.01 | 36.4 | 8 | |
| ImgCoT (w/ textual tokens)Backbone=Qwen2.5-1.5B-Instruction2026.01 | 36.4 | 8 | |
| Qwen2.5 7BPost-training=SFT+RL, Number of shots=02026.02 | 36.4 | — | |
| CoTBackbone=LLaMA-32025.10 | 36.3 | — | |
| DeepSeek-R1-1.5BReasoning Strategy=Long-CoT2026.01 | 34.5 | 9,492 | |
| Full-CoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 34.5 | 150.2 | |
| ImgCoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 34.5 | 8 | |
| BD3-LM 7BPost-training=SFT+RL, Number of shots=02026.02 | 34.3 | — | |
| MDLM 9BPost-training=SFT+RL, Number of shots=02026.02 | 32.8 | — | |
| ImgCoT (w/o layout)Backbone=Qwen2.5-0.5B-Instruction2026.01 | 32.7 | 8 | |
| LLaMA3 8BPost-training=SFT+RL, Number of shots=52026.02 | 31.9 | — | |
| LLaDA 8BPost-training=SFT, Number of shots=52026.02 | 31.8 | — | |
| CoconutBackbone=Qwen2.5-1.5B-Instruction2026.01 | 31.4 | 6 | |
| ICoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 30.7 | 0 | |
| ICoTBackbone=LLama3.2-3B-Instruction2026.01 | 28.9 | 0 | |
| RPAMMethod Category=Data-dependent Merging2026.01 | 28.8 | 3,670 | |
| CoconutBackbone=LLama3.2-3B-Instruction2026.01 | 28.6 | 6 | |
| Average MergingMethod Category=Data-free Merging2026.01 | 28.3 | 1,708 | |
| ACMMethod Category=Data-dependent Merging2026.01 | 27.8 | 3,815 | |
| CoLaRBackbone=Qwen2.5-0.5B-Instruction2026.01 | 27.7 | 28.7 | |
| CoDMethod Category=Prompt-guided2026.01 | 27.3 | 5,589 | |
| ImgCoT (w/ textual tokens)Backbone=Qwen2.5-0.5B-Instruction2026.01 | 27.3 | 8 | |
| CODIBackbone=Qwen2.5-0.5B-Instruction2026.01 | 27.1 | 6 | |
| AIMMethod Category=Data-dependent Merging2026.01 | 26.8 | 2,129 | |
| CoconutBackbone=Qwen2.5-0.5B-Instruction2026.01 | 26 | 6 | |
| TIES MergingMethod Category=Data-free Merging2026.01 | 24.2 | 2,188 | |
| ARD 7BPost-training=SFT+RL, Number of shots=52026.02 | 24.2 | — | |
| Task ArithmeticMethod Category=Data-free Merging2026.01 | 20.7 | 1,044 | |
| CoLaRBackbone=LLama3.2-3B-Instruction2026.01 | 20.4 | 33.9 | |
| Ada-R1Method Category=Training-based2026.01 | 20 | 3,655 | |
| Qwen2.5-Math-1.5BReasoning Strategy=Short-CoT2026.01 | 19.7 | 743 | |
| DARE-LinearMethod Category=Data-free Merging2026.01 | 19.2 | 4,569 | |
| Vanilla LoRABackbone=Llama-3.2-3B2025.05 | 13.87 | — | |
| SHE-LoRABackbone=Llama-3.2-3B2025.05 | 13.64 | — | |
| BaseBackbone=Llama-3.2-3B2025.05 | 11.11 | — | |
| DVPOTraining Domain=Math Domain2025.12 | 5.25 | — | |
| Reinforce++Training Domain=Math Domain2025.12 | 4.35 | — | |
| Dr.GRPOTraining Domain=Math Domain2025.12 | 4.35 | — | |
| GRPOTraining Domain=Math Domain2025.12 | 3.99 | — | |
| Robust BellmanTraining Domain=Math Domain2025.12 | 3.62 | — | |
| PPOTraining Domain=Math Domain2025.12 | 3.44 | — | |
| BaseTraining Domain=Math Domain2025.12 | 3.1 | — |