Mathematical Reasoning on MATH (pass@1, maj@8, rm@8)
81.1Pass@1GPT-4o-2024-08-06
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4o-2024-08-06Reasoning Mode=Chain-of-Thought, Shots=0-shot2024.09 | 81.1 | — | — | — | |
| Qwen2.5-Math-1.5B-InstructReasoning Mode=Tool-Integrated Reasoning, Shots=0-shot2024.09 | 79.9 | 85.3 | 88.9 | — | |
| GRPOBackbone=Qwen, Model Size=7B2026.01 | 78.2 | — | — | — | |
| Qwen2.5-Math-1.5B-InstructReasoning Mode=Chain-of-Thought, Shots=0-shot2024.09 | 75.8 | 80.3 | 83.9 | — | |
| One-Shot RLBackbone=Qwen, Model Size=7B2026.01 | 74 | — | — | — | |
| DoPRBackbone=Qwen, Model Size=7B2026.01 | 73.8 | — | — | — | |
| UFOBackbone=Qwen, Model Size=7B2026.01 | 73.5 | — | — | — | |
| DoPRBackbone=Qwen, Model Size=1.5B2026.01 | 73.5 | — | — | — | |
| UFOBackbone=Qwen, Model Size=1.5B2026.01 | 73.3 | — | — | — | |
| GRPOBackbone=Qwen, Model Size=1.5B2026.01 | 71.8 | — | — | — | |
| One-Shot RLBackbone=Qwen, Model Size=1.5B2026.01 | 66.6 | — | — | — | |
| Qwen2.5-baseModel Size=7B, RL Strategy=Base2026.01 | 53.4 | — | — | — | |
| Qwen2.5-Math-baseModel Size=7B, RL Strategy=Base2026.01 | 44.6 | — | — | — | |
| One-Shot RLBackbone=LLaMA, Model Size=8B2026.01 | 28 | — | — | — | |
| DoPRBackbone=LLaMA, Model Size=8B2026.01 | 28 | — | — | — | |
| UFOBackbone=LLaMA, Model Size=8B2026.01 | 26.8 | — | — | — | |
| GRPOBackbone=LLaMA, Model Size=8B2026.01 | 25.8 | — | — | — | |
| Qwen2.5-Math-baseModel Size=1.5B, RL Strategy=Base2026.01 | 22.6 | — | — | — | |
| CorDAnumber of parameters=320M, backbone=LLaMA-2-7B, adaptation mode=instruction-previewed adaptation mode, rank=1282024.06 | 8.52 | — | — | 22.55 | |
| PiSSAnumber of parameters=320M, backbone=LLaMA-2-7B, adaptation mode=instruction-previewed adaptation mode, rank=1282024.06 | 7.6 | — | — | 21.46 | |
| Full fine-tuningnumber of parameters=6738M, backbone=LLaMA-2-7B, adaptation mode=instruction-previewed adaptation mode2024.06 | 7.48 | — | — | 22.46 | |
| DoRAnumber of parameters=321M, backbone=LLaMA-2-7B, adaptation mode=instruction-previewed adaptation mode, rank=1282024.06 | 6.2 | — | — | 18.18 | |
| LoRAnumber of parameters=320M, backbone=LLaMA-2-7B, adaptation mode=instruction-previewed adaptation mode, rank=1282024.06 | 5.92 | — | — | 18.3 | |
| Qwen2.5-baseModel Size=1.5B, RL Strategy=Base2026.01 | 3.2 | — | — | — | |
| AutoTIRCategory=RL-only TIR Methods2026.01 | — | — | — | 90 | |
| AutoTrajCategory=SFT-RL TIR Methods2026.01 | — | — | — | 92 | |
| Qwen2.5-7B-InstructFramework=Multi-Dimensional Trajectory Evaluation2026.01 | — | — | — | 90 | |
| R1-SearcherCategory=RL-only TIR Methods2026.01 | — | — | — | 88 | |
| ReSearchCategory=RL-only TIR Methods2026.01 | — | — | — | 90 | |
| Tool-StarCategory=SFT-RL TIR Methods2026.01 | — | — | — | 84 | |
| Tool-Star-SFTCategory=SFT-only TIR Methods2026.01 | — | — | — | 92 | |
| ToRLCategory=RL-only TIR Methods2026.01 | — | — | — | 73 | |
| Vanilla SFT-RL TIRCategory=SFT-RL TIR Methods2026.01 | — | — | — | 81 |