Mathematical Reasoning on MATH 500 (pass@1 Accuracy)
94.8Accuracy (pass@1)BF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Model=DeepSeek-R1-Distill-Qwen-32B, Bit-width=BF162025.12 | 94.8 | |
| MIXSDModel backbone=Qwen3-4B-It, Fine-tuning method=MIXSD, Mixing rate λ=0.52026.05 | 94.8 | |
| MIXSDModel backbone=Qwen3-4B-It, Fine-tuning method=MIXSD, Mixing rate λ=0.72026.05 | 94.6 | |
| BaseModel backbone=Qwen3-4B-It, Fine-tuning method=Base2026.05 | 94.2 | |
| MixKVQModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=C2.32025.12 | 94 | |
| BF16Model=DeepSeek-R1-Distill-Qwen-14B, Bit-width=BF162025.12 | 93.6 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 93.4 | |
| RotateKVModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 93.4 | |
| MIXSDModel backbone=Qwen3-4B-It, Fine-tuning method=MIXSD, Mixing rate λ=0.32026.05 | 93 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 92.6 | |
| RotateKVModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 92.4 | |
| MixKVQModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=C2.32025.12 | 92.4 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 92.2 | |
| KVTunerModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=C2.912025.12 | 91.4 | |
| MIXSDModel backbone=Qwen3-4B-It, Fine-tuning method=MIXSD, Mixing rate λ=02026.05 | 90.8 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 90.4 | |
| BF16Model=DeepSeek-R1-Distill-Llama-8B, Bit-width=BF162025.12 | 89.8 | |
| KVTunerModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=C2.902025.12 | 89.8 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV22025.12 | 89.8 | |
| MixKVQModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=C2.72025.12 | 88.2 | |
| RotateKVModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 88 | |
| KVTunerModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=C3.252025.12 | 87.8 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV22025.12 | 87.8 | |
| KVQuantModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 87.4 | |
| KIVIModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 86.6 | |
| AAPO-1.5BZero-shot=true, Number of parameters=1.5B2025.05 | 86 | |
| KIVIModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV22025.12 | 85.8 | |
| Still-3-1.5B-PreviewZero-shot=true, Number of parameters=1.5B2025.05 | 85.5 | |
| MIXSDModel backbone=Qwen3-8B, Fine-tuning method=MIXSD, Mixing rate λ=0.72026.05 | 85.4 | |
| DAPOBase Model=Qwen-2.5 Math-7B2026.06 | 85.2 | |
| DAPO (WITH FORKING TOKENS)Base Model=Qwen-2.5 Math-7B2026.06 | 85.1 | |
| GPG-1.5BZero-shot=true, Number of parameters=1.5B2025.05 | 85 | |
| PS-PPO (OPTIMIZED, B=128)Base Model=Qwen-2.5 Math-7B2026.06 | 85 | |
| S-GRPOBase Model=Qwen-2.5 Math-7B2026.06 | 84.7 | |
| MIXSDModel backbone=Qwen3-8B, Fine-tuning method=MIXSD, Mixing rate λ=0.52026.05 | 84.6 | |
| PS-PPO (TIME-PRIOR)Base Model=Qwen-2.5 Math-7B2026.06 | 84.6 | |
| GRPO-1.5BZero-shot=true, Number of parameters=1.5B2025.05 | 84.4 | |
| DeepSeek-R1-Distill-Qwen-1.5BZero-shot=true, Number of parameters=1.5B2025.05 | 83.9 | |
| GRPOBase Model=Qwen-2.5 Math-7B2026.06 | 83.9 | |
| PS-PPO (HEURISTIC)Base Model=Qwen-2.5 Math-7B2026.06 | 83.7 | |
| ISPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 83.6 | |
| RLOOBase Model=Qwen-2.5 Math-7B2026.06 | 83.6 | |
| PS-PPO (UNIFORM)Base Model=Qwen-2.5 Math-7B2026.06 | 83.6 | |
| DR.GRPOBase Model=Qwen-2.5 Math-7B2026.06 | 83.5 | |
| BaseModel backbone=Qwen3-8B, Fine-tuning method=Base2026.05 | 83.4 | |
| FIXED LENGTH (L=512)Base Model=Qwen-2.5 Math-7B2026.06 | 83.3 | |
| ISPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 83 | |
| BASEBase Model=Qwen-2.5 Math-7B2026.06 | 82.5 | |
| AAPO-7BZero-shot=true, Number of parameters=7B2025.05 | 82.4 | |
| PACRBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 81.9 | |
| Dr. GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 81.8 | |
| ProGRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 80.5 | |
| GTPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 80.2 | |
| GRPO-SBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 80.1 | |
| GPG-7BZero-shot=true, Number of parameters=7B2025.05 | 80 | |
| Oat-Zero-7BZero-shot=true, Number of parameters=7B2025.05 | 80 | |
| MIXSDModel backbone=Qwen3-8B, Fine-tuning method=MIXSD, Mixing rate λ=0.32026.05 | 80 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 80 | |
| ISPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 79.2 | |
| SimpleRL-Zero-7BZero-shot=true, Number of parameters=7B2025.05 | 78.2 | |
| Eurus-2-7B-PRIMEZero-shot=true, Number of parameters=7B2025.05 | 78.2 | |
| PREPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 77.9 | |
| Scaf-GRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 77.8 | |
| PACRBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 77.4 | |
| Dr. GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 76.8 | |
| PREPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 76.3 | |
| PREPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 76.3 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 75.8 | |
| Vanilla GRPOBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 75.3 | |
| OPSDModel backbone=Qwen3-4B-It, Fine-tuning method=OPSD2026.05 | 74.4 | |
| OPSDModel backbone=Qwen3-8B, Fine-tuning method=OPSD2026.05 | 74.2 | |
| MIXSDModel backbone=Qwen3-8B, Fine-tuning method=MIXSD, Mixing rate λ=02026.05 | 73.6 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 73.4 | |
| BaseModel backbone=Qwen3-1.7B, Fine-tuning method=Base2026.05 | 72.4 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-1.5B, Decoding Strategy=Greedy (T=0)2026.06 | 72.2 | |
| p*LLM Family=Qwen 3 (1.7B and 14B)2026.04 | 64 | |
| GRPOModel=Qwen2.5-3B, Training Data=MATH, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 63.6 | |
| GRPO-PVModel=Qwen2.5-3B, Training Data=MATH, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 63.6 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=02026.04 | 61.8 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.22026.04 | 61.6 | |
| INTUITORModel=Qwen2.5-3B, Training Data=MATH, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 61.2 | |
| Imp. rewardLLM Family=Qwen 3 (1.7B and 14B)2026.04 | 60.6 | |
| Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.04 | 60.6 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.12026.04 | 59.8 | |
| Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.04 | 59.6 | |
| NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.04 | 57.8 | |
| INTUITOR-CodeModel=Qwen2.5-3B, Training Data=Codeforces, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 57.2 | |
| GRPOModel=Qwen2.5-1.5B, Training Data=MATH, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 56 | |
| Qwen2.5-Math-7BZero-shot=true, Number of parameters=7B2025.05 | 55.4 | |
| Base modelBase Model=Qwen2.5-7B, Decoding Strategy=Greedy (T=0)2026.06 | 54.7 | |
| BaseModel=Qwen2.5-3B, Training Data=-, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 54.4 | |
| INTUITORModel=Qwen2.5-1.5B, Training Data=MATH, Inference template=chat inference template, Decoding strategy=Greedy2025.05 | 53 | |
| q*LLM Family=Qwen 3 (1.7B and 14B)2026.04 | 53 | |
| NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.04 | 52.8 | |
| MIXSDModel backbone=Qwen3-1.7B, Fine-tuning method=MIXSD, Mixing rate λ=0.32026.05 | 52.2 | |
| pLLM Family=Qwen 3 (1.7B and 14B)2026.04 | 51.8 | |
| Base modelBase Model=Qwen2.5-Math-7B, Decoding Strategy=Greedy (T=0)2026.06 | 50.6 | |
| MIXSDModel backbone=Qwen3-1.7B, Fine-tuning method=MIXSD, Mixing rate λ=0.52026.05 | 47.6 | |
| PS-PPO (OPTIMIZED, B=128)Base Model=Llama-3.1 8B-Instruct2026.06 | 47.6 | |
| DAPO (WITH FORKING TOKENS)Base Model=Llama-3.1 8B-Instruct2026.06 | 47 |