Mathematical Reasoning on GSM8k (Accuracy)
100AccuracyPhi-4 pass@N (Upper Bound)
Evaluation Results
| Method | Links | |
|---|---|---|
| Phi-4 pass@N (Upper Bound)# Sample=-2025.11 | 100 | |
| Majority Voting# Sample=-2025.11 | 100 | |
| H4-Qwen2.5-PRM-1.5B-0.2# Sample=369K2025.11 | 100 | |
| Universal-PRM-Qwen2.5-Math-7B# Sample=690K2025.11 | 100 | |
| Qwen2.5-Math-7B-PRM800k# Sample=263K2025.11 | 100 | |
| Qwen2.5-Math-7B-PRM# Sample=860K2025.11 | 100 | |
| ReProbe, Attn+Logit, Qwen3-8B-anno# Sample=32K2025.11 | 100 | |
| Skywork-PRM-1.5B# Sample=Unk2025.11 | 99.5 | |
| RLHFlow-PRM-Deepseek-Data# Sample=253K2025.11 | 98.9 | |
| Math-Shepherd-PRM-7B# Sample=440K2025.11 | 98.4 | |
| RLHFlow-PRM-Mistral-Data# Sample=273K2025.11 | 98.4 | |
| Phi-4 pass@1 (Lower Bound)# Sample=-2025.11 | 97.9 | |
| SQ-formatModel=DeepSeek-R1, Setting=W(SQ5)A8, Sparsity=0.875, Evaluation Protocol=Generative2025.12 | 96.21 | |
| DeepSeek-R1 (BF16)Model=DeepSeek-R1, Setting=W16A16, Sparsity=0, Evaluation Protocol=Generative2025.12 | 95.83 | |
| ZEDABase Model=Qwen3-30B-A3B2026.05 | 95.5 | |
| Qwen3-30B-A3BBase Model=Qwen3-30B-A3B2026.05 | 95.4 | |
| NETSFT→OPDBase Model=Qwen3-30B-A3B2026.05 | 95.4 | |
| Dynamic SkippingBase Model=Qwen3-30B-A3B2026.05 | 95.2 | |
| GLM-4.7-FlashBase Model=GLM-4.7-Flash2026.05 | 95.2 | |
| ZEDASFTBase Model=GLM-4.7-Flash2026.05 | 95.2 | |
| ZEDASFTBase Model=Qwen3-30B-A3B2026.05 | 94.8 | |
| NETSFTBase Model=Qwen3-30B-A3B2026.05 | 94.7 | |
| ZEDABase Model=GLM-4.7-Flash2026.05 | 94.4 | |
| NETSFT→OPDBase Model=GLM-4.7-Flash2026.05 | 94.2 | |
| NETSFTBase Model=GLM-4.7-Flash2026.05 | 94.1 | |
| AdaMoEBase Model=GLM-4.7-Flash2026.05 | 93.9 | |
| Dynamic SkippingBase Model=GLM-4.7-Flash2026.05 | 93.8 | |
| SRQ-driven steeringBackbone=Qwen3-4B-Thinking, Steering Mechanism=Conceptor2026.04 | 93.3 | |
| SRQ-driven steeringBackbone=Qwen3-4B-Thinking, Steering Mechanism=PCA-CAA2026.04 | 93.1 | |
| PPoT + Qwen2.5 CoderModel Size=7B, k (LLM samples)=5, m (PPoT samples)=52026.04 | 93.02 | |
| DS-R1-Distill-Qwen-7B-ScaleQuestModel=DS-R1-Distill-Qwen-7B-ScaleQuest, Data=ScaleQuest2024.10 | 93 | |
| SRQ-driven steeringBackbone=Qwen3-4B-Thinking, Steering Mechanism=CAA2026.04 | 93 | |
| AUTOIF (LLaMA-3-70B w/ Online DPO)Backbone Model=LLaMA3-70B-Instruct, Supervision Model=LLaMA3-70B, Alignment Strategy=Self-Alignment, Training Method=Online DPO2024.06 | 92.7 | |
| LLaMA3-70B-InstructBackbone Model=LLaMA3-70B-Instruct2024.06 | 92.6 | |
| ConceptorBackbone=Qwen3-4B-Thinking2026.04 | 92.6 | |
| PCA-CAABackbone=Qwen3-4B-Thinking2026.04 | 92.4 | |
| AdaMoEBase Model=Qwen3-30B-A3B2026.05 | 92.4 | |
| CAABackbone=Qwen3-4B-Thinking2026.04 | 92.2 | |
| GPT-42024.03 | 92 | |
| Base LLMBackbone=Qwen3-4B-Thinking2026.04 | 91.9 | |
| VanillaTraining Strategy=Vanilla, Backbone=Qwen-2.5-32B-Instruct2026.01 | 91.66 | |
| DS-R1-Distill-Qwen-7BModel=DS-R1-Distill-Qwen-7B2024.10 | 91.5 | |
| Ans. AugTraining Strategy=Answer-Based Augmentation, Backbone=Qwen-2.5-32B-Instruct2026.01 | 91.5 | |
| Qwen2.5 CoderModel Size=7B, k (LLM samples)=5, m (PPoT samples)=02026.04 | 91.2 | |
| SATTraining Strategy=Structure-Aware Training, Backbone=Qwen-2.5-32B-Instruct2026.01 | 91 | |
| BF16Model=Qwen-3 30B-A3B, Setting=W16A162025.12 | 90.45 | |
| ES + TT-MVModel=OLMo3-7B-Inst2026.03 | 90.2 | |
| SRQ-driven steeringBackbone=R1-Distill-Qwen-7B, Steering Mechanism=PCA-CAA2026.04 | 89.9 | |
| SQ-formatModel=Qwen-3 30B-A3B, Setting=W4A(SQ5)2025.12 | 89.84 | |
| SRQ-driven steeringBackbone=R1-Distill-Qwen-7B, Steering Mechanism=Conceptor2026.04 | 89.6 | |
| RandOptModel=OLMo3-7B-Inst2026.03 | 89.5 | |
| SRQ-driven steeringBackbone=R1-Distill-Qwen-7B, Steering Mechanism=CAA2026.04 | 89.5 | |
| ESModel=OLMo3-7B2026.03 | 89.1 | |
| PCA-CAABackbone=R1-Distill-Qwen-7B2026.04 | 89.1 | |
| PPoT + Qwen2.5 CoderModel Size=3B, k (LLM samples)=5, m (PPoT samples)=52026.04 | 89.08 | |
| Know. AugTraining Strategy=Knowledge-Based Augmentation, Backbone=Qwen-2.5-32B-Instruct2026.01 | 88.8 | |
| SQ-formatModel=Qwen-3 30B-A3B, Setting=W4A(SQ6)2025.12 | 88.78 | |
| ConceptorBackbone=R1-Distill-Qwen-7B2026.04 | 88.6 | |
| Qwen3-8BCategory=Naive LLM2026.01 | 88.42 | |
| PPOModel=OLMo3-7B-Inst2026.03 | 88.4 | |
| CAABackbone=R1-Distill-Qwen-7B2026.04 | 88.4 | |
| SRQ-driven steeringBackbone=R1-Distill-Llama-8B, Steering Mechanism=PCA-CAA2026.04 | 88.4 | |
| AUTOIF (Qwen2-72B-Instruct w/ Online DPO)Backbone Model=Qwen2-72B-Instruct, Supervision Model=Qwen2-72B, Alignment Strategy=Self-Alignment, Training Method=Online DPO2024.06 | 88.2 | |
| SRQ-driven steeringBackbone=R1-Distill-Llama-8B, Steering Mechanism=Conceptor2026.04 | 88.2 | |
| SQ-formatModel=Qwen-3 30B-A3B, Setting=W(SQ6)A42025.12 | 88.17 | |
| SRQ-driven steeringBackbone=R1-Distill-Llama-8B, Steering Mechanism=CAA2026.04 | 88.1 | |
| Qwen2-72B-InstructBackbone Model=Qwen2-72B-Instruct2024.06 | 87.9 | |
| ES + TT-MVModel=Qwen2.5-3B-Inst2026.03 | 87.9 | |
| Base LLMBackbone=R1-Distill-Qwen-7B2026.04 | 87.8 | |
| SDAR-4B-ChatModel=SDAR-4B-Chat, Decoding Strategy=FourierSampler2026.01 | 87.64 | |
| ES + TT-MVModel=Llama3.1-8B-Inst2026.03 | 87.5 | |
| PCA-CAABackbone=R1-Distill-Llama-8B2026.04 | 87.5 | |
| SDAR-4B-ChatModel=SDAR-4B-Chat, Decoding Strategy=RWS2026.01 | 87.41 | |
| ESModel=OLMo3-7B-Inst2026.03 | 87.2 | |
| VIST2-8BCategory=Ours2026.01 | 87.15 | |
| RandOptModel=Qwen2.5-3B-Inst2026.03 | 87.1 | |
| ConceptorBackbone=R1-Distill-Llama-8B2026.04 | 87.1 | |
| GRPOModel=OLMo3-7B-Inst2026.03 | 87 | |
| RandOptModel=Llama3.1-8B-Inst2026.03 | 86.7 | |
| CAABackbone=R1-Distill-Llama-8B2026.04 | 86.7 | |
| SDAR-4B-ChatModel=SDAR-4B-Chat, Decoding Strategy=Vanilla2026.01 | 86.58 | |
| ES + TT-MVModel=OLMo3-7B2026.03 | 86.4 | |
| Qwen2.5 CoderModel Size=3B, k (LLM samples)=5, m (PPoT samples)=02026.04 | 86.2 | |
| Qwen3-4BCategory=Naive LLM2026.01 | 85.97 | |
| ESModel=Qwen2.5-3B-Inst2026.03 | 85.8 | |
| Base LLMBackbone=R1-Distill-Llama-8B2026.04 | 85.8 | |
| FP16Format=FP162025.09 | 85.1 | |
| Best-of-N‡Model=OLMo3-7B-Inst2026.03 | 85 | |
| RandOptModel=OLMo3-7B2026.03 | 85 | |
| RTNFormat=INT82025.09 | 84.8 | |
| PPoT + Qwen2.5 CoderModel Size=7B, k (LLM samples)=1, m (PPoT samples)=52026.04 | 84.76 | |
| RTNFormat=FP82025.09 | 84.7 | |
| GRPO (GT)Model=Qwen2.5-3B, Data=MATH2026.01 | 84.61 | |
| GPTQFormat=FP82025.09 | 84.5 | |
| GPTQFormat=INT82025.09 | 84.4 | |
| PRISMModel=Qwen2.5-7B, Data=DAPO-17k2026.01 | 84.31 | |
| VIST2-4BCategory=Ours2026.01 | 84.31 | |
| Qwen2.5-14BPrecision=FP162026.04 | 84 | |
| Bit-by-BitBackbone=Qwen2.5-14B, Precision=w2a162026.04 | 84 | |
| Best-of-N‡Model=Llama3.1-8B-Inst2026.03 | 83.5 |