Mathematical Reasoning on AMC 23 (Accuracy)
100AccuracyGemini 2.5 pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 2.5 proEvaluation Protocol=pass@12025.09 | 100 | |
| Thought-ICSModel=OSS-20B2026.02 | 92.5 | |
| Thought-ICSModel=Qwen-32B2026.02 | 92.5 | |
| Thought-ICSModel=OSS-120B2026.02 | 92.5 | |
| Skill-centric distillation frameworkBase Model=Qwen3-4B, SFT Setting=1K2026.01 | 91.9 | |
| STAPOBase Model=Qwen3-14B, Evaluation Setting=JustRL2026.02 | 90.62 | |
| PACS-4BEvaluation Protocol=pass@1, Backbone=Qwen3-4B2025.09 | 90.45 | |
| BaseBase Model=Qwen3-4B, SFT Setting=1K2026.01 | 90.1 | |
| Budget Allocation w/ V0Backbone=Qwen3-4B-Instruct-2507, budget_allocation_mode=real-time difficulty estimation2026.02 | 89.84 | |
| JustRLBase Model=Qwen3-14B, Evaluation Setting=JustRL2026.02 | 89.61 | |
| RandomBase Model=Qwen3-4B, SFT Setting=1K2026.01 | 89.5 | |
| s1Base Model=Qwen3-4B, SFT Setting=1K, Data Selection Mode=offline2026.01 | 89.1 | |
| s1Base Model=Qwen3-4B, SFT Setting=1K, Data Selection Mode=online2026.01 | 89.1 | |
| PACS-8BEvaluation Protocol=pass@1, Backbone=Qwen3-8B2025.09 | 88.69 | |
| LIMOBase Model=Qwen3-4B, SFT Setting=1K, Data Selection Mode=offline2026.01 | 88.4 | |
| 20-EntropyBase Model=Qwen3-14B, Evaluation Setting=JustRL2026.02 | 87.73 | |
| Claude Opus 4.1Evaluation Protocol=pass@12025.09 | 87.5 | |
| STAPOBase Model=Qwen3-14B, Evaluation Setting=Training-Aligned2026.02 | 87.11 | |
| Budget Allocation w/o V0Backbone=Qwen3-4B-Instruct-2507, budget_allocation_mode=lagged heuristics2026.02 | 87.03 | |
| GRPOBackbone=Qwen3-4B-Instruct-25072026.02 | 85.78 | |
| Token-ICSModel=Qwen-32B2026.02 | 85 | |
| Qwen3-14BEvaluation Protocol=pass@12025.09 | 84.88 | |
| 20-EntropyBase Model=Qwen3-14B, Evaluation Setting=Training-Aligned2026.02 | 84.45 | |
| Thought-ICS-AModel=OSS-120B2026.02 | 82.5 | |
| Qwen3-4BEvaluation Protocol=pass@12025.09 | 82.3 | |
| GRPOBase Model=Qwen3-14B, Evaluation Setting=Training-Aligned2026.02 | 81.95 | |
| JustRLBase Model=Qwen3-14B, Evaluation Setting=Training-Aligned2026.02 | 81.95 | |
| Qwen3-8BEvaluation Protocol=pass@12025.09 | 81.74 | |
| JustRLBase Model=Qwen3-8B, Evaluation Setting=JustRL2026.02 | 81.48 | |
| LIMOEvaluation Protocol=pass@12025.09 | 80.61 | |
| STAPOBase Model=Qwen3-8B, Evaluation Setting=JustRL2026.02 | 80.23 | |
| Thought-ICS-SModel=OSS-20B2026.02 | 80 | |
| Thought-ICS-AModel=OSS-20B2026.02 | 80 | |
| Thought-ICSModel=Qwen-14B2026.02 | 80 | |
| 20-EntropyBase Model=Qwen3-8B, Evaluation Setting=JustRL2026.02 | 80 | |
| 20-EntropyBase Model=Qwen3-8B, Evaluation Setting=Training-Aligned2026.02 | 79.92 | |
| STAPOBase Model=Qwen3-8B, Evaluation Setting=Training-Aligned2026.02 | 79.92 | |
| SOUPratioBackbone=Qwen2.5-Math-7B, Truncation=Length-ratio-based2026.01 | 79.84 | |
| SOUPratioBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Truncation=Length-ratio-based2026.01 | 79.22 | |
| On-policyBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.01 | 78.83 | |
| GRPOBase Model=Qwen3-14B, Evaluation Setting=JustRL2026.02 | 78.12 | |
| M2POBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.01 | 77.66 | |
| Thought-ICSModel=Qwen-7B2026.02 | 77.5 | |
| Thought-ICSModel=LLaMA-70B2026.02 | 77.5 | |
| SOUPentropyBackbone=Qwen2.5-Math-7B, Truncation=Entropy-based2026.01 | 75.39 | |
| GRPOBase Model=Qwen3-8B, Evaluation Setting=Training-Aligned2026.02 | 75.23 | |
| Thought-ICS-SModel=OSS-120B2026.02 | 75 | |
| GRPOBase Model=Qwen3-8B, Evaluation Setting=JustRL2026.02 | 74.14 | |
| M2POBackbone=Qwen2.5-Math-7B2026.01 | 73.75 | |
| JustRLBase Model=Qwen3-8B, Evaluation Setting=Training-Aligned2026.02 | 73.52 | |
| CoVeModel=Qwen-32B2026.02 | 72.5 | |
| MV@10Model=Qwen-14B2026.02 | 72.5 | |
| Token-ICSModel=LLaMA-70B2026.02 | 72.5 | |
| Token-ICSModel=OSS-120B2026.02 | 72.5 | |
| OpenThinker3-7BEvaluation Protocol=pass@12025.09 | 72.44 | |
| LUFFYBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.01 | 72.27 | |
| DPO-R1 (LOW)Backbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=N = 2, Trainable Pairs=3,403, Processing Time=1.42026.02 | 71.9 | |
| PACEBackbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=2 < N < 3, Trainable Pairs=9,028, Processing Time=1.72026.02 | 71.6 | |
| SOUPentropyBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Truncation=Entropy-based2026.01 | 71.48 | |
| On-policyBackbone=Qwen2.5-Math-7B2026.01 | 71.25 | |
| MV@10Model=Qwen-32B2026.02 | 70 | |
| SAGEBase Model=Qwen2.5-7B-Instruct2026.02 | 70 | |
| Sky-T1-7BEvaluation Protocol=pass@12025.09 | 69.59 | |
| PACEBackbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=2 < N < 3, Trainable Pairs=10,717, Processing Time=1.02026.02 | 69.4 | |
| DPO-R1 (ZHANG ET AL., 2025)Backbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=N = 8, Trainable Pairs=8,403, Processing Time=2.82026.02 | 68.8 | |
| DPO-R1 (MIDDLE)Backbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=N = 4, Trainable Pairs=5,127, Processing Time=1.22026.02 | 67.8 | |
| Thought-ICS-AModel=Qwen-32B2026.02 | 67.5 | |
| CoVeModel=OSS-120B2026.02 | 67.5 | |
| DPO-R1 (ZHANG ET AL., 2025)Backbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=N = 8, Trainable Pairs=10,211, Processing Time=4.82026.02 | 67.5 | |
| DPO-R1 (HIGH)Backbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=N = 16, Trainable Pairs=10,543, Processing Time=7.22026.02 | 67.5 | |
| LUFFYBackbone=Qwen2.5-Math-7B2026.01 | 67.5 | |
| Majority Voting (N=16)DM=7B2025.12 | 67.5 | |
| RSDTM=32B, DM=7B, PRM=1.5B2025.12 | 67.5 | |
| EASDTM=72B, DM=7B2025.12 | 67.5 | |
| DPO-R1 (HIGH)Backbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=N = 16, Trainable Pairs=8,964, Processing Time=5.22026.02 | 66.9 | |
| DPO-R1 (MIDDLE)Backbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=N = 4, Trainable Pairs=6,073, Processing Time=2.22026.02 | 66.2 | |
| DPO-R1 (LOW)Backbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=N = 2, Trainable Pairs=3,479, Processing Time=0.92026.02 | 65.6 | |
| Self-RefineModel=Qwen-32B2026.02 | 65 | |
| Thought-ICS-SModel=Qwen-32B2026.02 | 65 | |
| Thought-ICS-SModel=LLaMA-70B2026.02 | 65 | |
| Single ModelTM=72B2025.12 | 65 | |
| RSDTM=72B, DM=7B, PRM=1.5B2025.12 | 65 | |
| INSTRUCT (BASE)Backbone=Qwen3-4B-Instruct, Compute Cost (Sampling Budget N)=N/A, Trainable Pairs=N/A, Processing Time=N/A2026.02 | 64.4 | |
| INSTRUCT (BASE)Backbone=Qwen3-8B-Instruct, Compute Cost (Sampling Budget N)=N/A, Trainable Pairs=N/A, Processing Time=N/A2026.02 | 63.7 | |
| Eurus-2-7B-PRIMEEvaluation Protocol=pass@12025.09 | 63.65 | |
| Thought-ICS-AModel=LLaMA-70B2026.02 | 62.5 | |
| MV@10Model=LLaMA-70B2026.02 | 62.5 | |
| DPO (Full)Base Model=Qwen2.5-7B-Instruct2026.02 | 62.5 | |
| Beam Search (N=16)DM=7B2025.12 | 62.5 | |
| EASDTM=32B, DM=7B2025.12 | 62.5 | |
| CoTModel=LLaMA-70B2026.02 | 60 | |
| Self-RefineModel=LLaMA-70B2026.02 | 60 | |
| Self-RefineModel=OSS-120B2026.02 | 60 | |
| MV@10Model=Qwen-7B2026.02 | 60 | |
| Token-ICSModel=Qwen-14B2026.02 | 60 | |
| InitModel=LLaMA-70B2026.02 | 60 | |
| Single ModelTM=32B2025.12 | 60 | |
| SDTM=72B, DM=7B2025.12 | 60 | |
| Reasoning Palette (Linear Decay)Backbone=Qwen3-8B-Base, RL Algorithm=RLOO2025.12 | 59.38 | |
| CoVeModel=OSS-20B2026.02 | 57.5 |