Mathematical Reasoning on MathQA
98.84AccuracyClaude Opus 4.6 + Claude Haiku 4.5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Claude Opus 4.6 + Claude Haiku 4.5Tuple=2, Combos=812026.04 | 98.84 | — | 123.87 | — | — | |
| Thought-ICS-AModel=OSS-120B2026.02 | 90 | — | — | — | — | |
| FlowSteerMethodology=Ours, Backbone=GPT-4o-mini2026.02 | 88.67 | — | — | — | — | |
| GrACE-SCBackbone=Qwen2.5, Sampling budget T=82025.09 | 88.3 | 8 | — | — | — | |
| GrACE-ESBackbone=Qwen2.5, Sampling budget T=82025.09 | 87.7 | 3.87 | — | — | — | |
| Thought-ICS-SModel=OSS-120B2026.02 | 87 | — | — | — | — | |
| CoTModel=Qwen-32B2026.02 | 86 | — | — | — | — | |
| Thought-ICS-AModel=Qwen-32B2026.02 | 86 | — | — | — | — | |
| Thought-ICS-SModel=Qwen-32B2026.02 | 85 | — | — | — | — | |
| SCBackbone=Qwen2.5, Sampling budget T=82025.09 | 85 | 8 | — | — | — | |
| ESCBackbone=Qwen2.5, Sampling budget T=82025.09 | 84.7 | 5.08 | — | — | — | |
| ASCBackbone=Qwen2.5, Sampling budget T=82025.09 | 84.1 | 3.79 | — | — | — | |
| CoTModel=Qwen-14B2026.02 | 84 | — | — | — | — | |
| Self-RefineModel=Qwen-32B2026.02 | 84 | — | — | — | — | |
| CoVeModel=OSS-120B2026.02 | 84 | — | — | — | — | |
| Comb-EBackbone=GPT-4o-mini, Workflow Type=Combined-E2026.05 | 83.6 | — | — | — | — | |
| IC-QBackbone=GPT-4o-mini, Workflow Type=Learned2026.05 | 83.6 | — | — | — | — | |
| AFlow (4o-mini)Methodology=AFlow, Backbone=GPT-4o-mini2026.02 | 83.59 | — | — | — | — | |
| CoTBackbone=GPT-4o-mini, Workflow Type=Chain-of-Thought2026.05 | 83.3 | — | — | — | — | |
| SOTAProtocol=State-of-the-Art2023.05 | 83.2 | — | — | — | — | |
| Thought-ICS-SModel=Qwen-14B2026.02 | 83 | — | — | — | — | |
| Thought-ICS-AModel=Qwen-14B2026.02 | 83 | — | — | — | — | |
| Agentflow (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Agentflow2026.02 | 82.81 | — | — | — | — | |
| Reflect.Backbone=GPT-4o-mini, Workflow Type=Reflection2026.05 | 82.5 | — | — | — | — | |
| Comb-TBackbone=GPT-4o-mini, Workflow Type=Combined-T2026.05 | 82.3 | — | — | — | — | |
| Orchestrator (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Orchestrator2026.02 | 82.03 | — | — | — | — | |
| Thought-ICS-AModel=OSS-20B2026.02 | 82 | — | — | — | — | |
| GrACE-SCBackbone=Llama-3.1, Sampling budget T=82025.09 | 81.8 | 8 | — | — | — | |
| GrACE-ESBackbone=Llama-3.1, Sampling budget T=82025.09 | 80.5 | 4.48 | — | — | — | |
| Router-R1 (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Router-R12026.02 | 80.47 | — | — | — | — | |
| Ours2025.02 | 80.4 | — | — | — | — | |
| Thought-ICS-SModel=OSS-20B2026.02 | 80 | — | — | — | — | |
| Baseline (4o-mini)Methodology=Baseline, Backbone=GPT-4o-mini2026.02 | 79.69 | — | — | — | — | |
| SCBackbone=Llama-3.1, Sampling budget T=82025.09 | 79.6 | 8 | — | — | — | |
| ESCBackbone=Llama-3.1, Sampling budget T=82025.09 | 79.6 | 6.06 | — | — | — | |
| ASCBackbone=Llama-3.1, Sampling budget T=82025.09 | 79.1 | 4.35 | — | — | — | |
| Self-RefineModel=Qwen-14B2026.02 | 79 | — | — | — | — | |
| CoVeModel=Qwen-32B2026.02 | 79 | — | — | — | — | |
| CoTModel=OSS-120B2026.02 | 79 | — | — | — | — | |
| Self-RefineModel=OSS-120B2026.02 | 79 | — | — | — | — | |
| CoVeModel=Qwen-14B2026.02 | 77 | — | — | — | — | |
| ReFTFramework=Reasoning2025.02 | 76.3 | — | — | — | — | |
| Self-RefineModel=LLaMA-70B2026.02 | 76 | — | — | — | — | |
| Thought-ICS-AModel=LLaMA-70B2026.02 | 76 | — | — | — | — | |
| KTOFramework=RLHF2025.02 | 75.8 | — | — | — | — | |
| DPOFramework=RLHF2025.02 | 75.3 | — | — | — | — | |
| Baseline (Qwen3-8B)Methodology=Baseline, Backbone=Qwen3-8B2026.02 | 75 | — | — | — | — | |
| Thought-ICS-SModel=LLaMA-70B2026.02 | 74 | — | — | — | — | |
| o3-mini2025.02 | 73.9 | — | — | — | — | |
| Self-RefineModel=OSS-20B2026.02 | 73 | — | — | — | — | |
| CoTModel=LLaMA-70B2026.02 | 73 | — | — | — | — | |
| SkyworkFramework=RLAIF2025.02 | 72.9 | — | — | — | — | |
| PPOFramework=RLHF2025.02 | 71.9 | — | — | — | — | |
| DeepSeek2025.02 | 71.8 | — | — | — | — | |
| ChatGPTZero-shot=true2023.05 | 71.4 | — | — | — | — | |
| CoT+SFTFramework=Reasoning2025.02 | 71.2 | — | — | — | — | |
| Thought-ICS-SModel=Qwen-7B2026.02 | 71 | — | — | — | — | |
| Thought-ICS-AModel=Qwen-7B2026.02 | 71 | — | — | — | — | |
| CoTModel=OSS-20B2026.02 | 71 | — | — | — | — | |
| Self-RefineModel=Qwen-7B2026.02 | 70 | — | — | — | — | |
| Proxy LLM2025.02 | 69.4 | — | — | — | — | |
| CoTModel=Qwen-7B2026.02 | 69 | — | — | — | — | |
| CoVeModel=LLaMA-70B2026.02 | 69 | — | — | — | — | |
| SFT2025.02 | 66.7 | — | — | — | — | |
| CoVeModel=Qwen-7B2026.02 | 64 | — | — | — | — | |
| SFT (Qwen3-8B)Methodology=SFT, Backbone=Qwen3-8B2026.02 | 61.71 | — | — | — | — | |
| Dynamic Program PromptingEvaluation Protocol=Prompting, #Param=175B2023.05 | 61.7 | — | — | — | — | |
| GRPO (Qwen3-8B)Methodology=GRPO, Backbone=Qwen3-8B2026.02 | 60.15 | — | — | — | — | |
| OriginalRatio=0%, Backbone=Qwen3-30B-A3B, Evaluation Protocol=zero-shot2025.09 | 59 | — | — | — | — | |
| CoTModel=LLaMA-8B2026.02 | 57 | — | — | — | — | |
| EoRAModel=LLaMA3-8B, Compression Method=GPTQ, Compression Setting=W42024.10 | 56.04 | — | — | — | — | |
| Self-RefineModel=LLaMA-8B2026.02 | 55 | — | — | — | — | |
| EoRAModel=LLaMA3-8B, Compression Method=SparseGPT, Compression Setting=2:42024.10 | 54.67 | — | — | — | — | |
| Self-refineModel=PaLM2, Prompting Strategy=Self-refine2024.03 | 54.01 | — | — | — | — | |
| LoftQModel=LLaMA3-8B, Compression Method=GPTQ, Compression Setting=W42024.10 | 53.96 | — | — | — | — | |
| EoRAModel=LLaMA3-8B, Compression Method=GPTQ, Compression Setting=W32024.10 | 53.9 | — | — | — | — | |
| StandardModel=LLaMA3-8B, Compression Method=Full-precision, Compression Setting=-2024.10 | 53.56 | — | — | — | — | |
| Thought-ICS-SModel=LLaMA-8B2026.02 | 53 | — | — | — | — | |
| Thought-ICS-AModel=LLaMA-8B2026.02 | 53 | — | — | — | — | |
| CoVeModel=OSS-20B2026.02 | 53 | — | — | — | — | |
| QLoRAModel=LLaMA3-8B, Compression Method=GPTQ, Compression Setting=W42024.10 | 51.42 | — | — | — | — | |
| Program DistillationEvaluation Protocol=Fine-tuning, #Param=6B2023.05 | 50.6 | — | — | — | — | |
| HEAPrRatio=25%, Backbone=Qwen3-30B-A3B, Evaluation Protocol=zero-shot2025.09 | 50 | — | — | — | — | |
| BaselineModel=Qwen3-8B, Precision=FP162026.05 | 49.65 | — | — | — | — | |
| LoftQModel=LLaMA3-8B, Compression Method=SparseGPT, Compression Setting=2:42024.10 | 48.77 | — | — | — | — | |
| LoftQModel=LLaMA3-8B, Compression Method=GPTQ, Compression Setting=W32024.10 | 48.17 | — | — | — | — | |
| CoTModel=LLaMA-3B2026.02 | 48 | — | — | — | — | |
| TORQModel=Qwen3-8B2026.05 | 47.94 | — | — | — | — | |
| StoSignSGDModel backbone=Qwen2.5-7B, Zero-shot=true2026.04 | 46.47 | — | — | — | — | |
| SignAdamWModel backbone=Qwen2.5-7B, Zero-shot=true2026.04 | 45.43 | — | — | — | — | |
| QLoRAModel=LLaMA3-8B, Compression Method=SparseGPT, Compression Setting=2:42024.10 | 45.42 | — | — | — | — | |
| KL-DRTOBackbone=Llama3-8B2026.03 | 45.2 | — | — | — | — | |
| SignAdaMaxModel backbone=Qwen2.5-7B, Zero-shot=true2026.04 | 44.92 | — | — | — | — | |
| χ2-DRTOBackbone=Llama3-8B2026.03 | 44.9 | — | — | — | — | |
| IE-StoSignSGDModel backbone=Qwen2.5-7B, Zero-shot=true2026.04 | 44.86 | — | — | — | — | |
| DPOBackbone=Llama3-8B2026.03 | 44.5 | — | — | — | — | |
| RTOBackbone=Llama3-8B2026.03 | 44.1 | — | — | — | — | |
| GRPOBackbone=Llama3-8B2026.03 | 44.1 | — | — | — | — | |
| Thought-ICS-SModel=LLaMA-3B2026.02 | 44 | — | — | — | — | |
| Thought-ICS-AModel=LLaMA-3B2026.02 | 44 | — | — | — | — |