Arithmetic Reasoning on GSM8K
97.1AccuracyDUP
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DUPModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 97.1 | — | — | — | — | — | — | |
| DenseModel=Qwen3-8B, Calibration=N/A, Ratio=0%, 8-shot=true2025.10 | 96.74 | — | — | — | — | — | — | |
| Llama-3.1-405BPrompting=CoT2025.12 | 95.9 | — | — | — | — | — | — | |
| Automatic Model Selection with LLMsBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 95.6 | — | — | — | — | — | — | |
| Claude 3.5 SonnetPrompting=CoT2025.12 | 95.6 | — | — | — | — | — | — | |
| GHS-TDABackbone=GPT-4o-mini2026.02 | 95.2 | — | — | — | — | — | — | |
| GHS-TDABackbone=DeepSeek-V32026.02 | 95.2 | — | — | — | — | — | — | |
| AoTBackbone=DeepSeek-V3, Reasoning Paradigm=atomic reasoning2026.02 | 95.1 | — | — | — | — | — | — | |
| AoTBackbone=GPT-4o-mini, Reasoning Paradigm=atomic reasoning2026.02 | 95 | — | — | — | — | — | — | |
| ToTBackbone=DeepSeek-V3, Reasoning Paradigm=tree-based2026.02 | 95 | — | — | — | — | — | — | |
| ToTBackbone=GPT-4o-mini, Reasoning Paradigm=tree-based2026.02 | 94.9 | — | — | — | — | — | — | |
| GHS-TDABackbone=Qwen-Turbo2026.02 | 94.8 | — | — | — | — | — | — | |
| AoTBackbone=Qwen-Turbo, Reasoning Paradigm=atomic reasoning2026.02 | 94.7 | — | — | — | — | — | — | |
| CoTBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 94.6 | — | — | — | — | — | — | |
| Zero-shot CoTModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 94.6 | — | — | — | — | — | — | |
| GPT-4oPrompting=CoT2025.12 | 94.6 | — | — | — | — | — | — | |
| GoTBackbone=GPT-4o-mini, Reasoning Paradigm=graph-based2026.02 | 94.5 | — | — | — | — | — | — | |
| GoTBackbone=DeepSeek-V3, Reasoning Paradigm=graph-based2026.02 | 94.5 | — | — | — | — | — | — | |
| Zero-shot PS+Model=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 94.3 | — | — | — | — | — | — | |
| ToTBackbone=Qwen-Turbo, Reasoning Paradigm=tree-based2026.02 | 94.2 | — | — | — | — | — | — | |
| FoTBackbone=DeepSeek-V3, Reasoning Paradigm=forest-based, n=82026.02 | 94.2 | — | — | — | — | — | — | |
| PALBackbone=GPT-4, Decoding strategy=Greedy2023.05 | 94 | — | — | — | — | — | — | |
| FoTBackbone=GPT-4o-mini, Reasoning Paradigm=forest-based, n=82026.02 | 94 | — | — | — | — | — | — | |
| FoTBackbone=Qwen-Turbo, Reasoning Paradigm=forest-based, n=82026.02 | 93.9 | — | — | — | — | — | — | |
| GoTBackbone=Qwen-Turbo, Reasoning Paradigm=graph-based2026.02 | 93.8 | — | — | — | — | — | — | |
| AFlowBackbone=DeepSeek-V3, Reasoning Paradigm=search-based2026.02 | 93.6 | — | — | — | — | — | — | |
| AFlowBackbone=GPT-4o-mini, Reasoning Paradigm=search-based2026.02 | 93.5 | — | — | — | — | — | — | |
| AFlowBackbone=Qwen-Turbo, Reasoning Paradigm=search-based2026.02 | 93.1 | — | — | — | — | — | — | |
| LLM-PeerReview-WType=LLM Ensemble2025.12 | 93 | — | — | 67.8 | — | — | — | |
| LLM-PeerReviewType=LLM Ensemble2025.12 | 92.7 | — | — | 67.4 | — | — | — | |
| GPT-4oPrompting=decl. PL-1S2025.12 | 92.6 | — | — | — | — | — | — | |
| Least-to-MostModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 92.1 | — | — | — | — | — | — | |
| CoT-SCBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based, n=52026.02 | 92.1 | — | — | — | — | — | — | |
| Qwen2.5-7B-SelectionType=Our variants2025.12 | 92.1 | — | — | 65.5 | — | — | — | |
| CoT-SCBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based, n=52026.02 | 92 | — | — | — | — | — | — | |
| Self-RefineBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 91.9 | — | — | — | — | — | — | |
| GaCType=LLM Ensemble2025.12 | 91.8 | — | — | 60.2 | — | — | — | |
| Self-RefineBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 91.7 | — | — | — | — | — | — | |
| CoT-SCBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based, n=52026.02 | 91.5 | — | — | — | — | — | — | |
| Qwen2.5-7B-InstructType=Single LLM2025.12 | 91.5 | — | — | 62.7 | — | — | — | |
| Smoothie-GlobalType=LLM Ensemble2025.12 | 91.5 | — | — | 60.5 | — | — | — | |
| Self-RefineBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 91.4 | — | — | — | — | — | — | |
| CoTBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 91.3 | — | — | — | — | — | — | |
| CoTBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 90.9 | — | — | — | — | — | — | |
| Llama-3.1-8B-SelectionType=Our variants2025.12 | 90.8 | — | — | 66.4 | — | — | — | |
| Mistral-7B-SelectionType=Our variants2025.12 | 90.8 | — | — | 64.7 | — | — | — | |
| CoTBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 90.7 | — | — | — | — | — | — | |
| GISPModel=Qwen3-8B, Calibration=GSM8K, Ratio=20%, 8-shot=true2025.10 | 90.52 | — | — | — | — | — | — | |
| GPT-4oPrompting=decl. PY-1S2025.12 | 90.1 | — | — | — | — | — | — | |
| ComplexCoT + SCBase Model=gpt-3.5-turbo2023.06 | 89.2 | — | — | — | — | — | — | |
| Qwen2-7B-SelectionType=Our variants2025.12 | 88.8 | — | — | 62.1 | — | — | — | |
| Qwen2-7B-InstructType=Single LLM2025.12 | 88.5 | — | — | 56.6 | — | — | — | |
| BF16Bit=BF16, Model=Qwen3-8B, Group size (g)=N/A2026.02 | 88.48 | — | — | — | — | — | — | |
| CoK + SCBase Model=gpt-3.5-turbo2023.06 | 88.2 | — | — | — | — | — | — | |
| OJBKQBit=4-bit, Model=Qwen3-8B, Group size (g)=1282026.02 | 87.87 | — | — | — | — | — | — | |
| GPT-4oPrompting=PY-ZS2025.12 | 87.7 | — | — | — | — | — | — | |
| Analogical PromptingBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 87.6 | — | — | — | — | — | — | |
| Manual CoT + SCBase Model=gpt-3.5-turbo2023.06 | 87.6 | — | — | — | — | — | — | |
| Analogical PromptingBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 87.2 | — | — | — | — | — | — | |
| Analogical PromptingBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 87 | — | — | — | — | — | — | |
| Agent-ForestType=LLM Ensemble2025.12 | 86.8 | — | — | 60.1 | — | — | — | |
| GPTQBit=4-bit, Model=Qwen3-8B, Group size (g)=1282026.02 | 86.8 | — | — | — | — | — | — | |
| Wanda-spModel=Qwen3-8B, Calibration=GSM8K, Ratio=20%, 8-shot=true2025.10 | 85.9 | — | — | — | — | — | — | |
| GPT-4oPrompting=PL-ZS2025.12 | 85.8 | — | — | — | — | — | — | |
| Smoothie-LocalType=LLM Ensemble2025.12 | 85.5 | — | — | 59.8 | — | — | — | |
| AWQBit=4-bit, Model=Qwen3-8B, Group size (g)=1282026.02 | 85.44 | — | — | — | — | — | — | |
| OJBKQBit=4-bit, Model=Qwen3-4B, Group size (g)=1282026.02 | 84.29 | — | — | — | — | — | — | |
| BF16Bit=BF16, Model=Qwen3-4B, Group size (g)=N/A2026.02 | 83.7 | — | — | — | — | — | — | |
| CoKBase Model=gpt-3.5-turbo2023.06 | 83.2 | — | — | — | — | — | — | |
| Automatic Model Selection with LLMsBackbone=ChatGPT, Decoding strategy=Greedy2023.05 | 82.6 | — | — | — | — | — | — | |
| FullKVCompression Ratio=100%2025.02 | 82.35 | — | — | — | — | — | — | |
| DUPModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 82.3 | — | — | — | — | — | — | |
| Manual-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 81.6 | — | — | — | — | — | — | |
| Self-AgreementBackbone=GPT-3.5-turbo, Scenario=Question type unknown, answer format known2023.11 | 81.3 | — | — | — | — | — | — | |
| RandomType=LLM Ensemble2025.12 | 81.2 | — | — | 54.2 | — | — | — | |
| ShotKVCompression Ratio=40%2025.02 | 81.07 | — | — | — | — | — | — | |
| Theoretical averageType=Single LLM2025.12 | 80.9 | — | — | 54.1 | — | — | — | |
| ShotKVCompression Ratio=30%2025.02 | 80.82 | — | — | — | — | — | — | |
| CoTBackbone=ChatGPT, Decoding strategy=Greedy2023.05 | 80.8 | — | — | — | — | — | — | |
| ShotKVCompression Ratio=20%2025.02 | 80.57 | — | — | — | — | — | — | |
| BoHABackbone=Gemma-2-9B, #Params=108.04M2025.09 | 80.44 | — | — | — | — | — | — | |
| SnapKVCompression Ratio=30%2025.02 | 80.38 | — | — | — | — | — | — | |
| StreamingLLMCompression Ratio=40%2025.02 | 80.37 | — | — | — | — | — | — | |
| ShotKVCompression Ratio=10%2025.02 | 80.37 | — | — | — | — | — | — | |
| Multi-Agents (Debate)Backbone=GPT-3.5-turbo, Scenario=Question type unknown, answer format known2023.11 | 80.2 | — | — | — | — | — | — | |
| Auto-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 80.2 | — | — | — | — | — | — | |
| AWQBit=4-bit, Model=Qwen3-4B, Group size (g)=1282026.02 | 80.15 | — | — | — | — | — | — | |
| HiRABackbone=Gemma-2-9B, #Params=108.04M2025.09 | 79.91 | — | — | — | — | — | — | |
| FFTBackbone=Gemma-2-9B, #Params=9.24B2025.09 | 79.89 | — | — | — | — | — | — | |
| ABBABackbone=Gemma-2-9B, #Params=108.04M2025.09 | 79.83 | — | — | — | — | — | — | |
| GISPModel=Qwen3-8B, Calibration=GSM8K, Ratio=30%, 8-shot=true2025.10 | 79.68 | — | — | — | — | — | — | |
| SnapKVCompression Ratio=40%2025.02 | 79.35 | — | — | — | — | — | — | |
| PyramidKVCompression Ratio=30%2025.02 | 79.34 | — | — | — | — | — | — | |
| SnapKVCompression Ratio=20%2025.02 | 79.34 | — | — | — | — | — | — | |
| H2OCompression Ratio=30%2025.02 | 79.32 | — | — | — | — | — | — | |
| ChunkKVCompression Ratio=30%2025.02 | 79.32 | — | — | — | — | — | — | |
| ChunkKVCompression Ratio=10%2025.02 | 79.32 | — | — | — | — | — | — | |
| Zero-shot PS+Model=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 79.3 | — | — | — | — | — | — | |
| Llama-3.1-8B-InstructType=Single LLM2025.12 | 79.3 | — | — | 53.5 | — | — | — | |
| ComplexCoTBase Model=gpt-3.5-turbo2023.06 | 79.3 | — | — | — | — | — | — |