Arithmetic Reasoning on AQUA
77.1AccuracyDUP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DUPModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 77.1 | — | — | |
| Zero-shot PS+Model=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 75.5 | — | — | |
| StandPerturbation=No Dert.2025.06 | 74 | — | — | |
| Zero-shot CoTModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 72.8 | — | — | |
| Least-to-MostModel=GPT-4, Evaluation Protocol=Zero-shot2024.04 | 71.6 | — | — | |
| RoPPerturbation=SC2025.06 | 70.1 | — | — | |
| CoTPerturbation=SC2025.06 | 65.8 | — | — | |
| RoPPerturbation=WOO2025.06 | 65.8 | — | — | |
| RoPPerturbation=HW2025.06 | 64.6 | — | — | |
| CoTPerturbation=HW2025.06 | 63.4 | — | — | |
| RoPPerturbation=EC2025.06 | 63 | — | — | |
| RoPPerturbation=UIC2025.06 | 62.2 | — | — | |
| APEPerturbation=WOO2025.06 | 60.6 | — | — | |
| DUPModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 60.2 | — | — | |
| APEPerturbation=HW2025.06 | 59.8 | — | — | |
| StandPerturbation=WOO2025.06 | 59.5 | — | — | |
| PromptAgentPerturbation=SC2025.06 | 58.7 | — | — | |
| PromptAgentPerturbation=HW2025.06 | 58.3 | — | — | |
| StandPerturbation=EC2025.06 | 57.9 | — | — | |
| CoTPerturbation=EC2025.06 | 57.9 | — | — | |
| APEPerturbation=EC2025.06 | 57.9 | — | — | |
| Least-to-MostModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 57.4 | — | — | |
| PromptAgentPerturbation=WOO2025.06 | 57.1 | — | — | |
| StandPerturbation=UIC2025.06 | 57.1 | — | — | |
| CoTPerturbation=WOO2025.06 | 56.7 | — | — | |
| PromptAgentPerturbation=EC2025.06 | 56.3 | — | — | |
| StandPerturbation=HW2025.06 | 56.3 | — | — | |
| APEPerturbation=UIC2025.06 | 56.3 | — | — | |
| Zero-shot PS+Model=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 55.9 | — | — | |
| Manual-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 55.9 | — | — | |
| Auto-CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Few-shot2024.04 | 54.3 | — | — | |
| Zero-shot CoTModel=GPT-3.5-Turbo, Evaluation Protocol=Zero-shot2024.04 | 53 | — | — | |
| APEPerturbation=SC2025.06 | 52.8 | — | — | |
| StandPerturbation=SC2025.06 | 52.4 | — | — | |
| Self-consistencyBackbone=GPT-3 Code-davinci-0022022.03 | 52 | — | — | |
| BaseBackbone=Qwen2.5-3B2026.02 | 50 | — | — | |
| CoTPerturbation=UIC2025.06 | 50 | — | — | |
| PromptAgentPerturbation=UIC2025.06 | 48.4 | — | — | |
| Self-consistencyBackbone=PaLM-540B2022.03 | 48.3 | — | — | |
| HydraLoRABackbone=Qwen2.5-3B2026.02 | 42.91 | — | — | |
| HypLoRABackbone=Qwen2.5-3B2026.02 | 41.34 | — | — | |
| CoT-promptingBackbone=GPT-3 Code-davinci-0022022.03 | 39.8 | — | — | |
| MoSLoRABackbone=Qwen2.5-3B2026.02 | 39.37 | — | — | |
| LoRABackbone=Qwen2.5-3B2026.02 | 38.98 | — | — | |
| Amini et al. (2019)2022.03 | 37.9 | — | — | |
| Auto-CoTPrompting=Automatic Chain-of-Thought2022.10 | 36.5 | — | — | |
| Manual-CoTPrompting=Manual Chain-of-Thought2022.10 | 35.8 | — | — | |
| CoT-promptingBackbone=PaLM-540B2022.03 | 35.8 | — | — | |
| Zero-Shot-CoTPrompting=Zero-shot Chain-of-Thought2022.10 | 33.5 | — | — | |
| Self-consistencyBackbone=UL2-20B2022.03 | 26.9 | — | — | |
| Self-consistencyBackbone=LaMDA-137B2022.03 | 26.8 | — | — | |
| Self-consistencyBackbone=GPT-3 Code-davinci-0012022.03 | 25.6 | — | — | |
| Few-ShotPrompting=Few-shot2022.10 | 24.8 | — | — | |
| CoT-promptingBackbone=UL2-20B2022.03 | 23.6 | — | — | |
| Zero-ShotPrompting=Zero-shot2022.10 | 22.4 | — | — | |
| CoT-promptingBackbone=GPT-3 Code-davinci-0012022.03 | 18.9 | — | — | |
| CoT-promptingBackbone=LaMDA-137B2022.03 | 17.7 | — | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=SAF2025.07 | — | 25.6 | 26 | |
| CRFTBackbone=LLaMA-2-7B, Identify=MAF2025.07 | — | 27.6 | 24.8 | |
| CRFTBackbone=LLaMA-2-7B, Identify=SSF2025.07 | — | 26 | 26.8 | |
| CRFTBackbone=LLaMA-2-7B, Identify=MSF2025.07 | — | 27.2 | 22.8 | |
| CRFTBackbone=LLaMA-3-8B, Identify=SAF2025.07 | — | 47.2 | 47.2 | |
| CRFTBackbone=LLaMA-3-8B, Identify=MAF2025.07 | — | 48.4 | 50.4 | |
| CRFTBackbone=LLaMA-3-8B, Identify=SSF2025.07 | — | 50 | 49.2 | |
| CRFTBackbone=LLaMA-3-8B, Identify=MSF2025.07 | — | 48 | 51.6 | |
| CRFTBackbone=Mistral-7B, Identify=SAF2025.07 | — | 36.2 | 38.6 | |
| CRFTBackbone=Mistral-7B, Identify=MAF2025.07 | — | 39 | 38.2 | |
| CRFTBackbone=Mistral-7B, Identify=SSF2025.07 | — | 37.4 | 33.5 | |
| CRFTBackbone=Mistral-7B, Identify=MSF2025.07 | — | 41.3 | 37.8 | |
| ReFTBackbone=LLaMA-2-7B, Identify=p7+s72025.07 | — | 21.7 | 21.7 | |
| ReFTBackbone=LLaMA-3-8B, Identify=p7+s72025.07 | — | 46.9 | 46.9 | |
| ReFTBackbone=Mistral-7B, Identify=p7+s72025.07 | — | 32.3 | 32.3 |