Arithmetic Reasoning on SVAMP (test)
98.16AccuracySGE
Evaluation Results
| Method | Links | |
|---|---|---|
| SGEModel=GPT-4, Tool=Code Interpreter2024.05 | 98.16 | |
| Decomp PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 92.16 | |
| CoT PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 91.58 | |
| Refine PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 91 | |
| CoTTest Engine=Phi-3.5-Mini-Instruct2026.05 | 90.4 | |
| TextRegTest Engine=Qwen2-7B-Instruct2026.05 | 90.1 | |
| CoTTest Engine=Qwen2-7B-Instruct2026.05 | 89.8 | |
| TextGradTest Engine=Qwen2-7B-Instruct2026.05 | 89.3 | |
| REVOLVETest Engine=Qwen2-7B-Instruct2026.05 | 89.2 | |
| LLMBOOSTBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 88.8 | |
| TextRegTest Engine=Phi-3.5-Mini-Instruct2026.05 | 88.5 | |
| LLMBOOSTBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 88.4 | |
| IO PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 88.34 | |
| REVOLVETest Engine=Phi-3.5-Mini-Instruct2026.05 | 87 | |
| voteBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 86.7 | |
| voteBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 86.4 | |
| UNITEBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 86.4 | |
| UNITEBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 86.3 | |
| Qwen-2.5-7BBackbone=Qwen-2.5-7B, Params=1 × 7B2025.12 | 86.1 | |
| UNITEBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 86 | |
| LLMBOOSTBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 85.6 | |
| CoTTest Engine=Llama-3.1-8B-Instruct2026.05 | 85.5 | |
| TextRegTest Engine=Llama-3.1-8B-Instruct2026.05 | 85.5 | |
| T-copilotBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 85.3 | |
| LLMBOOSTBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 85.3 | |
| voteBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 85.2 | |
| TextGradTest Engine=Llama-3.1-8B-Instruct2026.05 | 84.8 | |
| UNITEBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 84.7 | |
| REVOLVETest Engine=Llama-3.1-8B-Instruct2026.05 | 84.6 | |
| REVOLVETest Engine=Llama-3-8B-Instruct2026.05 | 83.9 | |
| TextRegTest Engine=Llama-3-8B-Instruct2026.05 | 83.9 | |
| CoTTest Engine=Llama-3-8B-Instruct2026.05 | 83.7 | |
| Qwen-2.5-3BBackbone=Qwen-2.5-3B, Params=1 × 3B2025.12 | 83.4 | |
| TextGradTest Engine=Llama-3-8B-Instruct2026.05 | 83.2 | |
| T-copilotBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 82.7 | |
| voteBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 82.6 | |
| LLMBOOSTBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 81.3 | |
| TextGradTest Engine=Phi-3.5-Mini-Instruct2026.05 | 80.6 | |
| LLMBOOSTBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 80.1 | |
| voteBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 78.1 | |
| voteBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 77.5 | |
| Llama-3.1-8BBackbone=Llama-3.1-8B, Params=1 × 8B2025.12 | 77.4 | |
| UNITEBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 77.2 | |
| UNITEBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 76.5 | |
| T-copilotBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 75.7 | |
| DoRA + MoDEKBackbone=LLaMA3-8B2024.10 | 74.7 | |
| LoRAALL + MoDEKBackbone=LLaMA3-8B2024.10 | 74.2 | |
| DoRABackbone=LLaMA3-8B2024.10 | 74 | |
| LoRAALLBackbone=LLaMA3-8B2024.10 | 73.8 | |
| LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA3-8B2024.10 | 73.5 | |
| LoRA¬KBackbone=LLaMA3-8B2024.10 | 72.8 | |
| LLMBOOSTBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 70.1 | |
| LLMBOOSTBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 69.9 | |
| voteBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 68.2 | |
| REFTBackbone=LLaMA3-8B2024.10 | 67.5 | |
| T-copilotBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 66.7 | |
| voteBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 66.4 | |
| UNITEBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 66 | |
| LLama-3.2-3BBackbone=LLama-3.2-3B, Params=1 × 3B2025.12 | 65.8 | |
| UNITEBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 65.7 | |
| RoAd4Model=13B, #Params.=0.07%2024.08 | 59.5 | |
| RoAd2Model=13B, #Params.=0.03%2024.08 | 58.3 | |
| RoAd1Model=13B, #Params.=0.02%2024.08 | 57.3 | |
| AdapterPModel=13B, #Params.=2.89%2024.08 | 55.7 | |
| LORAModel=13B, #Params.=0.67%2024.08 | 54.6 | |
| LoReFTModel=13B, #Params.=0.03%2024.08 | 54.2 | |
| AdapterSModel=7B, #Params.=0.99%2024.08 | 52.3 | |
| LORAModel=7B, #Params.=0.83%2024.08 | 52.1 | |
| AdapterSModel=13B, #Params.=0.80%2024.08 | 50.8 | |
| AdapterPModel=7B, #Params.=3.54%2024.08 | 49.6 | |
| RoAd4Model=7B, #Params.=0.08%2024.08 | 49.4 | |
| RoAd2Model=7B, #Params.=0.04%2024.08 | 49.3 | |
| (IA)3Model=7B, #Params.=0.02%2024.08 | 48.5 | |
| DoRA + MoDEKBackbone=LLaMA2-7B2024.10 | 48.4 | |
| DoRABackbone=LLaMA2-7B2024.10 | 48.2 | |
| LoRAALL + MoDEKBackbone=LLaMA2-7B2024.10 | 48 | |
| LoRAALLBackbone=LLaMA2-7B2024.10 | 47.7 | |
| LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA2-7B2024.10 | 47.5 | |
| LoReFTModel=7B, #Params.=0.03%2024.08 | 46.8 | |
| RoAd1Model=7B, #Params.=0.02%2024.08 | 46.7 | |
| LoRA¬KBackbone=LLaMA2-7B2024.10 | 46.6 | |
| REFTBackbone=LLaMA2-7B2024.10 | 44 | |
| PrefixModel=13B, #Params.=0.03%2024.08 | 41.4 | |
| PrefixModel=7B, #Params.=0.04%2024.08 | 38.1 |