List your company
ResearchFormerly WizWand

Arithmetic Reasoning on SVAMP (test)

98.16AccuracySGE
35.697651.913868.1384.3462May 28, 2024Sep 25, 2024Jan 23, 2025May 24, 2025Sep 21, 2025Jan 19, 2026May 20, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SGEModel=GPT-4, Tool=Code Interpreter2024.0598.16
Decomp PromptingModel=GPT-4, Tool=Code Interpreter2024.0592.16
CoT PromptingModel=GPT-4, Tool=Code Interpreter2024.0591.58
Refine PromptingModel=GPT-4, Tool=Code Interpreter2024.0591
CoTTest Engine=Phi-3.5-Mini-Instruct2026.0590.4
TextRegTest Engine=Qwen2-7B-Instruct2026.0590.1
CoTTest Engine=Qwen2-7B-Instruct2026.0589.8
TextGradTest Engine=Qwen2-7B-Instruct2026.0589.3
REVOLVETest Engine=Qwen2-7B-Instruct2026.0589.2
LLMBOOSTBackbone=Qwen-2.5-7B, Params=2 × 7B2025.1288.8
TextRegTest Engine=Phi-3.5-Mini-Instruct2026.0588.5
LLMBOOSTBackbone=Qwen-2.5-7B, Params=3 × 7B2025.1288.4
IO PromptingModel=GPT-4, Tool=Code Interpreter2024.0588.34
REVOLVETest Engine=Phi-3.5-Mini-Instruct2026.0587
voteBackbone=Qwen-2.5-7B, Params=3 × 7B2025.1286.7
voteBackbone=Qwen-2.5-7B, Params=2 × 7B2025.1286.4
UNITEBackbone=Qwen-2.5-7B, Params=2 × 7B2025.1286.4
UNITEBackbone=Qwen-2.5-7B, Params=3 × 7B2025.1286.3
Qwen-2.5-7BBackbone=Qwen-2.5-7B, Params=1 × 7B2025.1286.1
UNITEBackbone=Qwen-2.5-3B, Params=3 × 3B2025.1286
LLMBOOSTBackbone=Qwen-2.5-3B, Params=3 × 3B2025.1285.6
CoTTest Engine=Llama-3.1-8B-Instruct2026.0585.5
TextRegTest Engine=Llama-3.1-8B-Instruct2026.0585.5
T-copilotBackbone=Qwen-2.5-7B, Params=2 × 7B2025.1285.3
LLMBOOSTBackbone=Qwen-2.5-3B, Params=2 × 3B2025.1285.3
voteBackbone=Qwen-2.5-3B, Params=3 × 3B2025.1285.2
TextGradTest Engine=Llama-3.1-8B-Instruct2026.0584.8
UNITEBackbone=Qwen-2.5-3B, Params=2 × 3B2025.1284.7
REVOLVETest Engine=Llama-3.1-8B-Instruct2026.0584.6
REVOLVETest Engine=Llama-3-8B-Instruct2026.0583.9
TextRegTest Engine=Llama-3-8B-Instruct2026.0583.9
CoTTest Engine=Llama-3-8B-Instruct2026.0583.7
Qwen-2.5-3BBackbone=Qwen-2.5-3B, Params=1 × 3B2025.1283.4
TextGradTest Engine=Llama-3-8B-Instruct2026.0583.2
T-copilotBackbone=Qwen-2.5-3B, Params=2 × 3B2025.1282.7
voteBackbone=Qwen-2.5-3B, Params=2 × 3B2025.1282.6
LLMBOOSTBackbone=Llama-3.1-8B, Params=3 × 8B2025.1281.3
TextGradTest Engine=Phi-3.5-Mini-Instruct2026.0580.6
LLMBOOSTBackbone=Llama-3.1-8B, Params=2 × 8B2025.1280.1
voteBackbone=Llama-3.1-8B, Params=3 × 8B2025.1278.1
voteBackbone=Llama-3.1-8B, Params=2 × 8B2025.1277.5
Llama-3.1-8BBackbone=Llama-3.1-8B, Params=1 × 8B2025.1277.4
UNITEBackbone=Llama-3.1-8B, Params=3 × 8B2025.1277.2
UNITEBackbone=Llama-3.1-8B, Params=2 × 8B2025.1276.5
T-copilotBackbone=Llama-3.1-8B, Params=2 × 8B2025.1275.7
DoRA + MoDEKBackbone=LLaMA3-8B2024.1074.7
LoRAALL + MoDEKBackbone=LLaMA3-8B2024.1074.2
DoRABackbone=LLaMA3-8B2024.1074
LoRAALLBackbone=LLaMA3-8B2024.1073.8
LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA3-8B2024.1073.5
LoRA¬KBackbone=LLaMA3-8B2024.1072.8
LLMBOOSTBackbone=LLama-3.2-3B, Params=2 × 3B2025.1270.1
LLMBOOSTBackbone=LLama-3.2-3B, Params=3 × 3B2025.1269.9
voteBackbone=LLama-3.2-3B, Params=2 × 3B2025.1268.2
REFTBackbone=LLaMA3-8B2024.1067.5
T-copilotBackbone=LLama-3.2-3B, Params=2 × 3B2025.1266.7
voteBackbone=LLama-3.2-3B, Params=3 × 3B2025.1266.4
UNITEBackbone=LLama-3.2-3B, Params=3 × 3B2025.1266
LLama-3.2-3BBackbone=LLama-3.2-3B, Params=1 × 3B2025.1265.8
UNITEBackbone=LLama-3.2-3B, Params=2 × 3B2025.1265.7
RoAd4Model=13B, #Params.=0.07%2024.0859.5
RoAd2Model=13B, #Params.=0.03%2024.0858.3
RoAd1Model=13B, #Params.=0.02%2024.0857.3
AdapterPModel=13B, #Params.=2.89%2024.0855.7
LORAModel=13B, #Params.=0.67%2024.0854.6
LoReFTModel=13B, #Params.=0.03%2024.0854.2
AdapterSModel=7B, #Params.=0.99%2024.0852.3
LORAModel=7B, #Params.=0.83%2024.0852.1
AdapterSModel=13B, #Params.=0.80%2024.0850.8
AdapterPModel=7B, #Params.=3.54%2024.0849.6
RoAd4Model=7B, #Params.=0.08%2024.0849.4
RoAd2Model=7B, #Params.=0.04%2024.0849.3
(IA)3Model=7B, #Params.=0.02%2024.0848.5
DoRA + MoDEKBackbone=LLaMA2-7B2024.1048.4
DoRABackbone=LLaMA2-7B2024.1048.2
LoRAALL + MoDEKBackbone=LLaMA2-7B2024.1048
LoRAALLBackbone=LLaMA2-7B2024.1047.7
LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA2-7B2024.1047.5
LoReFTModel=7B, #Params.=0.03%2024.0846.8
RoAd1Model=7B, #Params.=0.02%2024.0846.7
LoRA¬KBackbone=LLaMA2-7B2024.1046.6
REFTBackbone=LLaMA2-7B2024.1044
PrefixModel=13B, #Params.=0.03%2024.0841.4
PrefixModel=7B, #Params.=0.04%2024.0838.1