List your company
ResearchFormerly WizWand

Math Word Problem Solving on GSM8K (test set)

93.7AccuracyCAPO
6.02828.78951.5574.311Oct 5, 2023Feb 25, 2024Jul 18, 2024Dec 8, 2024May 1, 2025Sep 21, 2025Feb 12, 2026
Updated 8mo ago

Evaluation results

MethodLinks
CAPOFramework=promptolution, LLM=Gemma-3-27B-it2025.1293.7
EvoPromptFramework=promptolution, LLM=Gemma-3-27B-it2025.1291
AutoDiffFramework=AdalFlow, LLM=Gemma-3-27B-it2025.1288.7
Full-FTBackbone=Qwen3-8B-Base, KV Sharing=Not Supported2026.0285.8
PrefillShareBackbone=Qwen3-8B-Base, KV Sharing=Supported2026.0284.8
GEPAFramework=DSPy, LLM=Gemma-3-27B-it2025.1284.7
CoTCompilation=+ensemble, Training=trainset, Model=GPT-3.52023.1081.6
unoptimizedFramework=Baseline, LLM=Gemma-3-27B-it2025.1278.1
reflectionCompilation=bootstrap, Training=trainset, Model=GPT-3.52023.1076
CoTCompilation=bootstrap, Training=trainset, Model=GPT-3.52023.1072.9
CoTCompilation=fewshot, Training=+human_CoT, Model=GPT-3.52023.1072.4
PrefillShareBackbone=LLaMA3.1-8B, KV Sharing=Supported2026.0271.4
Full-FTBackbone=LLaMA3.1-8B, KV Sharing=Not Supported2026.0271.3
OPROFramework=promptolution, LLM=Gemma-3-27B-it2025.1269.7
vanillaCompilation=+ensemble, Training=trainset, Model=GPT-3.52023.1061.9
vanillaCompilation=bootstrap×2, Training=trainset, Model=GPT-3.52023.1061.7
PASERPruning Scheme=LLM-Pruner (25%)2025.0249.4
PASERPruning Scheme=Wanda (2:4)2025.0248.5
PASERPruning Scheme=SliceGPT (25%)2025.0247.8
PASERPruning Scheme=SparseGPT (50%)2025.0247.2
NuggetsPruning Scheme=LLM-Pruner (25%)2025.0247.1
reflectionCompilation=+ensemble, Training=trainset, Model=Llama2-13b-chat2023.1046.9
IFDPruning Scheme=LLM-Pruner (25%)2025.0246.8
Full DataPruning Scheme=LLM-Pruner (25%)2025.0246.5
Instruction MiningPruning Scheme=LLM-Pruner (25%)2025.0246.2
NuggetsPruning Scheme=Wanda (2:4)2025.0246.2
Full DataPruning Scheme=Wanda (2:4)2025.0245.9
RandomPruning Scheme=LLM-Pruner (25%)2025.0245.8
IFDPruning Scheme=Wanda (2:4)2025.0245.8
NuggetsPruning Scheme=SparseGPT (50%)2025.0245.7
NuggetsPruning Scheme=SliceGPT (25%)2025.0245.4
Instruction MiningPruning Scheme=Wanda (2:4)2025.0245.4
IFDPruning Scheme=SparseGPT (50%)2025.0245.4
Full DataPruning Scheme=SparseGPT (50%)2025.0245.2
IFDPruning Scheme=SliceGPT (25%)2025.0245.1
Instruction MiningPruning Scheme=SparseGPT (50%)2025.0244.9
Full DataPruning Scheme=SliceGPT (25%)2025.0244.8
RandomPruning Scheme=Wanda (2:4)2025.0244.7
Instruction MiningPruning Scheme=SliceGPT (25%)2025.0244.5
w/o TrainingPruning Scheme=LLM-Pruner (25%)2025.0244.3
RandomPruning Scheme=SparseGPT (50%)2025.0244.3
RandomPruning Scheme=SliceGPT (25%)2025.0243.9
w/o TrainingPruning Scheme=Wanda (2:4)2025.0243.8
w/o TrainingPruning Scheme=SparseGPT (50%)2025.0243.1
w/o TrainingPruning Scheme=SliceGPT (25%)2025.0242.5
reflectionCompilation=bootstrap, Training=trainset, Model=Llama2-13b-chat2023.1040.2
vanillaCompilation=bootstrap×2, Training=trainset, Model=Llama2-13b-chat2023.1036.5
vanillaCompilation=+ensemble, Training=trainset, Model=Llama2-13b-chat2023.1034.6
CoTCompilation=fewshot, Training=+human_CoT, Model=Llama2-13b-chat2023.1033.7
LLaMA3.1-8BKV Sharing=Inherent2026.0225.9
vanillaCompilation=none, Training=n/a, Model=GPT-3.52023.1025.2
Qwen3-8B-BaseKV Sharing=Inherent2026.0211.8
vanillaCompilation=none, Training=n/a, Model=Llama2-13b-chat2023.109.4