List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K TASA TACQ (test)

83.1AccuracyFP16
5.6225.73545.8565.965Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
FP16Base Model=Qwen2.5-7B, Bits=16.02026.0783.1
TACQ (GSM8K-cond.)Base Model=Qwen2.5-7B, Bits=~3, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.0779.8
TASA b3.5Base Model=Qwen2.5-7B, Bits=3.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0779.2
TACQ (GSM8K-cond.)Base Model=LLaMA-3-8B, Bits=~3, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.0768.3
TASA b3.0Base Model=Qwen2.5-7B, Bits=3.0, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0759.3
GPTQBase Model=Qwen2.5-7B, Bits=3.0, Calibration Data (Calib)=Wiki2026.0752
FP16Base Model=LLaMA-3-8B, Bits=16.02026.0749.8
TACQ (GSM8K-cond.)Base Model=Qwen2.5-7B, Bits=~2, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.0749.1
TASA b2.5Base Model=Qwen2.5-7B, Bits=2.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0747.9
TASA b3.5Base Model=LLaMA-3-8B, Bits=3.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0746.2
TASA b3.0Base Model=LLaMA-3-8B, Bits=3.0, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0739.5
TACQ (GSM8K-cond.)Base Model=LLaMA-3-8B, Bits=~2, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.0736.6
TASA b2.5Base Model=LLaMA-3-8B, Bits=2.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.0719.5
GPTQBase Model=LLaMA-3-8B, Bits=3.0, Calibration Data (Calib)=Wiki2026.078.6