Mathematical Reasoning on GSM8K TASA TACQ (test)
83.1AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Base Model=Qwen2.5-7B, Bits=16.02026.07 | 83.1 | |
| TACQ (GSM8K-cond.)Base Model=Qwen2.5-7B, Bits=~3, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.07 | 79.8 | |
| TASA b3.5Base Model=Qwen2.5-7B, Bits=3.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 79.2 | |
| TACQ (GSM8K-cond.)Base Model=LLaMA-3-8B, Bits=~3, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.07 | 68.3 | |
| TASA b3.0Base Model=Qwen2.5-7B, Bits=3.0, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 59.3 | |
| GPTQBase Model=Qwen2.5-7B, Bits=3.0, Calibration Data (Calib)=Wiki2026.07 | 52 | |
| FP16Base Model=LLaMA-3-8B, Bits=16.02026.07 | 49.8 | |
| TACQ (GSM8K-cond.)Base Model=Qwen2.5-7B, Bits=~2, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.07 | 49.1 | |
| TASA b2.5Base Model=Qwen2.5-7B, Bits=2.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 47.9 | |
| TASA b3.5Base Model=LLaMA-3-8B, Bits=3.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 46.2 | |
| TASA b3.0Base Model=LLaMA-3-8B, Bits=3.0, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 39.5 | |
| TACQ (GSM8K-cond.)Base Model=LLaMA-3-8B, Bits=~2, Calibration Data (Calib)=GSM8K, Evaluation Protocol=greedy decoding, 1319 samples2026.07 | 36.6 | |
| TASA b2.5Base Model=LLaMA-3-8B, Bits=2.5, Calibration Data (Calib)=Mixed, Evaluation Protocol=8-shot CoT, full test set2026.07 | 19.5 | |
| GPTQBase Model=LLaMA-3-8B, Bits=3.0, Calibration Data (Calib)=Wiki2026.07 | 8.6 |