Multiple-choice Language Understanding on MMLU (last 25% test split)
72AccuracyTACQ (MMLU-cond.)
Evaluation results
| Method | Links | |
|---|---|---|
| TACQ (MMLU-cond.)Base Model=Qwen2.5-7B, Bits=~3, Calibration Data (Calib)=MMLU, Evaluation Protocol=5-shot, last 25% of MMLU test set2026.07 | 72 | |
| TACQ (MMLU-cond.)Base Model=LLaMA-3-8B, Bits=~3, Calibration Data (Calib)=MMLU, Evaluation Protocol=5-shot, last 25% of MMLU test set2026.07 | 63.6 | |
| TACQ (MMLU-cond.)Base Model=Qwen2.5-7B, Bits=~2, Calibration Data (Calib)=MMLU, Evaluation Protocol=5-shot, last 25% of MMLU test set2026.07 | 56.4 | |
| TACQ (MMLU-cond.)Base Model=LLaMA-3-8B, Bits=~2, Calibration Data (Calib)=MMLU, Evaluation Protocol=5-shot, last 25% of MMLU test set2026.07 | 49.3 |