Common Sense Reasoning on Six common sense reasoning tasks
69.46AccuracyFP
Evaluation Results
| Method | Links | |
|---|---|---|
| FPBackbone=Llama 2 7B, Evaluation Protocol=Zero-shot2026.03 | 69.46 | |
| GPTQ FlatQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=FlatQuant, Train=true, Evaluation Protocol=Zero-shot2026.03 | 67.58 | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=true, Evaluation Protocol=Zero-shot2026.03 | 67.34 | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=false, Evaluation Protocol=Zero-shot2026.03 | 67.25 | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=true, Evaluation Protocol=Zero-shot2026.03 | 67.17 | |
| RTN FlatQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=FlatQuant, Train=true, Evaluation Protocol=Zero-shot2026.03 | 66.67 | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=false, Evaluation Protocol=Zero-shot2026.03 | 66.22 | |
| GPTQ SpinQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=SpinQuant, Train=true, Evaluation Protocol=Zero-shot2026.03 | 66.12 | |
| GPTQ QuaRotBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=QuaRot, Train=false, Evaluation Protocol=Zero-shot2026.03 | 66.06 | |
| RTN SpinQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=SpinQuant, Train=true, Evaluation Protocol=Zero-shot2026.03 | 63.64 | |
| TOGA-qPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 62.6 | |
| OriginalBackbone=Mistral-7B, Compression Ratio=0%2026.07 | 61 | |
| ResQPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 58.43 | |
| RTN QuaRotBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=QuaRot, Train=false, Evaluation Protocol=Zero-shot2026.03 | 57.54 | |
| OriginalBackbone=LLaMA 2-7B, Compression Ratio=0%2026.07 | 57 | |
| TOGA-qPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 56.03 | |
| OriginalBackbone=Vicuna-7B, Compression Ratio=0%2026.07 | 56 | |
| LACE-SVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.07 | 55 | |
| TOGA-qPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 54.2 | |
| LACE-SVDBackbone=Vicuna-7B, Compression Ratio=20%2026.07 | 54 | |
| SVD-LLMBackbone=LLaMA 2-7B, Compression Ratio=20%2026.07 | 53 | |
| ResQPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 52.63 | |
| OriginalBackbone=OPT-6.7B, Compression Ratio=0%2026.07 | 52 | |
| AtomPrecision=W(3.2)A(3.2), Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 51.71 | |
| SVD-LLMBackbone=Vicuna-7B, Compression Ratio=20%2026.07 | 51 | |
| TOGA-qPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 50.81 | |
| LACE-SVDBackbone=Mistral-7B, Compression Ratio=20%2026.07 | 50 | |
| AtomPrecision=W(3.2)A(3.2), Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 48.13 | |
| TOGA-qPrecision=W(1.6)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 47.58 | |
| TOGA-qPrecision=W(1.6)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 46.8 | |
| ResQPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 46.33 | |
| GPTQ SmoothQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=SmoothQuant, Train=false, Evaluation Protocol=Zero-shot2026.03 | 44.62 | |
| SilM-LLMPrecision=W2A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 43.94 | |
| PTQ-1.61†Precision=W(1.6)A16, Backbone=Qwen3-8B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 43.53 | |
| SilM-LLMPrecision=W2A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 43 | |
| LACE-SVDBackbone=OPT-6.7B, Compression Ratio=20%2026.07 | 43 | |
| ResQPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 42.45 | |
| TOGA-qPrecision=W(1.6)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 42.07 | |
| SVD-LLMBackbone=Mistral-7B, Compression Ratio=20%2026.07 | 42 | |
| PTQ-1.61†Precision=W(1.6)A16, Backbone=Llama-2-7B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 41.43 | |
| TOGA-qPrecision=W(1.6)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 41.33 | |
| BiLLMPrecision=W(2.1)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 41.16 | |
| SVD-LLMBackbone=OPT-6.7B, Compression Ratio=20%2026.07 | 41 | |
| SilM-LLMPrecision=W2A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 40.03 | |
| PTQ-1.61†Precision=W(1.6)A16, Backbone=Mistral-7B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 39.77 | |
| AtomPrecision=W(3.2)A(3.2), Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 39.57 | |
| BiLLMPrecision=W(2.1)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 39.16 | |
| BiLLMPrecision=W(2.1)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 39.11 | |
| AtomPrecision=W(3.2)A(3.2), Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 38.36 | |
| BiLLMPrecision=W(2.1)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 37.87 | |
| PTQ-1.61Precision=W(1.6)A16, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 37.15 | |
| SilM-LLMPrecision=W2A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 36.12 | |
| ASVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.07 | 36 | |
| PTQ-1.61†Precision=W(1.6)A16, Backbone=Llama-3-8B, Calibration Pre-processing=RedPajama dataset, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 35.59 | |
| RTN SmoothQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=SmoothQuant, Train=false, Evaluation Protocol=Zero-shot2026.03 | 34.49 | |
| SpinQuantPrecision=W3A3, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 34.26 | |
| PTQ-1.61Precision=W(1.6)A16, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 34.22 | |
| PTQ-1.61Precision=W(1.6)A16, Backbone=Qwen3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 34.02 | |
| SpinQuantPrecision=W3A3, Backbone=Mistral-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 33.96 | |
| PTQ-1.61Precision=W(1.6)A16, Backbone=Llama-3-8B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 33.76 | |
| SpinQuantPrecision=W3A3, Backbone=Llama-2-7B, Zero-shot Evaluation Protocol=Average of six tasks2026.06 | 33.06 | |
| ASVDBackbone=Vicuna-7B, Compression Ratio=20%2026.07 | 33 | |
| ASVDBackbone=OPT-6.7B, Compression Ratio=20%2026.07 | 32 | |
| ASVDBackbone=Mistral-7B, Compression Ratio=20%2026.07 | 32 | |
| GPTQ NoneBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=None, Train=false, Evaluation Protocol=Zero-shot2026.03 | 31.88 | |
| RTN NoneBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=None, Train=false, Evaluation Protocol=Zero-shot2026.03 | 31.51 | |
| FWSVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.07 | 12 | |
| SVDBackbone=LLaMA 2-7B, Compression Ratio=20%2026.07 | 9 | |
| FWSVDBackbone=Vicuna-7B, Compression Ratio=20%2026.07 | 9 | |
| FWSVDBackbone=Mistral-7B, Compression Ratio=20%2026.07 | 8 | |
| FWSVDBackbone=OPT-6.7B, Compression Ratio=20%2026.07 | 6 | |
| SVDBackbone=Vicuna-7B, Compression Ratio=20%2026.07 | 5 | |
| SVDBackbone=OPT-6.7B, Compression Ratio=20%2026.07 | 3 | |
| SVDBackbone=Mistral-7B, Compression Ratio=20%2026.07 | 3 |