Zero-shot Common Sense Reasoning on Common Sense Reasoning
72.6Zero-shot AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Family=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=16/16/162026.04 | 72.6 | |
| baselineBackbone=LLaMA-2 70B, #Bits=FP16, #Eff. (w)=162026.03 | 70.54 | |
| SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.442026.03 | 70.35 | |
| SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.432026.03 | 70.31 | |
| FP16Family=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=16/16/162026.04 | 70.13 | |
| CoQuantFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.04 | 70.01 | |
| L2QERBackbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.552026.03 | 69.78 | |
| SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.652026.03 | 69.68 | |
| ResQFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.04 | 69.64 | |
| SERQ-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.792026.03 | 69.58 | |
| SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.642026.03 | 69.38 | |
| L2QER-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.812026.03 | 69.03 | |
| QuaRotFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4/4/42026.04 | 68.68 | |
| CoQuantFamily=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=4.5/4.5/4.52026.04 | 67.44 | |
| baselineBackbone=LLaMA-3 8B, #Bits=FP16, #Eff. (w)=162026.03 | 67.16 | |
| L2QERBackbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=4.552026.03 | 66.92 | |
| FP16Backbone=LLaMA-2 13B2026.03 | 66.53 | |
| ResQFamily=Qwen2.5, Model=Qwen2.5-7B, W/A/KV=4.5/4.5/4.52026.04 | 66.46 | |
| SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.712026.03 | 66.4 | |
| SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.522026.03 | 66.23 | |
| DenseModel Size=Large, Model=Llama-7B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.12 | 66.07 | |
| L2QERBackbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=7.162026.03 | 65.68 | |
| FP16Model=Llama-3.2 3B-it, # Bits (W-A-KV)=16-16-162025.06 | 65.63 | |
| FlatQuantBackbone=LLaMA-2 13B, Calib. Time=∼225m2026.03 | 64.94 | |
| OstQuantBackbone=LLaMA-2 13B, Calib. Time=∼92m2026.03 | 64.92 | |
| DuQuantBackbone=LLaMA-2 13B, Calib. Time=∼486m2026.03 | 64.88 | |
| SERQBackbone=LLaMA-2 13B, Calib. Time=∼48m2026.03 | 64.82 | |
| SpinQuantBackbone=LLaMA-2 13B, Calib. Time=∼721m2026.03 | 64.8 | |
| FP16Family=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=16/16/162026.04 | 64.18 | |
| FP16Backbone=LLaMA-2 7B2026.03 | 64.09 | |
| baselineBackbone=LLaMA-2 7B, #Bits=FP16, #Eff. (w)=162026.03 | 64.09 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 63.84 | |
| baselineBackbone=Qwen-2.5-3B, #Bits=FP16, #Eff. (w)=162026.03 | 63.82 | |
| SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 63.78 | |
| SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 63.7 | |
| SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 63.41 | |
| QUIKFamily=Qwen2.5, Model=Qwen2.5-14B, W/A/KV=4.5/4.5/4.52026.04 | 63.39 | |
| L2QERBackbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 63.35 | |
| L2QER-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.832026.03 | 63.33 | |
| SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 63.04 | |
| FP16Family=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=16/16/162026.04 | 62.97 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 62.95 | |
| SERQ-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.632026.03 | 62.71 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 62.71 | |
| FP16Backbone=LLaMA-3.2 3B2026.03 | 62.66 | |
| baselineBackbone=LLaMA-3.2 3B, #Bits=FP16, #Eff. (w)=162026.03 | 62.66 | |
| QuaRotBackbone=LLaMA-2 13B, Calib. Time=∼44m2026.03 | 62.55 | |
| SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=8.072026.03 | 62.49 | |
| SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.752026.03 | 62.41 | |
| FlatQuantBackbone=LLaMA-2 7B, Calib. Time=∼131m2026.03 | 62.11 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 61.96 | |
| SERQBackbone=LLaMA-2 7B, Calib. Time=∼23m2026.03 | 61.87 | |
| SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 61.87 | |
| DuQuantBackbone=LLaMA-2 7B, Calib. Time=∼255m2026.03 | 61.84 | |
| SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 61.77 | |
| OstQuantBackbone=LLaMA-2 7B, Calib. Time=∼72m2026.03 | 61.72 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 61.54 | |
| SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 61.27 | |
| SERQ-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 61.26 | |
| SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 61.24 | |
| SpinQuantBackbone=LLaMA-2 7B, Calib. Time=∼598m2026.03 | 61 | |
| L2QER-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 60.95 | |
| LLM.int4()Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 60.86 | |
| SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 60.78 | |
| L2QER-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 60.56 | |
| SERQ-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 60.43 | |
| FlatQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼76m2026.03 | 60.41 | |
| L2QERBackbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 60.36 | |
| CoQuantFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.04 | 60.23 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 60.07 | |
| ResQFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.04 | 60.03 | |
| LLM.int4()Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 59.64 | |
| QuaRotBackbone=LLaMA-2 7B, Calib. Time=∼31m2026.03 | 59.53 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 59.43 | |
| OstQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼26m2026.03 | 58.81 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 58.65 | |
| SERQBackbone=LLaMA-3.2 3B, Calib. Time=∼15m2026.03 | 58.5 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 58.5 | |
| L2QER-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 58.41 | |
| SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 58.4 | |
| DuQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼140m2026.03 | 58.31 | |
| L2QERBackbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 57.67 | |
| CoQuantFamily=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=4.5/4.5/4.52026.04 | 57.59 | |
| SERQ-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 57.5 | |
| SpinQuantBackbone=LLaMA-3.2 3B, Calib. Time=∼156m2026.03 | 56.88 | |
| DenseModel Size=Medium, Model=Llama-3.2-1B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.12 | 56.54 | |
| FP16Family=Llama 3.2, Model=Llama-3.2-1B, W/A/KV=16/16/162026.04 | 56.48 | |
| BLASTModel Size=Large, Model=Llama-7B, Compression Factor (CF)=2x, Evaluation Protocol=Zero-shot2025.12 | 56.23 | |
| QuaRotFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4/4/42026.04 | 56.13 | |
| LLM.int4()Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 56.06 | |
| QuaRotBackbone=LLaMA-3.2 3B, Calib. Time=∼8m2026.03 | 55.76 | |
| ResQFamily=Qwen2.5, Model=Qwen2.5-1.5B, W/A/KV=4.5/4.5/4.52026.04 | 55.66 | |
| QUIKFamily=Llama 3.2, Model=Llama-3.2-3B, W/A/KV=4.5/4.5/4.52026.04 | 55.56 | |
| L2QERBackbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=11.442026.03 | 55.44 | |
| baselineBackbone=LLaMA-3.2 1B, #Bits=FP16, #Eff. (w)=162026.03 | 54.82 | |
| L2QERBackbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 54.71 | |
| FP16Family=Qwen2.5, Model=Qwen2.5-0.5B, W/A/KV=16/16/162026.04 | 54.02 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 53.93 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 53.44 | |
| SERQ (RTN)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 52.76 |