Commonsense Reasoning on Commonsense Reasoning Suite (Zero-shot)
82.81PIQA AccuracyLlama2-70B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Llama2-70B#Bits=W16A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 82.81 | — | 83.86 | 80.85 | 57.59 | 77.58 | — | 76.53 | |
| CAT-Q#Bits=W1.58A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 80.34 | — | 74.56 | 80.13 | 53.24 | 75.34 | — | 72.72 | |
| EfficientQAT#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 80.2 | — | 61.58 | 80.01 | 49.23 | 73.64 | — | 68.93 | |
| DB-LLM#Bits=W1*A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 79.27 | — | 76.16 | 55.93 | 44.45 | 73.32 | — | 65.82 | |
| Llama2-7B#Bits=W16A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 78.84 | — | 75.9 | 74.62 | 46.42 | 69.46 | — | 69.04 | |
| SliderQuant#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 78.79 | — | 73.02 | 77.14 | 52.71 | 73.75 | — | 71.08 | |
| OmniQuant#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 74.1 | — | 35.45 | 67.21 | 33.28 | 64.33 | — | 54.87 | |
| DB-LLM#Bits=W1*A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 73.18 | — | 61.98 | 45.2 | 33.53 | 61.72 | — | 55.12 | |
| CAT-Q#Bits=W1.58A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 72.91 | — | 60.95 | 60.06 | 33.62 | 60.93 | — | 57.69 | |
| SliderQuant#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 70.78 | — | 57.15 | 57.79 | 31.06 | 60.14 | — | 55.38 | |
| OmniQuant#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 65.13 | — | 40.28 | 50.13 | 23.46 | 55.88 | — | 46.98 | |
| DECO (NG)Model Scale=Medium, Routing Strategy=NG2026.05 | 64.36 | 36.18 | 29.47 | 51.98 | 20.73 | 52.09 | 19.48 | 39.18 | |
| BlockFFNModel Scale=Medium2026.05 | 63.38 | 35.82 | 29.61 | 48.82 | 20.99 | 50.51 | 19.95 | 38.44 | |
| DeepSeek-V3 (NG)Model Scale=Medium, Routing Strategy=NG2026.05 | 63.28 | 37.67 | 29.64 | 49.07 | 20.65 | 49.25 | 19.76 | 38.47 | |
| ReMoE (NG)Model Scale=Medium, Routing Strategy=NG2026.05 | 63.22 | 36.64 | 29.4 | 49.71 | 19.71 | 50.83 | 20.45 | 38.57 | |
| DenseModel Scale=Medium, Model Architecture=Dense2026.05 | 63 | 36.9 | 29.57 | 51.47 | 22.01 | 50.51 | 19.62 | 39.01 | |
| DeepSeek-V3 (GA)Model Scale=Medium, Routing Strategy=GA2026.05 | 62.79 | 37.87 | 29.61 | 48.53 | 20.9 | 52.17 | 19.84 | 38.82 | |
| ReMoE (GA)Model Scale=Medium, Routing Strategy=GA2026.05 | 62.62 | 37.67 | 29.1 | 48.65 | 20.05 | 50.67 | 18.28 | 38.15 | |
| DECO (GA)Model Scale=Medium, Routing Strategy=GA2026.05 | 61.86 | 37.1 | 28.44 | 47.18 | 19.37 | 49.64 | 17.43 | 37.29 | |
| TopPModel Scale=Medium2026.05 | 61.37 | 36.49 | 28.57 | 45.75 | 19.03 | 50.51 | 12.63 | 36.34 | |
| BiLLM#Bits=W1*A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 60.6 | — | 34.8 | 36.2 | 24.4 | 52.4 | — | 41.68 | |
| GPTQ#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 58.32 | — | 32.59 | 40.45 | 21.25 | 55.17 | — | 41.55 | |
| PB-LLM#Bits=W1*A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 55.22 | — | 30.49 | 29.88 | 22.01 | 50.36 | — | 37.59 | |
| AWQ#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-7B2026.06 | 50 | — | 26.14 | 26.52 | 26.79 | 49.64 | — | 35.82 | |
| GPTQ#Bits=W2A16, Zero-shot evaluation protocol=true, Backbone=Llama2-70B2026.06 | 49.51 | — | 25.04 | 25.08 | 22.7 | 49.57 | — | 34.38 |