Commonsense Reasoning on BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c
87.49BoolQ AccuracyQwen2.5-32B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-32BModel=Qwen2.5-32B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 87.49 | 82.26 | 84.13 | 77.61 | 81.03 | 58.45 | 78.5 | |
| SliderQuantModel=Qwen2.5-32B, #Bits=W4A16, Evaluation Protocol=Zero-shot2026.03 | 86.74 | 81.88 | 83.17 | 77.27 | 80.39 | 57.59 | 77.84 | |
| Qwen2.5-14BModel=Qwen2.5-14B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 85.26 | 82.1 | 82.95 | 75.61 | 79.59 | 58.87 | 77.4 | |
| SliderQuantModel=Qwen2.5-14B, #Bits=W4A16, Evaluation Protocol=Zero-shot2026.03 | 84.95 | 81.23 | 81.65 | 75.03 | 79.12 | 58.43 | 76.74 | |
| Qwen2.5-7BModel=Qwen2.5-7B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 84.71 | 79.71 | 78.13 | 71.27 | 76.05 | 49.57 | 73.24 | |
| SliderQuantModel=Qwen2.5-7B, #Bits=W4A16, Evaluation Protocol=Zero-shot2026.03 | 83.15 | 79.22 | 76.97 | 71.27 | 75.47 | 49.2 | 72.55 | |
| BF16Model=Qwen-3-8B, Bits=16, Protocol=Zero-shot2026.02 | 83.03 | 79.11 | 58.81 | 72.38 | 81.52 | 52.47 | 71.22 | |
| BF16Model=Llama-3-8B, Bits=16, Protocol=Zero-shot2026.02 | 81.96 | 80.09 | 60.01 | 73.56 | 81.57 | 51.45 | 71.44 | |
| Llama2-13BModel=Llama2-13B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 80.55 | 80.41 | 79.37 | 72.14 | 77.4 | 49.15 | 73.17 | |
| SliderQuantModel=Llama2-13B, #Bits=W4A16, Evaluation Protocol=Zero-shot2026.03 | 78.1 | 80.41 | 78.3 | 72.14 | 76.73 | 48.55 | 72.37 | |
| Llama2-7BModel=Llama2-7B, #Bits=W16A16, Evaluation Protocol=Zero-shot2026.03 | 78.01 | 78.84 | 75.9 | 69.46 | 74.62 | 46.42 | 70.54 | |
| LLaMA-7B BaselineDecomposition=Baseline, #Params (B)=6.7, #MACS=423.932024.05 | 75.04 | 78.67 | 76.22 | 70 | 72.85 | 44.88 | 69.61 | |
| SliderQuantModel=Llama2-7B, #Bits=W4A16, Evaluation Protocol=Zero-shot2026.03 | 74.89 | 78.67 | 74.77 | 69.14 | 71.55 | 42.49 | 68.59 | |
| Feature Space DecompositionDecomposition=Feature Space, #Params (B)=5.4, #MACS=339.992024.05 | 74.34 | 74.86 | 66.72 | 67.4 | 66.33 | 39.42 | 64.68 | |
| SliderQuantModel=Llama2-13B, #Bits=W2A16, Evaluation Protocol=Zero-shot2026.03 | 70.67 | 73.56 | 64.75 | 63.22 | 67.47 | 37.54 | 62.87 | |
| SliderQuantModel=Qwen2.5-14B, #Bits=W2A16, Evaluation Protocol=Zero-shot2026.03 | 70.14 | 72.09 | 62.12 | 63.3 | 68.1 | 40.7 | 62.74 | |
| LLaMASize=1.3B, SNN=no, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 69.54 | 74.32 | 60.76 | 59.67 | 68.52 | 38.05 | 61.8 | |
| LlambaSize=1.4B, SNN=no, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 68.62 | 73.78 | 61.31 | 60.69 | 69.57 | 37.32 | 61.88 | |
| HBLLMcolModel=Qwen-3-8B, Bits=3.25, Protocol=Zero-shot2026.02 | 68.53 | 71.65 | 45.68 | 63.85 | 68.43 | 36.77 | 54.3 | |
| MARSize=1.4B, SNN=yes, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 68.17 | 70.35 | 50.91 | 55.33 | 67.21 | 35.58 | 57.2 | |
| ARB-LLMRCModel=Qwen-3-8B, Bits=2.51, Protocol=Zero-shot2026.02 | 66.3 | 68.17 | 40.83 | 59.12 | 68.18 | 34.64 | 51.86 | |
| SliderQuantModel=Llama2-7B, #Bits=W2A16, Evaluation Protocol=Zero-shot2026.03 | 66.12 | 70.78 | 57.15 | 60.14 | 57.79 | 31.06 | 57.17 | |
| SliderQuantModel=Qwen2.5-7B, #Bits=W2A16, Evaluation Protocol=Zero-shot2026.03 | 65.05 | 68.5 | 53.02 | 59.91 | 60.31 | 34.56 | 56.89 | |
| SpikeLLMSize=7B, SNN=yes, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 64.37 | 65.45 | 56.59 | 54.3 | 41.67 | 32.51 | 52.48 | |
| ARB-LLMRCModel=Llama-3-8B, Bits=2.51, Protocol=Zero-shot2026.02 | 64.28 | 63.28 | 34.73 | 56.04 | 49.71 | 22.95 | 44.23 | |
| HBLLMcolModel=Llama-3-8B, Bits=3.25, Protocol=Zero-shot2026.02 | 63.03 | 70.35 | 43.46 | 63.77 | 60.02 | 29.1 | 50.45 | |
| STBLLMModel=Qwen-3-8B, Bits=4.13, Protocol=Zero-shot2026.02 | 62.84 | 64.15 | 38.38 | 57.54 | 52.78 | 27.13 | 46.15 | |
| BiLLMModel=Qwen-3-8B, Bits=2.88, Protocol=Zero-shot2026.02 | 62.23 | 55.01 | 32.42 | 51.3 | 28.96 | 22.35 | 38.18 | |
| Weight Space DecompositionDecomposition=Weight Space, #Params (B)=5.4, #MACS=339.992024.05 | 62.2 | 62.57 | 43.91 | 58.8 | 44.95 | 30.03 | 50.41 | |
| NANOQUANTModel=Qwen-3-8B, Bits=1, Protocol=Zero-shot2026.02 | 62.17 | 63.32 | 36.34 | 58.01 | 49.45 | 24.32 | 48.94 | |
| Weight Space DecompositionDecomposition=Weight Space, #Params (B)=3.4, #MACS=215.612024.05 | 62.08 | 53.59 | 27.88 | 48.46 | 27.15 | 27.05 | 41.1 | |
| Feature Space DecompositionDecomposition=Feature Space, #Params (B)=3.4, #MACS=215.612024.05 | 62.02 | 61.37 | 34.64 | 56.43 | 40.32 | 28.75 | 47.25 | |
| Bi-MambaSize=1.3B, SNN=no, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 62 | 69.2 | 43.1 | 53.7 | 43.9 | 24.4 | 49.38 | |
| NANOQUANTModel=Llama-3-8B, Bits=1, Protocol=Zero-shot2026.02 | 61.47 | 60.45 | 33.81 | 55.96 | 43.69 | 20.31 | 45.95 | |
| TinyLLaMASize=1.3B, SNN=no, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 57.8 | 73.3 | 59.9 | 59.1 | 55.3 | 30.1 | 55.91 | |
| SmoothQuantSize=1.3B, SNN=no, Zero-shot evaluation=true, Normalized logits (ARC-E, HS)=true2026.01 | 57.77 | 71.6 | 53.68 | 60.06 | 57.03 | 29.44 | 54.93 | |
| LLM-PrunerDecomposition=LLM-Pruner, #Params (B)=5.4, #MACS=339.602024.05 | 57.06 | 75.68 | 66.8 | 59.83 | 60.94 | 36.52 | 59.47 | |
| SliderQuantModel=Qwen2.5-32B, #Bits=W2A16, Evaluation Protocol=Zero-shot2026.03 | 56.85 | 74.59 | 65.12 | 66.3 | 72.01 | 46.42 | 63.55 | |
| BiLLMModel=Llama-3-8B, Bits=2.88, Protocol=Zero-shot2026.02 | 56.36 | 57.56 | 30.16 | 51.93 | 36.32 | 18.34 | 38.16 | |
| LLM-PrunerDecomposition=LLM-Pruner, #Params (B)=3.4, #MACS=206.592024.05 | 52.32 | 59.63 | 35.64 | 53.2 | 33.5 | 27.22 | 43.58 | |
| STBLLMModel=Llama-3-8B, Bits=4.13, Protocol=Zero-shot2026.02 | 48.01 | 61.48 | 36.47 | 57.62 | 36.87 | 19.97 | 39.83 | |
| GPTQ(w2g64)Model=Qwen-3-8B, Bits=2.28, Protocol=Zero-shot2026.02 | 43.64 | 54.68 | 29.52 | 50.04 | 28.62 | 20.99 | 37.92 | |
| GPTQ(w2g64)Model=Llama-3-8B, Bits=2.28, Protocol=Zero-shot2026.02 | 41.87 | 54.08 | 27 | 50.59 | 28.24 | 20.14 | 36.99 |