Zero-shot Evaluation on ARC (C/E), HellaSwag, LAMBADA (OpenAI/Standard), PIQA, and WinoGrande
76.71Accuracy (0-shot)FP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Backbone=LLaMA2-70B, #Bits (W)=16, #Bits (A)=162026.06 | 76.71 | |
| TWLABackbone=LLaMA2-70B, #Bits (W)=1.58, #Bits (A)=162026.06 | 73.6 | |
| TWLABackbone=LLaMA2-70B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 73.45 | |
| FP16Backbone=LLaMA3-8B, #Bits (W)=16, #Bits (A)=162026.06 | 72.51 | |
| FP16Backbone=Qwen3-14B, #Bits (W)=16, #Bits (A)=162026.06 | 72.47 | |
| FP16Backbone=Qwen3-32B, #Bits (W)=16, #Bits (A)=162026.06 | 72.42 | |
| FP16Backbone=LLaMA2-13B, #Bits (W)=16, #Bits (A)=162026.06 | 72.19 | |
| TWLABackbone=LLaMA2-70B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 71.1 | |
| TWLABackbone=Qwen3-32B, #Bits (W)=1.58, #Bits (A)=162026.06 | 69.54 | |
| FP16Backbone=LLaMA2-7B, #Bits (W)=16, #Bits (A)=162026.06 | 69.49 | |
| FP16Backbone=Qwen3-8B, #Bits (W)=16, #Bits (A)=162026.06 | 69.09 | |
| QuaRotBackbone=LLaMA2-70B, #Bits (W)=2, #Bits (A)=162026.06 | 68.79 | |
| TWLABackbone=Qwen3-14B, #Bits (W)=1.58, #Bits (A)=162026.06 | 68.48 | |
| TWLABackbone=Qwen3-32B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 68.13 | |
| QuaRotBackbone=LLaMA2-70B, #Bits (W)=2, #Bits (A)=62026.06 | 67.97 | |
| TWLABackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=162026.06 | 67.7 | |
| TWLABackbone=Qwen3-14B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 67.29 | |
| TWLABackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 66.88 | |
| TWLABackbone=Qwen3-32B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 65.25 | |
| SliM-LLMBackbone=Qwen3-32B, #Bits (W)=2MP, #Bits (A)=162026.06 | 64.79 | |
| TWLABackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 64.3 | |
| PB-LLMBackbone=Qwen3-32B, #Bits (W)=1.7, #Bits (A)=162026.06 | 63.92 | |
| TWLABackbone=LLaMA3-8B, #Bits (W)=1.58, #Bits (A)=162026.06 | 62.98 | |
| TWLABackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=162026.06 | 62.91 | |
| PT2-LLMBackbone=LLaMA2-70B, #Bits (W)=1.58, #Bits (A)=162026.06 | 62.47 | |
| TWLABackbone=Qwen3-8B, #Bits (W)=1.58, #Bits (A)=162026.06 | 62.05 | |
| TWLABackbone=Qwen3-14B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 62 | |
| TWLABackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 61.98 | |
| TWLABackbone=LLaMA3-8B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 61.44 | |
| TWLABackbone=Qwen3-8B, #Bits (W)=1.58, #Bits (A)=6MP2026.06 | 59.03 | |
| TWLABackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 58 | |
| ResQBackbone=LLaMA2-70B, #Bits (W)=2.3MP, #Bits (A)=6.1MP2026.06 | 57.81 | |
| PT2-LLMBackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=162026.06 | 56.54 | |
| ResQBackbone=LLaMA2-13B, #Bits (W)=2.3MP, #Bits (A)=6.1MP2026.06 | 56.49 | |
| ResQBackbone=LLaMA2-70B, #Bits (W)=2.3MP, #Bits (A)=4.2MP2026.06 | 56.43 | |
| TWLABackbone=LLaMA3-8B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 55.23 | |
| QuaRotBackbone=LLaMA2-70B, #Bits (W)=2, #Bits (A)=42026.06 | 53.64 | |
| SliM-LLMBackbone=Qwen3-32B, #Bits (W)=2MP, #Bits (A)=62026.06 | 52.43 | |
| SliM-LLMBackbone=LLaMA2-13B, #Bits (W)=2MP, #Bits (A)=162026.06 | 52 | |
| ResQBackbone=LLaMA2-13B, #Bits (W)=2.3MP, #Bits (A)=4.2MP2026.06 | 51.84 | |
| PB-LLMBackbone=Qwen3-32B, #Bits (W)=1.7, #Bits (A)=62026.06 | 51.45 | |
| ResQBackbone=LLaMA2-7B, #Bits (W)=2.3MP, #Bits (A)=6.1MP2026.06 | 51.18 | |
| PB-LLMBackbone=LLaMA2-70B, #Bits (W)=1.7, #Bits (A)=162026.06 | 50.88 | |
| PT2-LLMBackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=62026.06 | 50.82 | |
| TWLABackbone=Qwen3-8B, #Bits (W)=1.58, #Bits (A)=4MP2026.06 | 50.42 | |
| QuaRotBackbone=LLaMA2-13B, #Bits (W)=2, #Bits (A)=162026.06 | 49.3 | |
| SliM-LLMBackbone=LLaMA2-7B, #Bits (W)=2MP, #Bits (A)=162026.06 | 48.96 | |
| PT2-LLMBackbone=LLaMA2-70B, #Bits (W)=1.58, #Bits (A)=62026.06 | 48.96 | |
| SliM-LLMBackbone=Qwen3-14B, #Bits (W)=2MP, #Bits (A)=162026.06 | 48.1 | |
| QuaRotBackbone=LLaMA2-13B, #Bits (W)=2, #Bits (A)=62026.06 | 47.95 | |
| ResQBackbone=LLaMA3-8B, #Bits (W)=2.3MP, #Bits (A)=6.1MP2026.06 | 47.57 | |
| PT2-LLMBackbone=Qwen3-14B, #Bits (W)=1.58, #Bits (A)=162026.06 | 46.35 | |
| GPTQBackbone=LLaMA2-70B, #Bits (W)=2, #Bits (A)=162026.06 | 45.59 | |
| ResQBackbone=LLaMA2-7B, #Bits (W)=2.3MP, #Bits (A)=4.2MP2026.06 | 45.23 | |
| PB-LLMBackbone=Qwen3-14B, #Bits (W)=1.7, #Bits (A)=162026.06 | 44.8 | |
| SliM-LLMBackbone=LLaMA2-13B, #Bits (W)=2MP, #Bits (A)=62026.06 | 43.32 | |
| PT2-LLMBackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=162026.06 | 42.82 | |
| PB-LLMBackbone=LLaMA2-70B, #Bits (W)=1.7, #Bits (A)=62026.06 | 40.89 | |
| QuaRotBackbone=LLaMA2-13B, #Bits (W)=2, #Bits (A)=42026.06 | 39.42 | |
| SliM-LLMBackbone=Qwen3-8B, #Bits (W)=2MP, #Bits (A)=162026.06 | 39.13 | |
| ResQBackbone=LLaMA3-8B, #Bits (W)=2.3MP, #Bits (A)=4.2MP2026.06 | 39.12 | |
| PT2-LLMBackbone=LLaMA3-8B, #Bits (W)=1.58, #Bits (A)=162026.06 | 39.04 | |
| PT2-LLMBackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=62026.06 | 38.61 | |
| PB-LLMBackbone=Qwen3-8B, #Bits (W)=1.7, #Bits (A)=162026.06 | 38.43 | |
| GPTQBackbone=Qwen3-32B, #Bits (W)=2, #Bits (A)=162026.06 | 38.18 | |
| SliM-LLMBackbone=Qwen3-14B, #Bits (W)=2MP, #Bits (A)=62026.06 | 37.46 | |
| PB-LLMBackbone=Qwen3-14B, #Bits (W)=1.7, #Bits (A)=62026.06 | 37.28 | |
| SliM-LLMBackbone=LLaMA2-7B, #Bits (W)=2MP, #Bits (A)=62026.06 | 37.06 | |
| QuaRotBackbone=LLaMA2-7B, #Bits (W)=2, #Bits (A)=162026.06 | 36.3 | |
| PT2-LLMBackbone=Qwen3-14B, #Bits (W)=1.58, #Bits (A)=62026.06 | 36 | |
| GPTQBackbone=LLaMA2-70B, #Bits (W)=2, #Bits (A)=62026.06 | 35.89 | |
| QuaRotBackbone=LLaMA3-8B, #Bits (W)=2, #Bits (A)=162026.06 | 35.66 | |
| QuaRotBackbone=LLaMA3-8B, #Bits (W)=2, #Bits (A)=62026.06 | 35.51 | |
| QuaRotBackbone=LLaMA2-7B, #Bits (W)=2, #Bits (A)=62026.06 | 35.22 | |
| PB-LLMBackbone=Qwen3-8B, #Bits (W)=1.7, #Bits (A)=62026.06 | 31.67 | |
| SliM-LLMBackbone=Qwen3-8B, #Bits (W)=2MP, #Bits (A)=62026.06 | 31.41 | |
| GPTQBackbone=Qwen3-32B, #Bits (W)=2, #Bits (A)=62026.06 | 31.15 | |
| QuaRotBackbone=LLaMA2-7B, #Bits (W)=2, #Bits (A)=42026.06 | 30.83 | |
| PT2-LLMBackbone=LLaMA3-8B, #Bits (W)=1.58, #Bits (A)=62026.06 | 30.81 | |
| PB-LLMBackbone=LLaMA2-7B, #Bits (W)=1.7, #Bits (A)=62026.06 | 30.6 | |
| PB-LLMBackbone=LLaMA3-8B, #Bits (W)=1.7, #Bits (A)=62026.06 | 30.28 | |
| PB-LLMBackbone=LLaMA3-8B, #Bits (W)=1.7, #Bits (A)=162026.06 | 29.32 | |
| GPTQBackbone=Qwen3-8B, #Bits (W)=2, #Bits (A)=162026.06 | 29.25 | |
| PB-LLMBackbone=Qwen3-14B, #Bits (W)=1.7, #Bits (A)=42026.06 | 29.23 | |
| GPTQBackbone=LLaMA2-7B, #Bits (W)=2, #Bits (A)=162026.06 | 29.09 | |
| PT2-LLMBackbone=LLaMA2-13B, #Bits (W)=1.58, #Bits (A)=42026.06 | 29.07 | |
| PB-LLMBackbone=LLaMA2-7B, #Bits (W)=1.7, #Bits (A)=162026.06 | 29.01 | |
| GPTQBackbone=Qwen3-14B, #Bits (W)=2, #Bits (A)=162026.06 | 28.61 | |
| GPTQBackbone=LLaMA2-7B, #Bits (W)=2, #Bits (A)=62026.06 | 27.94 | |
| QuaRotBackbone=LLaMA3-8B, #Bits (W)=2, #Bits (A)=42026.06 | 27.69 | |
| SliM-LLMBackbone=LLaMA3-8B, #Bits (W)=2MP, #Bits (A)=162026.06 | 27.49 | |
| PT2-LLMBackbone=LLaMA2-7B, #Bits (W)=1.58, #Bits (A)=42026.06 | 27.31 | |
| GPTQBackbone=LLaMA2-13B, #Bits (W)=2, #Bits (A)=162026.06 | 27.21 | |
| SliM-LLMBackbone=LLaMA3-8B, #Bits (W)=2MP, #Bits (A)=62026.06 | 27.15 | |
| GPTQBackbone=Qwen3-14B, #Bits (W)=2, #Bits (A)=62026.06 | 26.76 | |
| GPTQBackbone=Qwen3-8B, #Bits (W)=2, #Bits (A)=62026.06 | 26.65 | |
| PB-LLMBackbone=LLaMA2-13B, #Bits (W)=1.7, #Bits (A)=62026.06 | 26.61 | |
| PB-LLMBackbone=LLaMA2-7B, #Bits (W)=1.7, #Bits (A)=42026.06 | 26.46 | |
| SliM-LLMBackbone=LLaMA2-13B, #Bits (W)=2MP, #Bits (A)=42026.06 | 26.44 | |
| GPTQBackbone=LLaMA3-8B, #Bits (W)=2, #Bits (A)=162026.06 | 26.38 |