Zero-shot Language Understanding on ARC-c, ARC-e, PIQA, Winogrande, and Hellaswag
6,257Mean AccuracyRIA+SQ+VC+EBFT
Evaluation Results
| Method | Links | |
|---|---|---|
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=8:162025.07 | 6,257 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=8:162025.07 | 6,215 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=8:162025.07 | 6,191 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=8:162025.07 | 6,181 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=8:162025.07 | 6,127 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=8:162025.07 | 6,118 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=8:162025.07 | 6,112 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=8:162025.07 | 6,108 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=8:162025.07 | 6,104 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=8:162025.07 | 6,101 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=8:162025.07 | 6,095 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=8:162025.07 | 6,056 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=2:42025.07 | 6,020 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=2:42025.07 | 5,974 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=2:42025.07 | 5,941 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=2:42025.07 | 5,916 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=2:42025.07 | 5,872 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=2:42025.07 | 5,853 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=2:42025.07 | 5,787 | |
| RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=2:42025.07 | 5,784 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=2:42025.07 | 5,748 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=2:42025.07 | 5,740 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=2:42025.07 | 5,644 | |
| RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=2:42025.07 | 5,597 | |
| AQLM (Ind.)Model=Qwen 14B, Codebooks=5 × 82026.06 | 76.85 | |
| AQLM (Drop)Model=Qwen 14B, Codebooks=5 × 82026.06 | 76.85 | |
| DbyD (35W)Model=Qwen 14B, Codebooks=5 × 82026.06 | 76.76 | |
| DbyD (Uni.)Model=Qwen 14B, Codebooks=5 × 82026.06 | 76.03 | |
| DbyD (35W)Model=Qwen 14B, Codebooks=4 × 82026.06 | 75.99 | |
| AQLM (Ind.)Model=Qwen 14B, Codebooks=4 × 82026.06 | 75.94 | |
| DbyD (Uni.)Model=Qwen 14B, Codebooks=4 × 82026.06 | 75.8 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=3 × 82026.06 | 74.94 | |
| AQLM (Ind.)Model=Qwen 14B, Codebooks=3 × 82026.06 | 74.89 | |
| DbyD (35W)Model=Qwen 32B, Codebooks=5 × 82026.06 | 74.74 | |
| AQLM (Drop)Model=Qwen 14B, Codebooks=4 × 82026.06 | 74.73 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=5 × 82026.06 | 74.53 | |
| AQLM (Drop)Model=Qwen 32B, Codebooks=5 × 82026.06 | 74.53 | |
| AQLM (Ind.)Model=Qwen 32B, Codebooks=4 × 82026.06 | 74.5 | |
| DbyD (Uni.)Model=Qwen 32B, Codebooks=5 × 82026.06 | 74.2 | |
| DbyD (35W)Model=Qwen 32B, Codebooks=4 × 82026.06 | 73.98 | |
| DbyD (35W)Model=Qwen 14B, Codebooks=3 × 82026.06 | 73.93 | |
| DbyD (Uni.)Model=Qwen 14B, Codebooks=3 × 82026.06 | 73.92 | |
| DbyD (35W)Model=Qwen 7B, Codebooks=5 × 82026.06 | 73.72 | |
| AQLM (Ind.)Model=Qwen 7B, Codebooks=5 × 82026.06 | 73.59 | |
| AQLM (Drop)Model=Qwen 7B, Codebooks=5 × 82026.06 | 73.59 | |
| AQLM (Drop)Model=Qwen 32B, Codebooks=4 × 82026.06 | 73.1 | |
| DbyD (35W)Model=Qwen 32B, Codebooks=3 × 82026.06 | 72.94 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=5 × 82026.06 | 72.85 | |
| DbyD (Uni.)Model=Qwen 7B, Codebooks=5 × 82026.06 | 72.75 | |
| DbyD (Uni.)Model=Qwen 32B, Codebooks=4 × 82026.06 | 72.56 | |
| DbyD (35W)Model=Qwen 7B, Codebooks=4 × 82026.06 | 72.52 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=4 × 82026.06 | 72.46 | |
| AQLM (Ind.)Model=MetaLlama 8B, Codebooks=5 × 82026.06 | 72.42 | |
| AQLM (Drop)Model=MetaLlama 8B, Codebooks=5 × 82026.06 | 72.42 | |
| AQLM (Ind.)Model=Qwen 7B, Codebooks=4 × 82026.06 | 72.37 | |
| DbyD (Uni.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 72.15 | |
| DbyD (35W)Model=MetaLlama 8B, Codebooks=5 × 82026.06 | 72.09 | |
| DbyD (Uni.)Model=Qwen 32B, Codebooks=3 × 82026.06 | 71.93 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=5 × 82026.06 | 71.72 | |
| AQLM (Drop)Model=Mistral 7B, Codebooks=5 × 82026.06 | 71.72 | |
| DbyD (Uni.)Model=MetaLlama 8B, Codebooks=5 × 82026.06 | 71.62 | |
| AQLM (Drop)Model=Qwen 7B, Codebooks=4 × 82026.06 | 71.4 | |
| DbyD (Uni.)Model=Qwen 7B, Codebooks=4 × 82026.06 | 71.32 | |
| AQLM (Drop)Model=Qwen 32B, Codebooks=3 × 82026.06 | 71.08 | |
| AQLM (Ind.)Model=MetaLlama 8B, Codebooks=4 × 82026.06 | 71 | |
| AQLM (Ind.)Model=Mistral 7B, Codebooks=3 × 82026.06 | 70.97 | |
| AQLM (Ind.)Model=Qwen 7B, Codebooks=3 × 82026.06 | 70.83 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=4 × 82026.06 | 70.81 | |
| AQLM (Ind.)Model=MetaLlama 8B, Codebooks=3 × 82026.06 | 70 | |
| AQLM (Drop)Model=Qwen 14B, Codebooks=3 × 82026.06 | 70 | |
| DbyD (Uni.)Model=Mistral 7B, Codebooks=4 × 82026.06 | 69.96 | |
| AQLM (Drop)Model=Mistral 7B, Codebooks=4 × 82026.06 | 69.94 | |
| DbyD (35W)Model=MetaLlama 8B, Codebooks=4 × 82026.06 | 69.6 | |
| DbyD (35W)Model=Qwen 7B, Codebooks=3 × 82026.06 | 69.54 | |
| DbyD (Uni.)Model=MetaLlama 8B, Codebooks=4 × 82026.06 | 68.52 | |
| DbyD (Uni.)Model=Qwen 3B, Codebooks=5 × 82026.06 | 68.18 | |
| DbyD (35W)Model=Qwen 3B, Codebooks=5 × 82026.06 | 68.17 | |
| AQLM (Ind.)Model=Qwen 3B, Codebooks=5 × 82026.06 | 67.83 | |
| AQLM (Drop)Model=Qwen 3B, Codebooks=5 × 82026.06 | 67.83 | |
| DbyD (35W)Model=Mistral 7B, Codebooks=3 × 82026.06 | 67.48 | |
| DbyD (Uni.)Model=Qwen 7B, Codebooks=3 × 82026.06 | 67.3 | |
| DbyD (35W)Model=Qwen 3B, Codebooks=4 × 82026.06 | 67.28 | |
| AQLM (Drop)Model=Qwen 7B, Codebooks=3 × 82026.06 | 66.77 | |
| AQLM (Ind.)Model=Qwen 3B, Codebooks=4 × 82026.06 | 66.68 | |
| DbyD (Uni.)Model=Qwen 3B, Codebooks=4 × 82026.06 | 65.85 | |
| AQLM (Drop)Model=MetaLlama 8B, Codebooks=4 × 82026.06 | 65.54 | |
| AQLM (Ind.)Model=Qwen 3B, Codebooks=3 × 82026.06 | 65.48 | |
| DbyD (35W)Model=MetaLlama 8B, Codebooks=3 × 82026.06 | 64.81 | |
| Dense ModelModel=LLaMA2-7B2025.07 | 64.79 | |
| DbyD (35W)Model=Qwen 3B, Codebooks=3 × 82026.06 | 63.06 | |
| DbyD (Uni.)Model=Qwen 3B, Codebooks=3 × 82026.06 | 62.53 | |
| AQLM (Drop)Model=Qwen 3B, Codebooks=4 × 82026.06 | 62.11 | |
| DbyD (Uni.)Model=Mistral 7B, Codebooks=3 × 82026.06 | 61.78 | |
| DbyD (Uni.)Model=MetaLlama 8B, Codebooks=3 × 82026.06 | 55.8 | |
| DbyD (35W)Model=Qwen 0.5B, Codebooks=5 × 82026.06 | 53.86 | |
| DbyD (Uni.)Model=Qwen 0.5B, Codebooks=5 × 82026.06 | 53.62 | |
| AQLM (Ind.)Model=Qwen 0.5B, Codebooks=4 × 82026.06 | 53.6 | |
| AQLM (Ind.)Model=Qwen 0.5B, Codebooks=5 × 82026.06 | 53.25 | |
| AQLM (Drop)Model=Qwen 0.5B, Codebooks=5 × 82026.06 | 53.25 | |
| DbyD (35W)Model=Qwen 0.5B, Codebooks=4 × 82026.06 | 53.14 |