Zero-shot Evaluation on 7 tasks zero-shot
72.79Mean Accuracy (Zero-shot)BTC-LLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BTC-LLMBackbone=Qwen2.5-14b, Bits=1.11bit2025.05 | 72.79 | — | |
| FP16Backbone=Qwen3-14b2025.05 | 72.71 | — | |
| FP16Backbone=Qwen2.5-14b2025.05 | 72.25 | — | |
| BTC-LLMBackbone=Qwen2.5-14b, Bits=0.9bit2025.05 | 71.5 | — | |
| FP16Backbone=Qwen3-8b2025.05 | 69.47 | — | |
| DenseModel=LLaMA3-8B, Sparsity=Dense2026.06 | 68.23 | — | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 67.78 | — | |
| BTC-LLMBackbone=Qwen2.5-14b, Bits=0.8bit2025.05 | 67.73 | — | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=30%2026.06 | 67.11 | — | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 67.1 | — | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=30%2026.06 | 67.06 | — | |
| BTC-LLMBackbone=Qwen3-14b, Bits=1.11bit2025.05 | 66.53 | — | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 66.01 | — | |
| DenseArchitecture=Qwen3-8B, Sparsity=0%2026.06 | 65.74 | — | |
| BTC-LLMBackbone=Qwen3-8b, Bits=0.9bit2025.05 | 65.53 | — | |
| BTC-LLMBackbone=Qwen3-8b, Bits=1.11bit2025.05 | 65.45 | — | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=40%2026.06 | 65.38 | — | |
| FP16Backbone=Qwen2.5-3b2025.05 | 65.24 | — | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 65 | — | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=40%2026.06 | 64.87 | — | |
| DenseArchitecture=Mistral-7B, Sparsity=0%2026.06 | 64.37 | — | |
| DenseModel=LLaMA-7B, Sparsity=Dense2026.06 | 64.24 | — | |
| DenseModel=LLaMA2-7B, Sparsity=Dense2026.06 | 64.21 | — | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.6 | — | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.55 | — | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.45 | — | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.42 | — | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=30%2026.06 | 63.37 | — | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=30%2026.06 | 63.33 | — | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.14 | — | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=30%2026.06 | 63.12 | — | |
| DenseArchitecture=LLaMA-2-13B, Sparsity=0%2026.06 | 63.05 | — | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 63.02 | — | |
| BTC-LLMBackbone=Qwen2.5-3b, Bits=1.11bit2025.05 | 62.77 | — | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 62.69 | — | |
| BTC-LLMBackbone=Qwen3-14b, Bits=0.9bit2025.05 | 62.65 | — | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=40%2026.06 | 62.65 | — | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 62.62 | — | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=40%2026.06 | 62.37 | — | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 62.25 | — | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 62.18 | — | |
| BTC-LLMBackbone=Qwen3-8b, Bits=0.8bit2025.05 | 62.11 | — | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=40%2026.06 | 62.04 | — | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 61.99 | — | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=40%2026.06 | 61.9 | — | |
| WandaModel=LLaMA3-8B, Criterion=S-S, Sparsity=50%2026.06 | 61.3 | — | |
| BTC-LLMBackbone=Qwen3-14b, Bits=0.8bit2025.05 | 60.71 | — | |
| DenseArchitecture=Vicuna-7B, Sparsity=0%2026.06 | 60.36 | — | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 60.28 | — | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=50%2026.06 | 60.26 | — | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 60.18 | — | |
| DenseModel=LLaMA-2 70B, Sparsity (%)=0.002025.09 | 60 | 0 | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 59.93 | — | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=50%2026.06 | 59.83 | — | |
| BTC-LLMBackbone=Qwen2.5-3b, Bits=0.9bit2025.05 | 59.8 | — | |
| DenseArchitecture=LLaMA-2-7B, Sparsity=0%2026.06 | 59.69 | — | |
| DenseModel=LLaMA 65B, Sparsity (%)=0.002025.09 | 59.28 | 0 | |
| BTC-LLMBackbone=Qwen3-8b, Bits=0.7bit2025.05 | 59 | — | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 58.88 | — | |
| WandaModel=LLaMA-2 70B, Sparsity (%)=50.002025.09 | 58.81 | -1.19 | |
| EMP-WandaModel=LLaMA-2 70B, Sparsity (%)=40.322025.09 | 58.74 | -1.26 | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=50%2026.06 | 58.62 | — | |
| EMP-MagnitudeModel=LLaMA-2 70B, Sparsity (%)=36.662025.09 | 58.57 | -1.43 | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=50%2026.06 | 58.27 | — | |
| BTC-LLMBackbone=Qwen3-14b, Bits=0.7bit2025.05 | 58.23 | — | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=50%2026.06 | 57.9 | — | |
| MagnitudeModel=LLaMA-2 70B, Sparsity (%)=50.002025.09 | 57.7 | -2.3 | |
| WandaModel=LLaMA 65B, Sparsity (%)=50.002025.09 | 57.4 | -1.88 | |
| DenseArchitecture=LLaMA-3.2-3B, Sparsity=0%2026.06 | 57.24 | — | |
| EMP-WandaModel=LLaMA 65B, Sparsity (%)=39.992025.09 | 57.08 | -2.2 | |
| BTC-LLMBackbone=Qwen2.5-14b, Bits=0.7bit2025.05 | 56.98 | — | |
| EMP-MagnitudeModel=LLaMA 65B, Sparsity (%)=36.612025.09 | 56.95 | -2.33 | |
| BTC-LLMBackbone=Qwen2.5-3b, Bits=0.8bit2025.05 | 55.88 | — | |
| MagnitudeModel=LLaMA 65B, Sparsity (%)=50.002025.09 | 55.83 | -3.45 | |
| EPTSModel=LLaMA-7B, Criterion=M-S, Sparsity=60%2026.06 | 55.35 | — | |
| SparseGPTModel=LLaMA3-8B, Criterion=S-S, Sparsity=60%2026.06 | 55.31 | — | |
| SparseGPTModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 55.21 | — | |
| DenseModel=LLaMA 30B, Sparsity (%)=0.002025.09 | 54.84 | 0 | |
| EMP-MagnitudeModel=LLaMA 30B, Sparsity (%)=36.602025.09 | 54.82 | -0.02 | |
| SparseGPTModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 54.75 | — | |
| EPTSModel=LLaMA2-7B, Criterion=M-S, Sparsity=60%2026.06 | 54.43 | — | |
| EPTSModel=LLaMA3-8B, Criterion=M-S, Sparsity=60%2026.06 | 54.22 | — | |
| WandaModel=LLaMA 30B, Sparsity (%)=50.002025.09 | 54.16 | -0.68 | |
| RIAModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 54.14 | — | |
| WandaModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 53.97 | — | |
| WandaModel=LLaMA2-7B, Criterion=S-S, Sparsity=60%2026.06 | 53.95 | — | |
| DenseModel=LLaMA-2 13B, Sparsity (%)=0.002025.09 | 53.64 | 0 | |
| DenseModel=LLaMA 13B, Sparsity (%)=0.002025.09 | 53.6 | 0 | |
| MagnitudeModel=LLaMA 30B, Sparsity (%)=50.002025.09 | 53.57 | -1.27 | |
| EMP-MagnitudeModel=LLaMA-2 13B, Sparsity (%)=36.622025.09 | 52.93 | -0.71 | |
| EMP-WandaModel=LLaMA 30B, Sparsity (%)=40.182025.09 | 52.9 | -1.94 | |
| EMP-WandaModel=LLaMA 13B, Sparsity (%)=40.682025.09 | 52.74 | -0.86 | |
| MagnitudeModel=LLaMA-2 13B, Sparsity (%)=50.002025.09 | 52.59 | -1.05 | |
| RIAModel=LLaMA3-8B, Criterion=S-S, Sparsity=60%2026.06 | 52.4 | — | |
| RIAModel=LLaMA-7B, Criterion=S-S, Sparsity=60%2026.06 | 52.28 | — | |
| WandaModel=LLaMA-2 13B, Sparsity (%)=50.002025.09 | 52.12 | -1.52 | |
| EMP-MagnitudeModel=LLaMA 13B, Sparsity (%)=36.582025.09 | 52.02 | -1.58 | |
| WandaModel=LLaMA 13B, Sparsity (%)=50.002025.09 | 52 | -1.6 | |
| EMP-WandaModel=LLaMA-2 13B, Sparsity (%)=40.482025.09 | 51.96 | -1.68 | |
| SparseGPT w. MRPArchitecture=LLaMA-2-13B, Sparsity=70%2026.06 | 51.87 | — |