Zero-shot Natural Language Understanding and Reasoning on MMLU, ARC-c, HellaSwag, BOOLQ, PIQA, WinoGrande
69.72Average Score (Zero-shot)Dense
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DenseModel=Mistral-7B, Sparsity=0%2026.02 | 69.72 | 62.65 | 50.34 | 61.24 | 83.64 | 80.58 | 73.8 | |
| DenseModel=LLaMA2-13B, Sparsity=0%2026.02 | 65.93 | 55.13 | 48.46 | 60.06 | 80.61 | 79.05 | 72.22 | |
| DenseModel=Qwen1.5-7B, Sparsity=0%2026.02 | 65.78 | 60.53 | 40.61 | 57.8 | 82.51 | 78.4 | 74.9 | |
| DenseModel=LLaMA2-7B, Sparsity=0%2026.02 | 62.72 | 45.84 | 43.43 | 57.13 | 77.74 | 78.07 | 74.11 | |
| ActTailModel=LLaMA2-13B, Sparsity=70%2026.02 | 62.18 | 36.88 | 40.27 | 49.55 | 72.51 | 74.1 | 63.77 | |
| ActTailModel=Mistral-7B, Sparsity=70%2026.02 | 59.42 | 33.2 | 34.39 | 46.78 | 67.49 | 73.23 | 64.25 | |
| TEALTop-KModel=Mistral-7B, Sparsity=70%2026.02 | 59.17 | 30.37 | 34.3 | 47.17 | 67.8 | 73.23 | 62.67 | |
| ActTailModel=LLaMA2-7B, Sparsity=70%2026.02 | 58.06 | 27.35 | 30.97 | 44.07 | 66.97 | 72.25 | 56.12 | |
| TEALTop-KModel=LLaMA2-13B, Sparsity=70%2026.02 | 58.06 | 30.63 | 33.96 | 46.51 | 68.23 | 72.8 | 61.4 | |
| TEALTop-KModel=LLaMA2-7B, Sparsity=70%2026.02 | 56.02 | 26.53 | 28.5 | 41.84 | 65.66 | 70.67 | 50.36 | |
| ActTailModel=Qwen1.5-7B, Sparsity=70%2026.02 | 55.85 | 36.5 | 32 | 44.66 | 69.69 | 70.78 | 58.25 | |
| TEALTop-KModel=Qwen1.5-7B, Sparsity=70%2026.02 | 54.48 | 31.99 | 31.23 | 43.24 | 67.06 | 71.11 | 53.04 | |
| TEALTop-KModel=Qwen1.5-7B, Sparsity=80%2026.02 | 53.32 | 25.84 | 19.62 | 30.72 | 60.73 | 64.15 | 50.99 | |
| ActTailModel=LLaMA2-13B, Sparsity=80%2026.02 | 52.61 | 26.61 | 25.51 | 36.23 | 64.16 | 63.22 | 52.93 | |
| ActTailModel=Qwen1.5-7B, Sparsity=80%2026.02 | 52.47 | 25.32 | 21.76 | 30.77 | 60.4 | 61.37 | 50.91 | |
| ActTailModel=LLaMA2-7B, Sparsity=80%2026.02 | 49.27 | 25.32 | 21.08 | 30.09 | 56.94 | 61.92 | 51.22 | |
| ActTailModel=Mistral-7B, Sparsity=80%2026.02 | 49.15 | 24.7 | 21.16 | 32.1 | 51.72 | 63.87 | 50.91 | |
| TEALTop-KModel=LLaMA2-13B, Sparsity=80%2026.02 | 48.73 | 24.95 | 20.22 | 30.62 | 51.47 | 63.38 | 51.78 | |
| TEALTop-KModel=Mistral-7B, Sparsity=80%2026.02 | 48.12 | 24.37 | 18.77 | 31.64 | 44.46 | 63.6 | 51.46 | |
| TEALTop-KModel=LLaMA2-7B, Sparsity=80%2026.02 | 47.87 | 23.58 | 19.62 | 28.97 | 44.62 | 60.5 | 48.78 |