Zero-shot Evaluation on ARC (c/e), OBQA, WinoGrande, PIQA, HellaSwag, MathQA, RTE, and BoolQ
71.08Mean AccuracyOriginal (Dense)
Evaluation Results
| Method | Links | |
|---|---|---|
| Original (Dense)Model=Qwen3-30B-A3B, Sparsity Ratio=20%2026.03 | 71.08 | |
| Original (Dense)Model=Qwen3-30B-A3B, Sparsity Ratio=40%2026.03 | 71.08 | |
| Original (Dense)Model=Qwen3-30B-A3B, Sparsity Ratio=60%2026.03 | 71.08 | |
| OursModel=Qwen3-30B-A3B, Sparsity Ratio=20%2026.03 | 70.04 | |
| NAEEModel=Qwen3-30B-A3B, Sparsity Ratio=20%2026.03 | 69.94 | |
| Camera-PModel=Qwen3-30B-A3B, Sparsity Ratio=20%2026.03 | 69.94 | |
| Original (Dense)Model=LLaMA-13B, Sparsity Ratio=20%2026.03 | 68.79 | |
| Original (Dense)Model=LLaMA-13B, Sparsity Ratio=50%2026.03 | 68.79 | |
| OursModel=Qwen3-30B-A3B, Sparsity Ratio=40%2026.03 | 67.65 | |
| Camera-PModel=Qwen3-30B-A3B, Sparsity Ratio=40%2026.03 | 67.46 | |
| DDPModel=LLaMA-13B, Sparsity Ratio=20%2026.03 | 66.89 | |
| Original (Dense)Model=LLaMA-2-7B, Sparsity Ratio=20%2026.03 | 66.63 | |
| Original (Dense)Model=LLaMA-2-7B, Sparsity Ratio=50%2026.03 | 66.63 | |
| D2-MoEModel=Qwen3-30B-A3B, Sparsity Ratio=20%2026.03 | 66.35 | |
| Original (Dense)Model=LLaMA-7B, Sparsity Ratio=20%2026.03 | 65.96 | |
| Original (Dense)Model=LLaMA-7B, Sparsity Ratio=50%2026.03 | 65.96 | |
| NAEEModel=Qwen3-30B-A3B, Sparsity Ratio=40%2026.03 | 65.77 | |
| DDPModel=LLaMA-2-7B, Sparsity Ratio=20%2026.03 | 64.82 | |
| LoRAPModel=LLaMA-13B, Sparsity Ratio=20%2026.03 | 64.71 | |
| SlimLLMModel=LLaMA-13B, Sparsity Ratio=20%2026.03 | 64.21 | |
| DDPModel=LLaMA-7B, Sparsity Ratio=20%2026.03 | 64.13 | |
| OursModel=Qwen3-30B-A3B, Sparsity Ratio=60%2026.03 | 63.35 | |
| D2-MoEModel=Qwen3-30B-A3B, Sparsity Ratio=40%2026.03 | 62.77 | |
| SlimLLMModel=LLaMA-7B, Sparsity Ratio=20%2026.03 | 62.41 | |
| Original (Dense)Model=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 62.28 | |
| Original (Dense)Model=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 62.28 | |
| Original (Dense)Model=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 62.28 | |
| DDPModel=LLaMA-13B, Sparsity Ratio=50%2026.03 | 62.14 | |
| OursModel=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 61.84 | |
| LoRAPModel=LLaMA-7B, Sparsity Ratio=20%2026.03 | 61.7 | |
| SlimLLMModel=LLaMA-2-7B, Sparsity Ratio=20%2026.03 | 61.7 | |
| LoRAPModel=LLaMA-2-7B, Sparsity Ratio=20%2026.03 | 61.2 | |
| Camera-PModel=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 61.03 | |
| HEAPrModel=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 60.62 | |
| NAEEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 60.51 | |
| OursModel=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 60.38 | |
| LoRAPruneModel=LLaMA-7B, Sparsity Ratio=20%2026.03 | 60.05 | |
| Camera-PModel=Qwen3-30B-A3B, Sparsity Ratio=60%2026.03 | 59.03 | |
| D2-MoEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=20%2026.03 | 58.97 | |
| Camera-PModel=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 58.58 | |
| LoRAPModel=LLaMA-13B, Sparsity Ratio=50%2026.03 | 58.2 | |
| OursModel=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 58.18 | |
| HEAPrModel=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 57.7 | |
| NAEEModel=Qwen3-30B-A3B, Sparsity Ratio=60%2026.03 | 57.13 | |
| DDPModel=LLaMA-2-7B, Sparsity Ratio=50%2026.03 | 56.7 | |
| DDPModel=LLaMA-7B, Sparsity Ratio=50%2026.03 | 56.07 | |
| NAEEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 54.94 | |
| SlimLLMModel=LLaMA-13B, Sparsity Ratio=50%2026.03 | 54.75 | |
| D2-MoEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=40%2026.03 | 54.32 | |
| SlimLLMModel=LLaMA-7B, Sparsity Ratio=50%2026.03 | 53.16 | |
| LoRAPModel=LLaMA-2-7B, Sparsity Ratio=50%2026.03 | 52.31 | |
| LoRAPModel=LLaMA-7B, Sparsity Ratio=50%2026.03 | 52.17 | |
| D2-MoEModel=Qwen3-30B-A3B, Sparsity Ratio=60%2026.03 | 52.02 | |
| SlimLLMModel=LLaMA-2-7B, Sparsity Ratio=50%2026.03 | 52.02 | |
| Camera-PModel=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 51.62 | |
| HEAPrModel=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 51.38 | |
| LoRAPruneModel=LLaMA-7B, Sparsity Ratio=50%2026.03 | 49.71 | |
| D2-MoEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 46.72 | |
| NAEEModel=DeepSeekMoE-16B-Base, Sparsity Ratio=60%2026.03 | 45.28 |