Commonsense Reasoning on LM Eval Harness Suite
48.89LM Eval ScoreSparseGPT+SEFT
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SparseGPT+SEFTBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 48.89 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+SPPBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 48.33 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+SEFTBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 47.95 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+SQFTBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 47.7 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+SPPBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 47.27 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+SQFTBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 47.09 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+LoRA*Backbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 46.11 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SEFTBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 45.61 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPT+LoRA*Backbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 45.42 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SQFTBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 44.73 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SEFTBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 44.55 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SPPBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 44.46 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+LoRA*Backbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 44.29 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SPPBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 43.68 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+SQFTBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 43.56 | — | — | — | — | — | — | — | — | — | — | — | |
| Wanda+LoRA*Backbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 43.32 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPTBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 41.7 | — | — | — | — | — | — | — | — | — | — | — | |
| SparseGPTBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 41.51 | — | — | — | — | — | — | — | — | — | — | — | |
| WandaBackbone=LLaMA V3-8B, Sparsity Level=0.72025.05 | 34.92 | — | — | — | — | — | — | — | — | — | — | — | |
| WandaBackbone=LLaMA V2-7B, Sparsity Level=0.72025.05 | 34.83 | — | — | — | — | — | — | — | — | — | — | — | |
| Gated DeltaNetarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 48.92 | 66.94 | 77.26 | 73.82 | 40.53 | 60.14 | 65.41 | 39.6 | 78 | 92.2 | 64.28 | |
| GRUarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 43.94 | 67.36 | 76.82 | 75.67 | 43.52 | 58.96 | 63.79 | 41 | 82 | 91.3 | 64.44 | |
| Hybrid Gated DeltaNetarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 50.2 | 67.23 | 77.09 | 73.06 | 41.64 | 59.12 | 62.39 | 40.8 | 82 | 92.9 | 64.64 | |
| Hybrid Gated DeltaNet + M2RNN-1architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 51.54 | 68.69 | 78.18 | 74.45 | 41.81 | 60.3 | 65.08 | 42 | 82 | 93.9 | 65.8 | |
| Hybrid Gated DeltaNet + M2RNN-5architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 51.8 | 68.99 | 76.82 | 76.73 | 43.6 | 60.69 | 63.3 | 40.2 | 81 | 93.5 | 65.66 | |
| Hybrid M2RNNarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 51.74 | 68.44 | 76.55 | 76.81 | 43.09 | 61.09 | 63.21 | 40.4 | 81 | 93.3 | 65.56 | |
| Hybrid Mamba-2architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 50.18 | 68.62 | 76.55 | 72.64 | 43.09 | 60.62 | 63.61 | 43 | 79 | 93 | 65.03 | |
| Hybrid Mamba-2 + M2RNN-1architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 50.92 | 68.35 | 77.86 | 74.66 | 42.24 | 60.54 | 62.57 | 40.4 | 83 | 92.8 | 65.33 | |
| Hybrid Mamba-2 + M2RNN-5architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 49.84 | 68.19 | 77.04 | 75.17 | 42.58 | 59.04 | 65.26 | 42.4 | 83 | 93.9 | 65.64 | |
| M2RNNarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 49.18 | 67.57 | 76.71 | 75.51 | 44.97 | 58.17 | 64.16 | 41.4 | 82 | 92 | 65.17 | |
| Mamba-2architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 50.09 | 67.43 | 76.82 | 74.33 | 43.52 | 61.64 | 64.01 | 41 | 80 | 93.4 | 65.22 | |
| RNNarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 30.25 | 64.49 | 76.61 | 72.64 | 41.04 | 55.64 | 58.35 | 40.2 | 79 | 87.6 | 60.58 | |
| Transformer++architecture=7B (1B active) MoE, setting=0-shot2026.03 | — | 43.9 | 62.99 | 75.63 | 71.97 | 38.91 | 57.54 | 63.12 | 37.2 | 75 | 91.4 | 61.76 |