Question Answering on ARC-e (Accuracy %)
82.55Accuracy (%)Dense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseParams=LLaMA-2 70B, Sparsity=0%, Evaluation protocol=zero-shot2025.11 | 82.55 | |
| SparseGPTParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 82.4 | |
| AutoPruneParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot, Quantization=8-bit2025.11 | 81.2 | |
| WandaParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 80.8 | |
| DenseParams=LLaMA-2 13B, Sparsity=0%, Evaluation protocol=zero-shot2025.11 | 79.42 | |
| DenseParams=LLaMA-2 7B, Sparsity=0%, Evaluation protocol=zero-shot2025.11 | 76.39 | |
| AutoPruneParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 76.35 | |
| WandaParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 76.18 | |
| MagnitudeParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 75.7 | |
| SparseGPTParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 74.83 | |
| AutoPruneParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 74.75 | |
| SparseGPTParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 73.27 | |
| CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 73.06 | |
| WandaParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 72.77 | |
| CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 72.54 | |
| GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 72.43 | |
| Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 72.38 | |
| GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 71.89 | |
| KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 71.59 | |
| KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.83 | |
| Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.68 | |
| MagnitudeParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 70.54 | |
| Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.54 | |
| Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.54 | |
| Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.5 | |
| Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.45 | |
| Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 70.41 | |
| TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 69.23 | |
| Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 68.81 | |
| MagnitudeParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.11 | 64.1 | |
| FP16Backbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.06 | 47.83 | |
| SAGE-PTQBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.06 | 46.49 | |
| FP16Backbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.06 | 43.81 | |
| BiLLMBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.06 | 43.8 | |
| SAGE-PTQBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.06 | 39.8 | |
| FP16Backbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.06 | 31.1 | |
| BiLLMBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.06 | 30.75 | |
| SAGE-PTQBackbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.06 | 26.76 | |
| BiLLMBackbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.06 | 23.9 |