List your company
ResearchFormerly WizWand

Question Answering on ARC-e (Accuracy %)

82.55Accuracy (%)Dense
21.55437.389553.22569.0605Nov 19, 2025Dec 25, 2025Jan 30, 2026Mar 8, 2026Apr 13, 2026May 19, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
DenseParams=LLaMA-2 70B, Sparsity=0%, Evaluation protocol=zero-shot2025.1182.55
SparseGPTParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1182.4
AutoPruneParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot, Quantization=8-bit2025.1181.2
WandaParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1180.8
DenseParams=LLaMA-2 13B, Sparsity=0%, Evaluation protocol=zero-shot2025.1179.42
DenseParams=LLaMA-2 7B, Sparsity=0%, Evaluation protocol=zero-shot2025.1176.39
AutoPruneParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1176.35
WandaParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1176.18
MagnitudeParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1175.7
SparseGPTParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1174.83
AutoPruneParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1174.75
SparseGPTParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1173.27
CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0673.06
WandaParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1172.77
CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0672.54
GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0672.43
Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0672.38
GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0671.89
KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0671.59
KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0670.83
Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0670.68
MagnitudeParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1170.54
Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0670.54
Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0670.54
Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0670.5
Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0670.45
Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0670.41
TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0669.23
Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0668.81
MagnitudeParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1164.1
FP16Backbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0647.83
SAGE-PTQBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0646.49
FP16Backbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0643.81
BiLLMBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0643.8
SAGE-PTQBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0639.8
FP16Backbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.0631.1
BiLLMBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0630.75
SAGE-PTQBackbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.0626.76
BiLLMBackbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.0623.9