List your company
ResearchFormerly WizWand

Question Answering on BoolQ (Accuracy)

90.03AccuracyShortGPT
62.750869.832976.91583.9971Jun 17, 2025Aug 18, 2025Oct 19, 2025Dec 20, 2025Feb 20, 2026Apr 23, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
ShortGPTModel=Llama3-8B, Ratio=21.90%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0590.03
ShortGPTModel=Llama3-8B, Ratio=12.50%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.85
IT-PrunModel=Llama3-8B, Ratio=31.20%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.66
IT-PrunModel=Llama3-8B, Ratio=12.50%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.64
SLEBModel=Llama3-8B, Ratio=12.50%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.6
IT-PrunModel=Llama3-8B, Ratio=21.90%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.18
ShortGPTModel=Llama3-8B, Ratio=31.20%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.14
SafeMERGEBackbone=Qwen2.5-7B2026.0689.1
BEAMModel=Qwen3-30B-A3B, Beta (β)=0.01, Avg. K=4.232026.0588.07
Fine-tunedBackbone=Qwen2.5-7B2026.0687.98
SafeGeneBackbone=Qwen2.5-7B2026.0687.86
SafeLoRABackbone=Qwen2.5-7B2026.0686.97
SaLoRABackbone=Qwen2.5-7B2026.0686.9
SLEBModel=Llama3-8B, Ratio=31.20%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0586.76
Qwen3-30B-A3BModel=Qwen3-30B-A3B, Target Top-K (K)=8, Avg. K=8.002026.0586.76
MKAModel=Llama3-8B, Ratio=21.90%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0586.15
Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct, Sparsity=Dense2026.0685.81
FP16*#Bits=16-16, Evaluation Protocol=original paper result2026.0585.23
FP16#Bits=16-162026.0585.05
GEMQ#Bits=3-162026.0585.02
DenseBackbone=LLaMA-1, Model Parameters=65B, Sparsity=0%, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1184.83
Qwen3-4B-InstructModel=Qwen3-4B-Instruct, Sparsity=Dense2026.0684.83
AutoPruneBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1184.79
WandaBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1184.7
SparseGPTBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1184.6
SUBFITModel=Llama-3.1-8B-Instruct, Sparsity=25%2026.0684.43
DeepSeek-7B-chatModel=DeepSeek-7B-chat, Sparsity=Dense2026.0683.98
Llama-3.1-8B-InstructModel=Llama-3.1-8B-Instruct, Sparsity=Dense2026.0683.88
AutoPruneParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot, Quantization=8-bit2025.1183.6
SparseGPTParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1183.55
DenseParams=LLaMA-2 70B, Sparsity=0%, Evaluation protocol=zero-shot2025.1183.4
FP16Backbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0683.36
SUBFITModel=DeepSeek-7B-chat, Sparsity=25%2026.0683.09
FP16Backbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0683.03
MoEQuant*#Bits=3-16, Evaluation Protocol=original paper result2026.0582.81
DenseBackbone=LLaMA-1, Model Parameters=30B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1182.69
WandaParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1182.5
SparseGPTBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1182.32
AutoPruneBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1182.11
WandaBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1181.9
WandaParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1181.84
SparseGPTParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1181.44
DenseModel=LLaMA-2-7B, Sparsity Pattern=Dense (None), Evaluation Protocol=Zero-shot2026.0680.61
AutoPruneParams=LLaMA-2 13B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1180.58
BF16Backbone=Llama-2-13B, Quantization=None2026.0580.55
DenseParams=LLaMA-2 13B, Sparsity=0%, Evaluation protocol=zero-shot2025.1180.52
CRePEModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.0680.31
SAGE-PTQBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0680.28
GEMQ#Bits=2.5-162026.0580.09
LoRA-MixerBackbone=LLaMA3-8B, LoRA Rank (r)=322025.0679.37
MagnitudeBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1179.15
BEAMModel=Qwen1.5-MoE-A2.7B, Beta (β)=0.01, Avg. K=1.562026.0578.32
DenseBackbone=LLaMA-1, Model Parameters=13B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1177.89
CRePEModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.0677.8
DenseParams=LLaMA-2 7B, Sparsity=0%, Evaluation protocol=zero-shot2025.1177.74
BF16Backbone=LLaMA-2-7B, Quantization=None2026.0577.71
SUBFITModel=Llama-3.2-3B-Instruct, Sparsity=25%2026.0677.13
SparseGPTBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1176.97
AutoPruneBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1176.94
BEAMModel=DeepSeekV2-Lite, Beta (β)=0.01, Avg. K=2.612026.0576.57
WandaParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1175.99
WandaBackbone=LLaMA-1, Model Parameters=13B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1175.9
Llama-3.2-3B-InstructModel=Llama-3.2-3B-Instruct, Sparsity=Dense2026.0675.63
AutoPruneParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1175.5
DeepSeekV2-LiteModel=DeepSeekV2-Lite, Target Top-K (K)=6, Avg. K=6.002026.0575.2
DenseBackbone=LLaMA-1, Model Parameters=7B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1175.05
SparseGPTParams=LLaMA-2 7B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1175.02
LoRABackbone=LLaMA3-8B, LoRA Rank (r)=322025.0674.46
SAGE-PTQBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0674.43
LLaMA3Size=3.0B, Bit=16.0, Zero-shot=true2025.0873.4
Qwen1.5-MoEModel=Qwen1.5-MoE-A2.7B, Target Top-K (K)=4, Avg. K=4.002026.0572.63
AutoPruneBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1172.29
SparseGPTBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1172.05
BaseBackbone=LLaMA3-8B2025.0671.25
WandaBackbone=LLaMA-1, Model Parameters=7B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1171.22
MagnitudeParams=LLaMA-2 70B, Sparsity=50% unstructured, Evaluation protocol=zero-shot2025.1170.55
FO (Adam)Base Model=LLaMA-3.2-1B, Optimization Regime=First-Order2025.1069
FP16Backbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.0668.66
SUBFITModel=Qwen2.5-7B-Instruct, Sparsity=25%2026.0668.1
LGPBackbone=Llama-2-13B, Quantization=W2A16, alpha_1=1e52026.0567.09
OmniQuantBackbone=Llama-2-13B, Quantization=W2A162026.0566.69
Entropy-based + SPSequence Reduction=6.3†, FLOPs/byte Reduction=2.72026.0566.3
ZO Fine-tunerBase Model=LLaMA-3.2-1B, Optimization Regime=Zeroth-Order2025.1066
BiLLMBackbone=LLaMA-3-8B, Evaluation Protocol=zero-shot2026.0665.93
H-Net + SPSequence Reduction=5.3†, FLOPs/byte Reduction=2.42026.0565.9
Fixed (p = 4) + SPSequence Reduction=4.0, FLOPs/byte Reduction=2.12026.0565.2
TransformerSize=1.7B, Optimizer=Muon, RoPE on ρ=—2026.0564.92
SAGE-PTQBackbone=OPT-6.7B, Evaluation Protocol=zero-shot2026.0664.71
PythiaSize=2.8B, Bit=16.0, Zero-shot=true2025.0864.7
ParallaxSize=1.7B, Optimizer=Muon, RoPE on ρ=✓2026.0564.59
Byte-levelSequence Reduction=1.0, FLOPs/byte Reduction=1.02026.0564.5
H-NetSequence Reduction=4.8, FLOPs/byte Reduction=3.42026.0564.5
MagnitudeBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1164.34
Transformerevaluation_mode=zero-shot, parameters=2.8B, context_length=< 2K tokens2025.1164.28
SpaceByte + SPSequence Reduction=6.3, FLOPs/byte Reduction=2.72026.0564.2
CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0664.04
LLaMA3Size=1.3B, Bit=16.0, Zero-shot=true2025.0864
BiLLMBackbone=Qwen-3-8B, Evaluation Protocol=zero-shot2026.0663.9
BF16Backbone=Qwen-3-0.6B, Quantization=None2026.0563.82
Qwen3-0.6B (Pretrained)2026.0563.8