List your company
ResearchFormerly WizWand

Zero-shot Language Understanding on ARC-c, ARC-e, PIQA, Winogrande, and Hellaswag

6,257Mean AccuracyRIA+SQ+VC+EBFT
-195.01441,480.02783,155.074,830.1122Jul 3, 2025Aug 29, 2025Oct 25, 2025Dec 21, 2025Feb 16, 2026Apr 14, 2026Jun 11, 2026
Updated 4mo ago

Evaluation results

MethodLinks
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=8:162025.076,257
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=8:162025.076,215
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=8:162025.076,191
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=8:162025.076,181
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=8:162025.076,127
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=8:162025.076,118
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=8:162025.076,112
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=8:162025.076,108
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=8:162025.076,104
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=8:162025.076,101
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=8:162025.076,095
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=8:162025.076,056
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=2:42025.076,020
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=2:42025.075,974
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=16:256, Sparsity=2:42025.075,941
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=16:256, Sparsity=2:42025.075,916
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=2:42025.075,872
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=2:42025.075,853
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=2:42025.075,787
RIA+SQ+VC+EBFTModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=2:42025.075,784
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=8:256, Sparsity=2:42025.075,748
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=8:256, Sparsity=2:42025.075,740
RIA+SQModel=LLaMA2-7B, Calibration Set=WikiText2, Outliers=4:256, Sparsity=2:42025.075,644
RIA+SQModel=LLaMA2-7B, Calibration Set=C4, Outliers=4:256, Sparsity=2:42025.075,597
AQLM (Ind.)Model=Qwen 14B, Codebooks=5 × 82026.0676.85
AQLM (Drop)Model=Qwen 14B, Codebooks=5 × 82026.0676.85
DbyD (35W)Model=Qwen 14B, Codebooks=5 × 82026.0676.76
DbyD (Uni.)Model=Qwen 14B, Codebooks=5 × 82026.0676.03
DbyD (35W)Model=Qwen 14B, Codebooks=4 × 82026.0675.99
AQLM (Ind.)Model=Qwen 14B, Codebooks=4 × 82026.0675.94
DbyD (Uni.)Model=Qwen 14B, Codebooks=4 × 82026.0675.8
AQLM (Ind.)Model=Qwen 32B, Codebooks=3 × 82026.0674.94
AQLM (Ind.)Model=Qwen 14B, Codebooks=3 × 82026.0674.89
DbyD (35W)Model=Qwen 32B, Codebooks=5 × 82026.0674.74
AQLM (Drop)Model=Qwen 14B, Codebooks=4 × 82026.0674.73
AQLM (Ind.)Model=Qwen 32B, Codebooks=5 × 82026.0674.53
AQLM (Drop)Model=Qwen 32B, Codebooks=5 × 82026.0674.53
AQLM (Ind.)Model=Qwen 32B, Codebooks=4 × 82026.0674.5
DbyD (Uni.)Model=Qwen 32B, Codebooks=5 × 82026.0674.2
DbyD (35W)Model=Qwen 32B, Codebooks=4 × 82026.0673.98
DbyD (35W)Model=Qwen 14B, Codebooks=3 × 82026.0673.93
DbyD (Uni.)Model=Qwen 14B, Codebooks=3 × 82026.0673.92
DbyD (35W)Model=Qwen 7B, Codebooks=5 × 82026.0673.72
AQLM (Ind.)Model=Qwen 7B, Codebooks=5 × 82026.0673.59
AQLM (Drop)Model=Qwen 7B, Codebooks=5 × 82026.0673.59
AQLM (Drop)Model=Qwen 32B, Codebooks=4 × 82026.0673.1
DbyD (35W)Model=Qwen 32B, Codebooks=3 × 82026.0672.94
DbyD (35W)Model=Mistral 7B, Codebooks=5 × 82026.0672.85
DbyD (Uni.)Model=Qwen 7B, Codebooks=5 × 82026.0672.75
DbyD (Uni.)Model=Qwen 32B, Codebooks=4 × 82026.0672.56
DbyD (35W)Model=Qwen 7B, Codebooks=4 × 82026.0672.52
AQLM (Ind.)Model=Mistral 7B, Codebooks=4 × 82026.0672.46
AQLM (Ind.)Model=MetaLlama 8B, Codebooks=5 × 82026.0672.42
AQLM (Drop)Model=MetaLlama 8B, Codebooks=5 × 82026.0672.42
AQLM (Ind.)Model=Qwen 7B, Codebooks=4 × 82026.0672.37
DbyD (Uni.)Model=Mistral 7B, Codebooks=5 × 82026.0672.15
DbyD (35W)Model=MetaLlama 8B, Codebooks=5 × 82026.0672.09
DbyD (Uni.)Model=Qwen 32B, Codebooks=3 × 82026.0671.93
AQLM (Ind.)Model=Mistral 7B, Codebooks=5 × 82026.0671.72
AQLM (Drop)Model=Mistral 7B, Codebooks=5 × 82026.0671.72
DbyD (Uni.)Model=MetaLlama 8B, Codebooks=5 × 82026.0671.62
AQLM (Drop)Model=Qwen 7B, Codebooks=4 × 82026.0671.4
DbyD (Uni.)Model=Qwen 7B, Codebooks=4 × 82026.0671.32
AQLM (Drop)Model=Qwen 32B, Codebooks=3 × 82026.0671.08
AQLM (Ind.)Model=MetaLlama 8B, Codebooks=4 × 82026.0671
AQLM (Ind.)Model=Mistral 7B, Codebooks=3 × 82026.0670.97
AQLM (Ind.)Model=Qwen 7B, Codebooks=3 × 82026.0670.83
DbyD (35W)Model=Mistral 7B, Codebooks=4 × 82026.0670.81
AQLM (Ind.)Model=MetaLlama 8B, Codebooks=3 × 82026.0670
AQLM (Drop)Model=Qwen 14B, Codebooks=3 × 82026.0670
DbyD (Uni.)Model=Mistral 7B, Codebooks=4 × 82026.0669.96
AQLM (Drop)Model=Mistral 7B, Codebooks=4 × 82026.0669.94
DbyD (35W)Model=MetaLlama 8B, Codebooks=4 × 82026.0669.6
DbyD (35W)Model=Qwen 7B, Codebooks=3 × 82026.0669.54
DbyD (Uni.)Model=MetaLlama 8B, Codebooks=4 × 82026.0668.52
DbyD (Uni.)Model=Qwen 3B, Codebooks=5 × 82026.0668.18
DbyD (35W)Model=Qwen 3B, Codebooks=5 × 82026.0668.17
AQLM (Ind.)Model=Qwen 3B, Codebooks=5 × 82026.0667.83
AQLM (Drop)Model=Qwen 3B, Codebooks=5 × 82026.0667.83
DbyD (35W)Model=Mistral 7B, Codebooks=3 × 82026.0667.48
DbyD (Uni.)Model=Qwen 7B, Codebooks=3 × 82026.0667.3
DbyD (35W)Model=Qwen 3B, Codebooks=4 × 82026.0667.28
AQLM (Drop)Model=Qwen 7B, Codebooks=3 × 82026.0666.77
AQLM (Ind.)Model=Qwen 3B, Codebooks=4 × 82026.0666.68
DbyD (Uni.)Model=Qwen 3B, Codebooks=4 × 82026.0665.85
AQLM (Drop)Model=MetaLlama 8B, Codebooks=4 × 82026.0665.54
AQLM (Ind.)Model=Qwen 3B, Codebooks=3 × 82026.0665.48
DbyD (35W)Model=MetaLlama 8B, Codebooks=3 × 82026.0664.81
Dense ModelModel=LLaMA2-7B2025.0764.79
DbyD (35W)Model=Qwen 3B, Codebooks=3 × 82026.0663.06
DbyD (Uni.)Model=Qwen 3B, Codebooks=3 × 82026.0662.53
AQLM (Drop)Model=Qwen 3B, Codebooks=4 × 82026.0662.11
DbyD (Uni.)Model=Mistral 7B, Codebooks=3 × 82026.0661.78
DbyD (Uni.)Model=MetaLlama 8B, Codebooks=3 × 82026.0655.8
DbyD (35W)Model=Qwen 0.5B, Codebooks=5 × 82026.0653.86
DbyD (Uni.)Model=Qwen 0.5B, Codebooks=5 × 82026.0653.62
AQLM (Ind.)Model=Qwen 0.5B, Codebooks=4 × 82026.0653.6
AQLM (Ind.)Model=Qwen 0.5B, Codebooks=5 × 82026.0653.25
AQLM (Drop)Model=Qwen 0.5B, Codebooks=5 × 82026.0653.25
DbyD (35W)Model=Qwen 0.5B, Codebooks=4 × 82026.0653.14