List your company
ResearchFormerly WizWand

Common Sense Reasoning on HellaSwag (Accuracy)

91.84AccuracyMistral-Small
22.856840.765958.67576.5841May 20, 2025Jul 28, 2025Oct 5, 2025Dec 13, 2025Feb 20, 2026Apr 30, 2026Jul 8, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Mistral-SmallModel Size=24B, GPU Hour=∼1.6M2025.0591.84
Qwen2.5-InstructModel Size=14B, GPU Hour=∼1.8M2025.0588.28
MiniLogitModel Size=14B, GPU Hour=2202025.0588.25
InfiGFusionModel Size=14B, GPU Hour=1952025.0588.22
InfiFusionModel Size=14B, GPU Hour=1602025.0587.91
FuseLLMModel Size=14B, GPU Hour=2252025.0587.81
Pivot-SFTModel Size=14B, GPU Hour=1202025.0587.76
Phi-4Model Size=14B, GPU Hour=∼1.0M2025.0587.62
FuseChatModel Size=14B, GPU Hour=6502025.0587.42
Qwen2.5-CoderModel Size=14B, GPU Hour=∼1.8M2025.0579.83
LoLaTok. (B)=0.04, Cache Size=1282026.0779.8
LLaMA 3.1 8BCache Size=∞2026.0779.34
OURS (GDN)Tok. (B)=0.01, Cache Size=128, Param.=10.3M2026.0779.34
LizardTok. (B)=0.04, Cache Size=1322026.0779.3
LolCaTTok. (B)=0.04, Cache Size=642026.0779.07
STILL (concurrent)Tok. (B)=0.04, Cache Size=NA2026.0779
FP16BIT=16, Evaluation Protocol=Few-shot2026.0778.84
OURS (GDN)Tok. (B)=0.01, Cache Size=64, Param.=10.3M2026.0778.79
LlambaTok. (B)=12, Cache Size=0, Param.=8B2026.0777.6
GSRQBIT=2, Evaluation Protocol=Few-shot2026.0776.38
Liger-GLATok. (B)=0.02, Cache Size=642026.0775.4
VQLLMBIT=2, Evaluation Protocol=Few-shot2026.0775.23
GSRQBIT=1, Evaluation Protocol=Few-shot2026.0769.56
VanillaBackbone=Mixtral-8x7B-v0.1, Sparsity=0%2025.1164.9
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1164.2
GSRQBIT=0.75, Evaluation Protocol=Few-shot2026.0763.74
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1163.3
Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1162
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1161.9
HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1161.3
D2Backbone=Mixtral-8x7B-v0.1, Sparsity=20%2025.1161
STUNBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1160.8
VanillaBackbone=Qwen3-MoE-30B-A3B, Sparsity=0%2025.1159.5
WandaBackbone=Mixtral-8x7B-v0.1, Sparsity=2:42025.1158.7
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1158.3
VanillaBackbone=Deepseek-MoE-16b, Sparsity=0%2025.1158.1
VanillaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=0%2025.1158
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1157.8
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.1157.3
PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1157.2
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1157.1
D2Backbone=Mixtral-8x7B-v0.1, Sparsity=40%2025.1157
Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1157
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.1156.5
PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.1156.3
HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1155.5
D2Backbone=Deepseek-MoE-16b, Sparsity=20%2025.1155
Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1154
WandaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=2:42025.1152.6
VQLLMBIT=1, Evaluation Protocol=Few-shot2026.0751.65
WandaBackbone=Deepseek-MoE-16b, Sparsity=2:42025.1151.4
WandaBackbone=Qwen3-MoE-30B-A3B, Sparsity=2:42025.1151.1
D2Backbone=Qwen1.5-MoE-A2.7B, Sparsity=20%2025.1151
HC-SMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.1150.4
HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1148.1
Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1147
PrismScale=Large, Evaluation Protocol=Zero-shot2026.0646.5
BaselineScale=Large, Evaluation Protocol=Zero-shot2026.0646.3
D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=20%2025.1145.1
D2Backbone=Deepseek-MoE-16b, Sparsity=40%2025.1145
MOE+Lngram2026.0544.81
MOE+Lngram-23Lconfiguration=23L2026.0544.75
HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1144.6
HC-SMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1144.3
MOE+Engram2026.0544.18
Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.1144
MOE2026.0543.94
D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=40%2025.1143.1
D2Backbone=Qwen1.5-MoE-A2.7B, Sparsity=40%2025.1142.7
Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1141
PrismScale=Medium, Evaluation Protocol=Zero-shot2026.0639
BaselineScale=Medium, Evaluation Protocol=Zero-shot2026.0638.7
HC-SMoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.1131.2
HC-SMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.1131.2
PrismScale=Small, Evaluation Protocol=Zero-shot2026.0630
BaselineScale=Small, Evaluation Protocol=Zero-shot2026.0629.6
BF16Size=1.0B2026.0527.13
BF16Size=1.7B2026.0527.07
BF16Size=0.4B2026.0526.85
+LGRSize=0.4B2026.0526.22
+LGRSize=1.7B2026.0526.17
B158Size=1.7B2026.0526.07
+LGRSize=1.0B2026.0525.75
B158Size=1.0B2026.0525.7
B158Size=0.4B2026.0525.51