List your company
ResearchFormerly WizWand

Commonsense Reasoning on PIQA (Accuracy)

89.99AccuracyBaLoRA
78.456481.450784.44587.4393May 29, 2025Aug 4, 2025Oct 11, 2025Dec 17, 2025Feb 23, 2026May 1, 2026Jul 8, 2026
Updated 3mo ago

Evaluation results

MethodLinks
BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.0489.99
HiRAModel=Llama-3-8B, Params (%)=0.70022026.0489.7
DoRAModel=Llama-3-8B, Params (%)=0.70022026.0489.3
IT-PrunModel=Llama3-8B, Ratio=21.90%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0589.23
IT-PrunModel=Llama3-8B, Ratio=12.50%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0588.9
IT-PrunModel=Llama3-8B, Ratio=31.20%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.0588.03
MoRAModel=Llama-3-8B, Params (%)=0.69972026.0487.43
FFTModel=Llama-3.2-3B, #Params=3.21B2025.0987.25
ABBAModel=Llama-3.2-3B, #Params=48.63M2025.0987.2
BoHAModel=Llama-3.2-3B, #Params=48.63M2025.0986.69
HiRAModel=Llama-3.2-3B, #Params=48.63M2025.0986.57
GraLoRAModel=Llama-3.2-3B, #Params=48.63M2025.0986.53
DoRAModel=Llama-3.2-3B, #Params=49.40M2025.0985.47
Kimi-K2 Baseshots=0-shot2026.0685.47
ChatGPTModel=ChatGPT2026.0485.4
LoRAModel=Llama-3.2-3B, #Params=48.63M2025.0985.2
LoRAModel=Llama-3-8B, Params (%)=0.70022026.0485.2
DeepSeek-V3.2 Exp-Baseshots=0-shot2026.0685.09
GLM-4.5 Baseshots=0-shot2026.0685.09
rsLoRAModel=Llama-3.2-3B, #Params=48.63M2025.0985.05
LoRA-MixerBackbone=LLaMA3-8B, LoRA Rank (r)=322025.0684.94
BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.0484.93
Mistral-Large-3 675B-Base-2512shots=0-shot2026.0684.82
PiSSAModel=Llama-3.2-3B, #Params=48.63M2025.0984.71
Qwen3-8B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0584.44
Qwen3-8B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0584.11
Qwen3-8B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0584.06
Nemotron-3-Ultra 550B-A55B-Baseshots=0-shot2026.0683.79
DoRAModel=Llama-2-7B, Params (%)=0.82562026.0483.7
HiRAModel=Llama-2-7B, Params (%)=0.82562026.0483.35
Qwen3-235B-A22B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0682.92
Llama2-70B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0682.81
VanillaBackbone=Mixtral-8x7B-v0.1, Sparsity=0%2025.1182.4
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1182.1
LizardTok. (B)=0.04, Cache Size=1322026.0782
Qwen3-32B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0681.88
RF2-100B-A6.1B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0681.83
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1181.7
LoLA-8B (ours)Architecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot, η=64, λ=642025.0581.6
LoLaTok. (B)=0.04, Cache Size=1282026.0781.6
STILL (concurrent)Tok. (B)=0.04, Cache Size=NA2026.0781.3
OriginalBase Model=LLaMA3-8B2026.0581.28
DreamReasoner-8BType=Block Diffusion2026.0681.2
Llama-3.1-8BArchitecture=Transformer, Tokens (B)=15000, Evaluation Protocol=zero-shot2025.0581.1
Griffin 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.0581
LoLCATs-8BArchitecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot2025.0581
D2Backbone=Mixtral-8x7B-v0.1, Sparsity=20%2025.1181
Qwen3-4B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0580.96
LolCaTTok. (B)=0.04, Cache Size=642026.0780.96
LLaMA8BModel=LLaMA-3.1, Router=None2025.1080.9
Llamba-8BArchitecture=Subquadratic (Distilled), Tokens (B)=12, Evaluation Protocol=zero-shot2025.0580.9
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1180.9
LlambaTok. (B)=12, Cache Size=0, Param.=8B2026.0780.9
MoRAModel=Llama-2-7B, Params (%)=0.82412026.0480.79
LoRABackbone=LLaMA3-8B, LoRA Rank (r)=322025.0680.71
HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1180.7
Qwen3-4B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0580.69
FFTModel=Llama-3.2-1B, #Params=1.24B2025.0980.62
RecurrentGemma-9BArchitecture=Subquadratic (Pretrained), Tokens (B)=2000, Evaluation Protocol=zero-shot2025.0580.6
Llama 3-8BParadigm=AR, Training Tokens=15T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.0680.6
BoHAModel=Llama-3.2-1B, #Params=22.54M2025.0980.59
OLMoE-1B-7BActive Parameters=1.3B, Total Parameters=6.9B, Few-shot count=0-shot2026.0580.5
Falcon-7BParadigm=AR, Training Tokens=1.5T, Training Data=Partially Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0680.5
Liger-GLATok. (B)=0.02, Cache Size=642026.0780.5
CAT-QBase Model=Llama2-70B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0680.34
Qwen3-4B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0580.3
Qwen3-30B-A3B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0680.14
ABBAModel=Llama-3.2-1B, #Params=22.54M2025.0980
MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=0-shot2026.0580
Hawk 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.0580
VanillaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=0%2025.1180
LoRAModel=Llama-2-7B, Params (%)=0.82562026.0479.9
MiMo-7BType=AR2026.0679.9
GraLoRAModel=Llama-3.2-1B, #Params=22.54M2025.0979.87
CleanTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0579.8
CleanTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0579.8
Mamba2-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=3500, Evaluation Protocol=zero-shot2025.0579.8
OLMo-7BParadigm=AR, Training Tokens=2.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0679.8
Qwen2.5-7BType=AR2026.0679.8
Qwen7BModel=Qwen2.5, Router=None2025.1079.7
Falcon3-Mamba-7BArchitecture=Subquadratic (Pretrained), Tokens (B)=7300, Evaluation Protocol=zero-shot2025.0579.7
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.1179.7
Qwen3-14B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0679.65
VanillaBackbone=Qwen3-MoE-30B-A3B, Sparsity=0%2025.1179.6
WandaBackbone=Mixtral-8x7B-v0.1, Sparsity=2:42025.1179.5
Qwen3-8BType=AR2026.0679.4
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.1179.4
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1179.3
HiRAModel=Llama-3.2-1B, #Params=22.54M2025.0979.22
Dr.LLMModel=LLaMA-3.1, Router=Enabled2025.1079.2
CAT-QBase Model=Qwen3-235B-A22B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0679.16
OURS (GDN)Tok. (B)=0.01, Cache Size=128, Param.=10.3M2026.0779.16
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1179.1
OURS (GDN)Tok. (B)=0.01, Cache Size=64, Param.=10.3M2026.0779.07
PiSSAModel=Llama-3.2-1B, #Params=22.54M2025.0979.05
LLaMA 3.1 8BCache Size=∞2026.0779.05
MobileMoE-LMem (GB)=2.75, INT4 QAT=true, Zero-shot=true2026.0579
Dr.LLMModel=Qwen2.5, Router=Enabled2025.1078.9
Mamba-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=1100, Evaluation Protocol=zero-shot2025.0578.9
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1178.9