Commonsense Reasoning on PIQA (Accuracy)
89.99AccuracyBaLoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.04 | 89.99 | |
| HiRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 89.7 | |
| DoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 89.3 | |
| IT-PrunModel=Llama3-8B, Ratio=21.90%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.05 | 89.23 | |
| IT-PrunModel=Llama3-8B, Ratio=12.50%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.05 | 88.9 | |
| IT-PrunModel=Llama3-8B, Ratio=31.20%, Zero-shot=true, Supervised Fine-Tuning (SFT)=true2026.05 | 88.03 | |
| MoRAModel=Llama-3-8B, Params (%)=0.69972026.04 | 87.43 | |
| FFTModel=Llama-3.2-3B, #Params=3.21B2025.09 | 87.25 | |
| ABBAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 87.2 | |
| BoHAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 86.69 | |
| HiRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 86.57 | |
| GraLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 86.53 | |
| DoRAModel=Llama-3.2-3B, #Params=49.40M2025.09 | 85.47 | |
| Kimi-K2 Baseshots=0-shot2026.06 | 85.47 | |
| ChatGPTModel=ChatGPT2026.04 | 85.4 | |
| LoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 85.2 | |
| LoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 85.2 | |
| DeepSeek-V3.2 Exp-Baseshots=0-shot2026.06 | 85.09 | |
| GLM-4.5 Baseshots=0-shot2026.06 | 85.09 | |
| rsLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 85.05 | |
| LoRA-MixerBackbone=LLaMA3-8B, LoRA Rank (r)=322025.06 | 84.94 | |
| BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.04 | 84.93 | |
| Mistral-Large-3 675B-Base-2512shots=0-shot2026.06 | 84.82 | |
| PiSSAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 84.71 | |
| Qwen3-8B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 84.44 | |
| Qwen3-8B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 84.11 | |
| Qwen3-8B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 84.06 | |
| Nemotron-3-Ultra 550B-A55B-Baseshots=0-shot2026.06 | 83.79 | |
| DoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 83.7 | |
| HiRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 83.35 | |
| Qwen3-235B-A22B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 82.92 | |
| Llama2-70B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 82.81 | |
| VanillaBackbone=Mixtral-8x7B-v0.1, Sparsity=0%2025.11 | 82.4 | |
| PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 82.1 | |
| LizardTok. (B)=0.04, Cache Size=1322026.07 | 82 | |
| Qwen3-32B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 81.88 | |
| RF2-100B-A6.1B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 81.83 | |
| PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 81.7 | |
| LoLA-8B (ours)Architecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot, η=64, λ=642025.05 | 81.6 | |
| LoLaTok. (B)=0.04, Cache Size=1282026.07 | 81.6 | |
| STILL (concurrent)Tok. (B)=0.04, Cache Size=NA2026.07 | 81.3 | |
| OriginalBase Model=LLaMA3-8B2026.05 | 81.28 | |
| DreamReasoner-8BType=Block Diffusion2026.06 | 81.2 | |
| Llama-3.1-8BArchitecture=Transformer, Tokens (B)=15000, Evaluation Protocol=zero-shot2025.05 | 81.1 | |
| Griffin 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.05 | 81 | |
| LoLCATs-8BArchitecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot2025.05 | 81 | |
| D2Backbone=Mixtral-8x7B-v0.1, Sparsity=20%2025.11 | 81 | |
| Qwen3-4B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 80.96 | |
| LolCaTTok. (B)=0.04, Cache Size=642026.07 | 80.96 | |
| LLaMA8BModel=LLaMA-3.1, Router=None2025.10 | 80.9 | |
| Llamba-8BArchitecture=Subquadratic (Distilled), Tokens (B)=12, Evaluation Protocol=zero-shot2025.05 | 80.9 | |
| NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 80.9 | |
| LlambaTok. (B)=12, Cache Size=0, Param.=8B2026.07 | 80.9 | |
| MoRAModel=Llama-2-7B, Params (%)=0.82412026.04 | 80.79 | |
| LoRABackbone=LLaMA3-8B, LoRA Rank (r)=322025.06 | 80.71 | |
| HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 80.7 | |
| Qwen3-4B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 80.69 | |
| FFTModel=Llama-3.2-1B, #Params=1.24B2025.09 | 80.62 | |
| RecurrentGemma-9BArchitecture=Subquadratic (Pretrained), Tokens (B)=2000, Evaluation Protocol=zero-shot2025.05 | 80.6 | |
| Llama 3-8BParadigm=AR, Training Tokens=15T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.06 | 80.6 | |
| BoHAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 80.59 | |
| OLMoE-1B-7BActive Parameters=1.3B, Total Parameters=6.9B, Few-shot count=0-shot2026.05 | 80.5 | |
| Falcon-7BParadigm=AR, Training Tokens=1.5T, Training Data=Partially Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 80.5 | |
| Liger-GLATok. (B)=0.02, Cache Size=642026.07 | 80.5 | |
| CAT-QBase Model=Llama2-70B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 80.34 | |
| Qwen3-4B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 80.3 | |
| Qwen3-30B-A3B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 80.14 | |
| ABBAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 80 | |
| MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=0-shot2026.05 | 80 | |
| Hawk 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.05 | 80 | |
| VanillaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=0%2025.11 | 80 | |
| LoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 79.9 | |
| MiMo-7BType=AR2026.06 | 79.9 | |
| GraLoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 79.87 | |
| CleanTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 79.8 | |
| CleanTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 79.8 | |
| Mamba2-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=3500, Evaluation Protocol=zero-shot2025.05 | 79.8 | |
| OLMo-7BParadigm=AR, Training Tokens=2.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 79.8 | |
| Qwen2.5-7BType=AR2026.06 | 79.8 | |
| Qwen7BModel=Qwen2.5, Router=None2025.10 | 79.7 | |
| Falcon3-Mamba-7BArchitecture=Subquadratic (Pretrained), Tokens (B)=7300, Evaluation Protocol=zero-shot2025.05 | 79.7 | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.11 | 79.7 | |
| Qwen3-14B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 79.65 | |
| VanillaBackbone=Qwen3-MoE-30B-A3B, Sparsity=0%2025.11 | 79.6 | |
| WandaBackbone=Mixtral-8x7B-v0.1, Sparsity=2:42025.11 | 79.5 | |
| Qwen3-8BType=AR2026.06 | 79.4 | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.11 | 79.4 | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.11 | 79.3 | |
| HiRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 79.22 | |
| Dr.LLMModel=LLaMA-3.1, Router=Enabled2025.10 | 79.2 | |
| CAT-QBase Model=Qwen3-235B-A22B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 79.16 | |
| OURS (GDN)Tok. (B)=0.01, Cache Size=128, Param.=10.3M2026.07 | 79.16 | |
| NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 79.1 | |
| OURS (GDN)Tok. (B)=0.01, Cache Size=64, Param.=10.3M2026.07 | 79.07 | |
| PiSSAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 79.05 | |
| LLaMA 3.1 8BCache Size=∞2026.07 | 79.05 | |
| MobileMoE-LMem (GB)=2.75, INT4 QAT=true, Zero-shot=true2026.05 | 79 | |
| Dr.LLMModel=Qwen2.5, Router=Enabled2025.10 | 78.9 | |
| Mamba-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=1100, Evaluation Protocol=zero-shot2025.05 | 78.9 | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.11 | 78.9 |