Commonsense Reasoning on PIQA (test)
90.1AccuracyUNICORN
Evaluation results
| Method | Links | |
|---|---|---|
| UNICORNsetting=Supervised2022.01 | 90.1 | |
| MT-NLGsetting=Few-shot2022.01 | 83.19 | |
| GPT-3setting=Few-shot2022.01 | 82.3 | |
| MT-NLGsetting=Zero-shot2022.01 | 81.99 | |
| Gophersetting=Zero-shot2022.01 | 81.8 | |
| LoTRBackbone=Llama-2-13b, Rank=64, Param × 10⁵=9.83, Protocol=Fine-tuning2025.06 | 81.5 | |
| LoRABackbone=Llama-2-13b, Rank=32, Param × 10⁵=262.1, Protocol=Fine-tuning2025.06 | 81.4 | |
| LoRABackbone=Llama-2-13b, Rank=8, Param × 10⁵=65.5, Protocol=Fine-tuning2025.06 | 81.3 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.7, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 81.23 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.75, Protocol=Fine-tuning2025.06 | 81.2 | |
| DenseModel=OPT-175B, Sparsity=0%, Evaluation Protocol=Zero-shot2023.01 | 81.07 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.5, Training Protocol=TF, Base Model=Qwen3-4B2026.05 | 81.03 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.7, Training Protocol=TF, Base Model=Qwen3-4B2026.05 | 81.01 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.5, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 81.01 | |
| GPT-3setting=Zero-shot2022.01 | 81 | |
| LoTRBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.84, Protocol=Fine-tuning2025.06 | 81 | |
| MT-NLGsetting=One-shot2022.01 | 80.96 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.1, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 80.96 | |
| VeRABackbone=Llama-2-13b, Rank=256, Param × 10⁵=4.3, Protocol=Fine-tuning2025.06 | 80.8 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=64, Param × 10⁵=8.3, Protocol=Fine-tuning2025.06 | 80.8 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.1, Training Protocol=TF, Base Model=Qwen3-4B2026.05 | 80.69 | |
| SparseGPTModel=OPT-175B, Sparsity=50%, Evaluation Protocol=Zero-shot2023.01 | 80.63 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.3, Training Protocol=TF, Base Model=Qwen3-4B2026.05 | 80.52 | |
| GPT-3setting=One-shot2022.01 | 80.5 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.3, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 80.47 | |
| Qwen3-4B + SFTMask Rate=-, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 80.3 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=64, Param × 10⁵=6.6, Protocol=Fine-tuning2025.06 | 80.3 | |
| Zero ShotBackbone=Llama-2-13b, Protocol=Zero-shot2025.06 | 80.3 | |
| LoRABackbone=Llama-2-7b, Rank=16, Param × 10⁵=83.9, Protocol=Fine-tuning2025.06 | 80.1 | |
| LoRABackbone=Llama-2-7b, Rank=8, Param × 10⁵=41.9, Protocol=Fine-tuning2025.06 | 80 | |
| LoTRBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.47, Protocol=Fine-tuning2025.06 | 79.9 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.40, Protocol=Fine-tuning2025.06 | 79.7 | |
| SparseGPTModel=OPT-175B, Sparsity=4:8, Evaluation Protocol=Zero-shot2023.01 | 79.54 | |
| SparseGPTModel=OPT-175B, Sparsity=2:4, Evaluation Protocol=Zero-shot2023.01 | 79.54 | |
| LoTRBackbone=Llama-2-7b, Rank=64, Param × 10⁵=7.86, Protocol=Fine-tuning2025.06 | 79.3 | |
| GPT-3 175Balpha=02021.10 | 79.27 | |
| Coherence Boosting (GPT-3 175B)alpha=-0.62021.10 | 78.94 | |
| VeRABackbone=Llama-2-7b, Rank=1024, Param × 10⁵=3.27, Protocol=Fine-tuning2025.06 | 78.9 | |
| Zero ShotBackbone=Llama-2-7b, Protocol=Zero-shot2025.06 | 78.8 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=1.0, Training Protocol=SFT, Base Model=Qwen3-4B2026.05 | 77.2 | |
| Pythia-12BParameters=12B, Zero-shot=true2023.04 | 76 | |
| Pythia-6.9BParameters=6.9B, Zero-shot=true2023.04 | 75.2 | |
| Pythia-2.8BParameters=2.8B, Zero-shot=true2023.04 | 73.9 | |
| Dip-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 73.4 | |
| SAES-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 73.4 | |
| SVD-LLMRatio=0.2, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 72.5 | |
| Coherence Boosting (GPT-2 XL)alpha=-0.432021.10 | 71.49 | |
| Pythia-1.4BParameters=1.4B, Zero-shot=true2023.04 | 71.1 | |
| Muon+CWDOptimizer=Muon+CWD, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 71 | |
| GPT-2 XL (1.6B)alpha=02021.10 | 70.84 | |
| Pythia-1BParameters=1B, Zero-shot=true2023.04 | 70.7 | |
| AdamW+CWDOptimizer=AdamW+CWD, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 69 | |
| MuonOptimizer=Muon, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 68 | |
| AdamWOptimizer=AdamW, Model Parameters=1B, Training Tokens=100B, Codebase=OLMo2025.10 | 67 | |
| Pythia-410MParameters=410M, Zero-shot=true2023.04 | 66.8 | |
| GPT-3 175Balpha=-12021.10 | 66.32 | |
| Dip-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 66.1 | |
| SVD-LLMRatio=0.4, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 63.7 | |
| Coherence Boosting (GPT-2 Small)alpha=-0.612021.10 | 63.44 | |
| SAES-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 63.1 | |
| GPT-2 Small (125M)alpha=02021.10 | 62.89 | |
| Pythia-160MParameters=160M, Zero-shot=true2023.04 | 62.7 | |
| GPT-2 XL (1.6B)alpha=-12021.10 | 60.45 | |
| PGM 6 / 6 (1024)distillation=After Distillation (SDTT)2025.05 | 59.85 | |
| Pythia-70MParameters=70M, Zero-shot=true2023.04 | 59.5 | |
| PGM 6 / 6 (1024)distillation=Before2025.05 | 59.19 | |
| PGM 8 / 8distillation=After Distillation (SDTT)2025.05 | 59.03 | |
| PGM 8 / 8distillation=Before2025.05 | 58.92 | |
| MDLMdistillation=Before2025.05 | 58.27 | |
| MDLMdistillation=After Distillation (SDTT)2025.05 | 57.73 | |
| GPT-2 Small (125M)alpha=-12021.10 | 57.45 | |
| MagnitudeModel=OPT-175B, Sparsity=50%, Evaluation Protocol=Zero-shot2023.01 | 54.73 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=1.0, Training Protocol=TF, Base Model=Qwen3-4B2026.05 | 49.44 |