Commonsense Reasoning on HellaSwag (val)
95.3AccuracyGPT-4
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4number_of_shots=10-shot2023.03 | 95.3 | — | — | |
| D2-LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 91 | — | — | |
| LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 90.2 | — | — | |
| DoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 90 | — | — | |
| ALUMbenchmark-specific tuning=true2023.03 | 85.6 | — | — | |
| GPT-3.5number_of_shots=10-shot2023.03 | 85.5 | — | — | |
| LLaMAsplit=validation set2023.03 | 84.2 | — | — | |
| D2-LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 82.4 | — | — | |
| DoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 81.8 | — | — | |
| LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 77.6 | — | — | |
| Qwen3-4B2026.06 | 68.4 | — | — | |
| EvoGMBackbone=Qwen2.5-1.5B2026.05 | 66 | — | — | |
| PSO-MergingBackbone=Qwen2.5-1.5B2026.05 | 65 | — | — | |
| CMABase model=Qwen2.5-1.5B2026.05 | 64.5 | — | — | |
| EvoGMBase model=Qwen2.5-1.5B2026.05 | 63.5 | — | — | |
| Model SwarmBackbone=Qwen2.5-1.5B2026.05 | 63.5 | — | — | |
| Single BestBase model=Qwen2.5-1.5B2026.05 | 62 | — | — | |
| Single BestBackbone=Qwen2.5-1.5B2026.05 | 62 | — | — | |
| DAREBase model=Qwen2.5-1.5B2026.05 | 61.5 | — | — | |
| DAREBackbone=Qwen2.5-1.5B2026.05 | 61.5 | — | — | |
| Task ArithmeticBase model=Qwen2.5-1.5B2026.05 | 60.5 | — | — | |
| TABackbone=Qwen2.5-1.5B2026.05 | 60.5 | — | — | |
| CMABackbone=Qwen2.5-1.5B2026.05 | 60 | — | — | |
| BaseBase model=Qwen2.5-1.5B2026.05 | 59 | — | — | |
| Model SoupBase model=Qwen2.5-1.5B2026.05 | 59 | — | — | |
| BaseBackbone=Qwen2.5-1.5B2026.05 | 59 | — | — | |
| Model SoupBackbone=Qwen2.5-1.5B2026.05 | 59 | — | — | |
| Model SwarmBase model=Qwen2.5-1.5B2026.05 | 58.5 | — | — | |
| FedAvgBackbone=SmolLM-360M, Federated Partitioning=10-client non-IID, Evaluation Protocol=3-seed mean2026.06 | 58.1 | 1.1078 | 97.77 | |
| Bank of Values (BoV)Output=Ev[i], Layers=last 1/3, Coeff.=γv, Model Scale=780M, Zero-shot=true2026.06 | 57.4 | — | — | |
| V1 variant (last 1/3)Output=V1, Layers=last 1/3, Coeff.=γv, Model Scale=780M, Zero-shot=true2026.06 | 56.6 | — | — | |
| Subspace-RegBackbone=SmolLM-360M, Federated Partitioning=10-client non-IID, Evaluation Protocol=3-seed mean2026.06 | 56.5 | 1.1254 | 99.989 | |
| x0WV variantOutput=x0WV, Layers=last 1/3, Coeff.=γv, Model Scale=780M, Zero-shot=true2026.06 | 56.2 | — | — | |
| Standard AttentionOutput=V, Model Scale=780M, Zero-shot=true2026.06 | 55.8 | — | — | |
| DeepSeekMoE# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.3B, FLOPs per 2K Tokens=4.3T, # Training Tokens=100B2024.01 | 54.8 | — | — | |
| SVDBackbone=SmolLM-360M, Federated Partitioning=10-client non-IID, Evaluation Protocol=3-seed mean2026.06 | 53.6 | 1.1708 | 99.06 | |
| PSO-MergingBase model=Qwen2.5-1.5B2026.05 | 53 | — | — | |
| MTLBase model=Qwen2.5-1.5B2026.05 | 52.5 | — | — | |
| MTLBackbone=Qwen2.5-1.5B2026.05 | 52.5 | — | — | |
| FedSVDBackbone=SmolLM-360M, Federated Partitioning=10-client non-IID, Evaluation Protocol=3-seed mean2026.06 | 52.1 | 1.1848 | 96.8 | |
| V1 variant (every layer)Output=V1, Layers=every, Coeff.=1, Model Scale=780M, Zero-shot=true2026.06 | 51.6 | — | — | |
| TIESBase model=Qwen2.5-1.5B2026.05 | 51 | — | — | |
| TIESBackbone=Qwen2.5-1.5B2026.05 | 51 | — | — | |
| GPT-2 XL2026.06 | 50.9 | — | — | |
| Mistral (Full-Attention)Model Scale=1.4B2024.07 | 50.7 | — | — | |
| GShard# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.3B, FLOPs per 2K Tokens=4.3T, # Training Tokens=100B2024.01 | 50.5 | — | — | |
| Talkie-1930variant=base2026.06 | 49.8 | — | — | |
| Switch# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.2B, FLOPs per 2K Tokens=2.9T, # Training Tokens=100B2024.01 | 49.1 | — | — | |
| BMoJo (Fading + Eidetic)Model Scale=1.4B2024.07 | 46.8 | — | — | |
| Hash Layer# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.2B, FLOPs per 2K Tokens=2.9T, # Training Tokens=100B2024.01 | 46.2 | — | — | |
| BMoJo (Fading)Model Scale=1.4B2024.07 | 46 | — | — | |
| Mamba (SSM)Model Scale=1.4B2024.07 | 45 | — | — | |
| Dense# Shot=0-shot, # Total Params=0.2B, # Activated Params=0.2B, FLOPs per 2K Tokens=2.9T, # Training Tokens=100B2024.01 | 38.8 | — | — | |
| Hybrid (Sliding Attention + SSM)Model Scale=1.4B2024.07 | 38.8 | — | — | |
| AMOOptimizer=AMO, Model=Llama3.1-1.4B, Few-shot=02026.05 | 37.46 | — | — | |
| TYPEWRITERLMvariant=base2026.06 | 35.9 | — | — | |
| GPT-1900variant=base2026.06 | 34.8 | — | — | |
| Mistral (Full-Attention)Model Scale=370M2024.07 | 33.8 | — | — | |
| AMOOptimizer=AMO, Model=Llama3.1-760M, Few-shot=02026.05 | 33.79 | — | — | |
| Mamba (SSM)Model Scale=370M2024.07 | 33.4 | — | — | |
| BMoJo (Fading + Eidetic)Model Scale=370M2024.07 | 33.3 | — | — | |
| BMoJo (Fading)Model Scale=370M2024.07 | 33.2 | — | — | |
| TMMFormerBackbone=12-layer Transformer (d=768)2026.05 | 31.82 | — | — | |
| YuriiFormerBackbone=12-layer Transformer (d=768)2026.05 | 31.58 | — | — | |
| Hybrid (Sliding Attention + SSM)Model Scale=370M2024.07 | 31.3 | — | — | |
| AdamFormerBackbone=12-layer Transformer (d=768)2026.05 | 30.96 | — | — | |
| VanillaTransformerBackbone=12-layer Transformer (d=768)2026.05 | 30.2 | — | — | |
| AdamWFormerBackbone=12-layer Transformer (d=768)2026.05 | 30.08 | — | — |