Commonsense Reasoning on HellaSwag (Score)
95.45HellaSwag ScoreLD-MoLE
Evaluation Results
| Method | Links | |
|---|---|---|
| LD-MoLEModel=Llama3.1-8B, TP=2.33 %2025.09 | 95.45 | |
| ReMoLEModel=Llama3.1-8B, TP=2.17 %2025.09 | 94.95 | |
| MoLA(8888)Model=Llama3.1-8B, TP=2.17 %2025.09 | 94.06 | |
| MoLA(2468)Model=Llama3.1-8B, TP=1.26 %2025.09 | 94.03 | |
| LD-MoLEModel=Llama3.2-3B, TP=3.28 %2025.09 | 93.6 | |
| ReMoLEModel=Llama3.2-3B, TP=3.11 %2025.09 | 93.44 | |
| Trinity Large Baseshot=5-shot2026.02 | 90.11 | |
| LD-MoLEModel=Qwen3-1.7B, TP=4.23 %2025.09 | 88.71 | |
| ReMoLEModel=Qwen3-1.7B, TP=4.12 %2025.09 | 88 | |
| MoLA(8888)Model=Llama3.2-3B, TP=3.11 %2025.09 | 87.47 | |
| MoLA(2468)Model=Llama3.2-3B, TP=1.80 %2025.09 | 87.31 | |
| MoLA(2468)Model=Qwen3-1.7B, TP=2.39 %2025.09 | 87.09 | |
| MoLA(8888)Model=Qwen3-1.7B, TP=4.12 %2025.09 | 83.35 | |
| FSWModel=Mistral-7B-v0.32026.03 | 69 | |
| Clean FTModel=LLaMA-3-8B2026.03 | 68.6 | |
| Clean FTModel=Mistral-7B-v0.32026.03 | 68.6 | |
| FSWModel=LLaMA-3-8B2026.03 | 67.8 | |
| Clean FTModel=DeepSeek-7B-Chat2026.03 | 67.4 | |
| Clean FTModel=Qwen2.5-7B2026.03 | 67 | |
| FSWModel=Qwen2.5-7B2026.03 | 66.8 | |
| Clean FTModel=LLaMA-2-7B-hf2026.03 | 66.4 | |
| FSWModel=LLaMA-2-7B-hf2026.03 | 64.8 | |
| UM-190kBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 61.62 | |
| UM-187kBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 60.69 | |
| FSWModel=DeepSeek-7B-Chat2026.03 | 60.4 | |
| UM-170kBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 59.9 | |
| TuluDPOBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 59.52 | |
| AdaGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 57.94 | |
| ORPOBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 57.82 | |
| SFTBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 57.32 | |
| UltraFBBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 57.14 | |
| HelpSteerBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 56.9 | |
| GlobalGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 56.06 | |
| AdaGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.02 | 55.85 | |
| AGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 55.47 | |
| UM-190kshots=10-shot2025.11 | 55.09 | |
| UM-187kshots=10-shot2025.11 | 54.93 | |
| UltraFBshots=10-shot2025.11 | 53.98 | |
| ORPOshots=10-shot2025.11 | 53.69 | |
| ClipByValueBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 53.49 | |
| TuluDPOshots=10-shot2025.11 | 53.42 | |
| ClippyBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 53.34 | |
| GlobalGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.02 | 53.08 | |
| SFTshots=10-shot2025.11 | 52.91 | |
| UM-170kshots=10-shot2025.11 | 52.84 | |
| HelpSteershots=10-shot2025.11 | 52.73 | |
| CodePrefBackbone=Apertus-8B-SFT, Evaluation Protocol=10-shot2025.11 | 51.31 | |
| CodePrefshots=10-shot2025.11 | 50.23 | |
| AdaGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 47.29 | |
| GlobalGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 46.62 | |
| ClippyBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 45.43 | |
| ClipByValueBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 44.11 | |
| BYOL-mriParameters=12B, Training Stage=CPT2026.01 | 38.11 | |
| BYOL-mriParameters=4B, Training Stage=CPT2026.01 | 37.8 | |
| BYOL-mriParameters=1B, Training Stage=CPT2026.01 | 30.83 |