Commonsense Reasoning on HellaSwag EN
76.53AccuracyNeuronMoE
Evaluation results
| Method | Links | |
|---|---|---|
| NeuronMoEBackbone=Llama-3.2-3B, Stage=Stage 2, #Experts=49 (-42%)2026.03 | 76.53 | |
| LayerMoEBackbone=Llama-3.2-3B, Stage=Stage 2, #Experts=842026.03 | 76.5 | |
| NeuronMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=47 (-44%)2026.03 | 76.47 | |
| NeuronMoE-ENBackbone=Llama-3.2-3B, Stage=Stage 1, #Experts=37 (-56%)2026.03 | 76.45 | |
| NeuronMoE-S1Backbone=Llama-3.2-3B, Stage=Stage 1, #Experts=49 (-42%)2026.03 | 76.34 | |
| DenseTarget Language=Turkish, Backbone=Llama-3.2-3B2026.03 | 76.33 | |
| DenseTarget Language=Hungarian, Backbone=Llama-3.2-3B2026.03 | 76.33 | |
| Llama-3.2-3B DenseBackbone=Llama-3.2-3B, Stage=Dense, #Experts=-2026.03 | 76.33 | |
| LayerMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=842026.03 | 76.28 | |
| LayerMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=842026.03 | 76.28 | |
| NeuronMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=50 (-40%)2026.03 | 76.22 | |
| LayerMoE-S1Backbone=Llama-3.2-3B, Stage=Stage 1, #Experts=842026.03 | 75.84 | |
| NeuronMoE-S1Backbone=Qwen-1.5-1.8B, Stage=Stage 1, #Experts=36 (-50%)2026.03 | 61.2 | |
| LayerMoE-S1Backbone=Qwen-1.5-1.8B, Stage=Stage 1, #Experts=722026.03 | 59.88 |