List your company
ResearchFormerly WizWand

Reading Comprehension on Belebele EN

75.33AccuracyLayerMoE
23.714837.114950.51563.9151Dec 3, 2025Dec 18, 2025Jan 2, 2026Jan 18, 2026Feb 2, 2026Feb 17, 2026Mar 5, 2026
Updated 8mo ago

Evaluation results

MethodLinks
LayerMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=842026.0375.33
LayerMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=842026.0375.33
NeuronMoE-S1Backbone=Llama-3.2-3B, Stage=Stage 1, #Experts=49 (-42%)2026.0375.33
NeuronMoEBackbone=Llama-3.2-3B, Stage=Stage 2, #Experts=49 (-42%)2026.0375.11
NeuronMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=47 (-44%)2026.0374.78
LayerMoEBackbone=Llama-3.2-3B, Stage=Stage 2, #Experts=842026.0374.78
NeuronMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=50 (-40%)2026.0374.44
Llama-3.2-3BTraining Status=No training2025.1274.2
DenseTarget Language=Turkish, Backbone=Llama-3.2-3B2026.0374.11
DenseTarget Language=Hungarian, Backbone=Llama-3.2-3B2026.0374.11
Llama-3.2-3B DenseBackbone=Llama-3.2-3B, Stage=Dense, #Experts=-2026.0374.11
LayerMoE-S1Backbone=Llama-3.2-3B, Stage=Stage 1, #Experts=842026.0373.89
NeuronMoE-ENBackbone=Llama-3.2-3B, Stage=Stage 1, #Experts=37 (-56%)2026.0373.56
Llama-3.2-3BTraining Status=Continued pretraining, Tokenizer Strategy=prune+ext (16k, naive)2025.1266
Llama-3.2-3BTraining Status=Continued pretraining, Tokenizer Strategy=prune+ext (16k, ours)2025.1265
Llama-3.2-3BTraining Status=Continued pretraining, Tokenizer Strategy=default2025.1264.7
NeuronMoE-S1Backbone=Qwen-1.5-1.8B, Stage=Stage 1, #Experts=36 (-50%)2026.0358.67
LayerMoE-S1Backbone=Qwen-1.5-1.8B, Stage=Stage 1, #Experts=722026.0354.89
Llama-3.2-1BTraining Status=No training2025.1235.7
Llama-3.2-1BTraining Status=Continued pretraining, Tokenizer Strategy=prune+ext (16k, ours)2025.1227.1
Llama-3.2-1BTraining Status=Continued pretraining, Tokenizer Strategy=default2025.1226.9
Llama-3.2-1BTraining Status=Continued pretraining, Tokenizer Strategy=prune+ext (16k, naive)2025.1225.7