Generalization on LLaMA Evaluation Expanded Languages 3.1-8B
69.37Overall ScoreDeltaMoE
Evaluation results
| Method | Links | |
|---|---|---|
| DeltaMoECategory=Baseline w/ Matched Parameters2026.05 | 69.37 | |
| MoLACategory=Baseline w/ Matched Parameters2026.05 | 68.39 | |
| Dense-FT-DeltaCategory=Baselines w/ Same CPT Data2026.05 | 66.88 | |
| Dense-FT-AvgCategory=Baselines w/ Same CPT Data2026.05 | 66.39 | |
| Dense-FT-Avg-2FLOPsCategory=Baselines w/ Matched FLOPs (2x CPT Data)2026.05 | 65.81 | |
| LLaMA-3.1-8B-InstructCategory=Base Model2026.05 | 64.91 | |
| Dense-FT-Delta-2FLOPsCategory=Baselines w/ Matched FLOPs (2x CPT Data)2026.05 | 64.38 | |
| LLaMA-ProCategory=Baseline w/ Matched Parameters2026.05 | 58.92 |