Machine Translation on NLLB-200 202 languages (test)
46.68chrF++ (High->High)54.5B MoE model
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 54.5B MoE modelEnc experts=768, Dec experts=768, Pruning Percentage=0%, Decoding Hardware=1-GPU2022.12 | 46.68 | 39.36 | 33.56 | 40.53 | 35.49 | 30.07 | 40.46 | 35.49 | 30.16 | 35.74 | |
| Fixed per layer (lang)Enc experts=216, Dec experts=72, Pruning Percentage=80%, Pruning Metric=importance metric, Decoding Hardware=1-GPU2022.12 | 46.67 | 39.59 | 33.33 | 40.19 | 35.5 | 29.67 | 39.94 | 35.29 | 29.5 | 35.46 | |
| 3.3B dense modelEnc experts=6, Dec experts=6, Pruning Percentage=0%, Decoding Hardware=1-GPU2022.12 | 45.54 | 38.84 | 32.72 | 39.18 | 34.87 | 29.07 | 38.39 | 34.11 | 29.21 | 34.64 |