Machine Translation on FLORES (val)
299WPSExpert Pruning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Expert PruningPruning rate=80%, Batch size=16k, GPUs=42022.12 | 299 | 79 | |
| 3.3B Dense BaselineBatch size=4k, GPUs=12022.12 | 246 | 86 | |
| 54.5B MoEBatch size=16k, GPUs=82022.12 | 195 | 105 | |
| Expert PruningPruning rate=80%, Batch size=4k, GPUs=12022.12 | 172 | 135 | |
| 54.5B MoEBatch size=16k, GPUs=42022.12 | 156 | 131 |