Machine Translation on Flores-101 (test)
3,920Average ScoreNLLB-3.3B
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NLLB-3.3Bparameters=3.3B2024.05 | 3,920 | — | — | — | — | — | — | — | — | — | 3,870 | 3,320 | 4,630 | 3,060 | 3,220 | 4,720 | 4,870 | 3,910 | 3,650 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NLLB-200-1.3Bparameters=1.3B2024.05 | 3,740 | — | — | — | — | — | — | — | — | — | 3,680 | 3,070 | 4,390 | 2,860 | 3,040 | 4,590 | 4,740 | 3,800 | 3,480 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=NLLB-200-3.3B, training=w/o T2024.05 | 3,550 | — | — | — | — | — | — | — | — | — | 3,450 | 2,950 | 4,390 | 2,660 | 2,900 | 4,320 | 4,520 | 3,580 | 3,180 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lego-MT2024.05 | 3,440 | — | — | — | — | — | — | — | — | — | 3,160 | 2,640 | 3,720 | 2,550 | 2,960 | 4,610 | 4,640 | 3,520 | 3,150 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=Lego-MT, training=w/o T2024.05 | 3,360 | — | — | — | — | — | — | — | — | — | 3,020 | 2,560 | 3,760 | 2,450 | 2,900 | 4,370 | 4,500 | 3,530 | 3,160 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| M2M100-1.2Bparameters=1.2B2024.05 | 3,200 | — | — | — | — | — | — | — | — | — | 2,770 | 2,390 | 3,410 | 2,480 | 2,670 | 4,290 | 4,390 | 3,430 | 3,000 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=NLLB-200-1.3B, training=w/o T2024.05 | 3,150 | — | — | — | — | — | — | — | — | — | 3,010 | 2,640 | 3,850 | 2,180 | 2,440 | 3,830 | 4,180 | 3,210 | 3,030 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=M2M100-1.2B, training=w/o T2024.05 | 3,070 | — | — | — | — | — | — | — | — | — | 2,660 | 2,000 | 3,260 | 2,450 | 2,700 | 3,930 | 4,170 | 3,330 | 3,120 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=mT5-xl, training=w/o T2024.05 | 2,910 | — | — | — | — | — | — | — | — | — | 2,590 | 2,350 | 3,410 | 1,980 | 2,230 | 3,730 | 3,910 | 3,070 | 2,900 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=M2M100-418M, training=w/o T2024.05 | 2,830 | — | — | — | — | — | — | — | — | — | 2,630 | 1,500 | 2,730 | 2,380 | 2,560 | 3,680 | 3,940 | 3,130 | 2,960 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| M2M100-418Mparameters=418M2024.05 | 2,680 | — | — | — | — | — | — | — | — | — | 2,510 | 1,820 | 2,660 | 2,020 | 2,120 | 3,640 | 3,900 | 2,840 | 2,580 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=mT5-large, training=w/o T2024.05 | 2,290 | — | — | — | — | — | — | — | — | — | 1,800 | 1,810 | 2,510 | 1,380 | 1,630 | 3,040 | 3,430 | 2,530 | 2,460 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=XLM-ROBERTa-large, training=w/o T2024.05 | 1,720 | — | — | — | — | — | — | — | — | — | 1,260 | 1,420 | 1,750 | 920 | 1,260 | 2,460 | 2,670 | 2,020 | 1,690 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=mBERT, training=w/o T2024.05 | 1,640 | — | — | — | — | — | — | — | — | — | 1,040 | 820 | 1,240 | 1,250 | 1,630 | 2,390 | 2,610 | 1,870 | 1,900 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MindMerger-Softbackbone=mGPT, training=w/o T2024.05 | 500 | — | — | — | — | — | — | — | — | — | 270 | 230 | 380 | 260 | 470 | 670 | 1,030 | 470 | 670 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MetaMath-Llama-7Bfew-shot=5-shots, parameters=7B2024.05 | 180 | — | — | — | — | — | — | — | — | — | 70 | 100 | 60 | 190 | 230 | 260 | 270 | 260 | 210 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMAX3-8B-Alpaca + WALARMethod=WALAR, Metric=xCOMET*2026.03 | 71.34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.44 | 76.42 | 68.06 | 68.48 | 73.36 | 71.05 | 58.56 | 68.03 | 82.39 | 74.65 | 71.99 | 72.8 | 62.2 | 60.31 | — | |
| NLLB-200-1.3BArchitecture=Encoder-Decoder, Metric=xCOMET*2026.03 | 71.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.08 | 79 | 62.78 | 70.5 | 73.12 | 72.77 | 62.91 | 69.44 | 81.46 | 76.23 | 65.36 | 70.49 | 61.29 | 61.93 | — | |
| LLaMAX3-8B-Alpaca + WALARMethod=WALAR, Metric=Gemini*2026.03 | 67.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.75 | 63.45 | 63.25 | 64.83 | 68.68 | 65.22 | 70.18 | 65.55 | 70 | 67.6 | 69.93 | 67.28 | 62.51 | 61.17 | — | |
| Qwen3-8B + WALARMethod=WALAR, Metric=xCOMET*2026.03 | 57.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84.77 | 62.98 | 55.15 | 56.07 | 64.55 | 57.13 | 48.27 | 55.78 | 76.71 | 61.17 | 71.37 | 59.61 | 21.84 | 35.17 | — | |
| LLaMAX3-8B-AlpacaMetric=Gemini*2026.03 | 57.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74.61 | 52.86 | 51.14 | 53.71 | 57.18 | 52.99 | 61.04 | 54.84 | 66.08 | 56.51 | 64.18 | 55.43 | 50.8 | 50.14 | — | |
| Qwen3-8BParameters=8B, Metric=xCOMET*2026.03 | 54.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.88 | 57.85 | 52.64 | 51.18 | 60.66 | 52.08 | 43.63 | 51.38 | 73.77 | 56.58 | 70.45 | 54.91 | 16.45 | 31.31 | — | |
| NLLB-200-1.3BArchitecture=Encoder-Decoder, Parameters=1.3B, Metric=spBLEU2026.03 | 24.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.03 | 30.23 | 24.91 | 22.57 | 23.3 | 22.47 | 24.51 | 22.22 | 25.8 | 22.18 | 18.71 | 18.4 | 24.37 | 21.77 | — | |
| Byte-nCFEmbedding=DENSE2022.05 | 19.9 | — | — | 32.1 | 25.5 | 19.8 | 19.5 | 22.2 | 13.5 | 6.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Byte-WSFEmbedding=DENSE2022.05 | 19.7 | — | — | 31.7 | 25.4 | 19.9 | 18.9 | 22.1 | 13.9 | 6.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMAX3-8B-Alpaca + WALARParameters=8B, Method=WALAR, Metric=spBLEU2026.03 | 19.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 32.56 | 23.68 | 17.81 | 17.14 | 18.02 | 17.86 | 18 | 17.61 | 22.45 | 17.9 | 20.08 | 15.23 | 17.15 | 17.35 | — | |
| Byte-nCFEmbedding=ONE-HOT2022.05 | 19.4 | — | — | 31.6 | 25.6 | 19.1 | 19.1 | 21.1 | 12.6 | 6.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Byte-WSFEmbedding=ONE-HOT2022.05 | 19.3 | — | — | 31.3 | 24.9 | 19.2 | 18.8 | 20.8 | 13.1 | 6.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SubwordEmbedding=DENSE2022.05 | 19.2 | — | — | 31.5 | 24.8 | 18.1 | 18.2 | 20.3 | 12.6 | 9.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMAX3-8B-Alpaca + WALAR-SFTParameters=8B, Method=WALAR-SFT, Metric=spBLEU2026.03 | 18.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 32.78 | 22.72 | 17.95 | 16.51 | 16.82 | 16.49 | 18.15 | 16.6 | 22.1 | 17.39 | 19.13 | 14.52 | 16.04 | 16.3 | — | |
| ByteEmbedding=DENSE2022.05 | 18.6 | — | — | 31.3 | 23.9 | 18.2 | 17.9 | 20.5 | 12.4 | 5.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByteEmbedding=ONE-HOT2022.05 | 18.5 | — | — | 31.1 | 24.3 | 17.9 | 18.3 | 19.9 | 12.2 | 5.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CharEmbedding=DENSE2022.05 | 17.4 | — | — | 29.2 | 21.9 | 17.3 | 17.4 | 17.5 | 11.1 | 7.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMAX3-8B-AlpacaParameters=8B, Config=Alpaca-tuned, Metric=spBLEU2026.03 | 17.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 32.24 | 21.34 | 17.5 | 14.14 | 12.23 | 14.89 | 16.64 | 15.42 | 21.48 | 16.2 | 18.37 | 12.99 | 13.21 | 15.12 | — | |
| Translategemma-4B-it + WALARParameters=4B, Method=WALAR, Metric=spBLEU2026.03 | 16.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.26 | 19.81 | 19.06 | 13.7 | 16.65 | 14.34 | 17.76 | 14.41 | 21.11 | 14.75 | 19.9 | 12.21 | 12.44 | 13.24 | — | |
| Translategemma-4B-itParameters=4B, Config=Instruction-tuned, Metric=spBLEU2026.03 | 16.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.45 | 18.19 | 18.86 | 12.58 | 16.39 | 13.05 | 17.85 | 12.75 | 20.62 | 13.74 | 19.46 | 11.39 | 10.17 | 11.61 | — | |
| Tower-Plus-9BParameters=9B, Metric=spBLEU2026.03 | 14.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.55 | 15.32 | 9.42 | 10.69 | 13.36 | 11.15 | 20.53 | 11.39 | 23.06 | 11.54 | 23.01 | 9.98 | 3.29 | 9.46 | — | |
| Qwen3-8B + WALARParameters=8B, Method=WALAR, Metric=spBLEU2026.03 | 14.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.59 | 17.11 | 15.71 | 11.89 | 14.73 | 12.38 | 12.03 | 12.05 | 19.31 | 12.68 | 20.87 | 11.08 | 3.35 | 7.76 | — | |
| Qwen3-8BParameters=8B, Metric=spBLEU2026.03 | 13.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.22 | 16.67 | 15.7 | 11.11 | 14 | 11.45 | 12.1 | 11.48 | 19.05 | 12.08 | 21.08 | 10.58 | 1.29 | 6.72 | — | |
| Hunyuan-MT-7BParameters=7B, Metric=spBLEU2026.03 | 12.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 21.04 | 14.32 | 16.29 | 9.84 | 15.37 | 10.72 | 13.19 | 9.66 | 16.17 | 10.46 | 15.55 | 9.83 | 8.18 | 7.46 | — | |
| Aya-Expanse-8BParameters=8B, Metric=spBLEU2026.03 | 12.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.03 | 14.25 | 14.59 | 10.33 | 12.73 | 10.62 | 14.07 | 10.2 | 17.22 | 10.72 | 15.49 | 9.29 | 2.36 | 4.71 | — | |
| Contrastive DecodingBase Model=LLaMA2-7b-Chat2024.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.5 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSLTotal Parameters (|θ|)=328M, Effective Parameters (|θ|eff)=186M, architecture=shared decoder, LSL configuration=SRC={4} & TGT={13-16}2023.05 | — | 46.1 | 26.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSL-NASTotal Parameters (|θ|)=441M, Effective Parameters (|θ|eff)=186M, architecture=separate decoder2023.05 | — | 46.4 | 27.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSL-NAS-SDTotal Parameters (|θ|)=356M, Effective Parameters (|θ|eff)=186M, architecture=shared decoder2023.05 | — | 46.3 | 26.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSL-NAS-SDTotal Parameters (|θ|)=356M, Effective Parameters (|θ|eff)=186M, architecture=shared decoder, pre-training=dense2023.05 | — | 46.6 | 27.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Reweight AttentionBase Model=LLaMA2-7b-Chat2024.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.8 | 84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Separate Decoder BaselineTotal Parameters (|θ|)=299M, Effective Parameters (|θ|eff)=186M, architecture=separate decoder2023.05 | — | 45.5 | 26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder AdaptersTotal Parameters (|θ|)=211M, Effective Parameters (|θ|eff)=189M, architecture=shared decoder, bottleneck size=1282023.05 | — | 44.6 | 24.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder AdaptersTotal Parameters (|θ|)=236M, Effective Parameters (|θ|eff)=191M, architecture=shared decoder, bottleneck size=2562023.05 | — | 44.9 | 25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder AdaptersTotal Parameters (|θ|)=286M, Effective Parameters (|θ|eff)=196M, architecture=shared decoder, bottleneck size=5122023.05 | — | 45.3 | 25.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder AdaptersTotal Parameters (|θ|)=311M, Effective Parameters (|θ|eff)=199M, architecture=shared decoder, bottleneck size=6402023.05 | — | 45.3 | 25.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder BaselineTotal Parameters (|θ|)=240M, Effective Parameters (|θ|eff)=240M, architecture=shared decoder2023.05 | — | 44.7 | 24.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder BaselineTotal Parameters (|θ|)=268M, Effective Parameters (|θ|eff)=268M, architecture=shared decoder, hidden dimension=6402023.05 | — | 45.1 | 25.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder BaselineTotal Parameters (|θ|)=285M, Effective Parameters (|θ|eff)=285M, architecture=shared decoder, hidden dimension=7042023.05 | — | 45.8 | 26.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Shared Decoder BaselineTotal Parameters (|θ|)=296M, Effective Parameters (|θ|eff)=296M, architecture=shared decoder, hidden dimension=7682023.05 | — | 45.8 | 26.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Target-constrained tuning LoRABase Model=LLaMA2-7b-Chat, tuning_method=LoRA2024.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.5 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla Instruction tuning LoRABase Model=LLaMA2-7b-Chat, tuning_method=LoRA2024.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.9 | 84.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla ZeroshotBase Model=LLaMA2-7b-Chat2024.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.3 | 83.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |