Machine Translation on Multi-Domain (test)
56.5IT Domain BLEU ScoreCANEFT
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CANEFTBackbone=Qwen2.5-7B-Instruct, TP (M)=127, AP (M)=02026.02 | 56.5 | — | — | — | — | — | — | 91.4 | 49.8 | 87.2 | 52.3 | 90.1 | 47.2 | 88.3 | 21.5 | 76.1 | 45.5 | 86.6 | |
| CANEFTBackbone=LLaMA3.1-8B-Instruct, TP (M)=146, AP (M)=02026.02 | 54.8 | — | — | — | — | — | — | 90.7 | 50.9 | 87.1 | 52.3 | 90.3 | 46.3 | 88 | 20.9 | 75.9 | 45 | 86.4 | |
| BaseBackbone=Qwen2.5-7B-Instruct, TP (M)=0, AP (M)=02026.02 | 54.3 | — | — | — | — | — | — | 90.5 | 42.6 | 86.2 | 50.3 | 89.5 | 44 | 87.8 | 19.4 | 75.1 | 42.1 | 85.8 | |
| FullBackbone=Qwen2.5-7B-Instruct, TP (M)=7000, AP (M)=02026.02 | 53.2 | — | — | — | — | — | — | 87.8 | 53.5 | 85.9 | 53.1 | 89.9 | 42.2 | 87.1 | 21.2 | 75.6 | 44.6 | 85.2 | |
| CANEFTBackbone=LLaMA2-7B-Chat, TP (M)=91, AP (M)=02026.02 | 48.8 | — | — | — | — | — | — | 88.7 | 43.8 | 84.4 | 50 | 88.3 | 43.7 | 86.8 | 19.2 | 74.1 | 40.5 | 84.5 | |
| BaseBackbone=LLaMA3.1-8B-Instruct, TP (M)=0, AP (M)=02026.02 | 48.1 | — | — | — | — | — | — | 87.8 | 38.3 | 84.4 | 46.4 | 87.4 | 44.4 | 87.4 | 19.3 | 75.5 | 39.3 | 84.5 | |
| FullBackbone=LLaMA3.1-8B-Instruct, TP (M)=8000, AP (M)=02026.02 | 47.2 | — | — | — | — | — | — | 84.3 | 56.3 | 88 | 51.4 | 88.7 | 42.1 | 87.2 | 21 | 75.5 | 43.6 | 84.7 | |
| LoRABackbone=LLaMA2-7B-Chat, TP (M)=20, AP (M)=202026.02 | 46.7 | — | — | — | — | — | — | 87.4 | 35.8 | 80.1 | 49.9 | 87.7 | 33.5 | 82.5 | 15.3 | 69.6 | 36.2 | 81.4 | |
| L-MLPBackbone=LLaMA2-7B-Chat, TP (M)=12, AP (M)=122026.02 | 46.4 | — | — | — | — | — | — | 86.9 | 40.8 | 82.5 | 49.7 | 88 | 33.7 | 82.5 | 14.4 | 70.2 | 37 | 82 | |
| DoRABackbone=LLaMA2-7B-Chat, TP (M)=22, AP (M)=222026.02 | 45.9 | — | — | — | — | — | — | 87.8 | 42.4 | 84.7 | 46.7 | 87.7 | 37.5 | 84.3 | 17.6 | 73.2 | 38 | 83.5 | |
| FullBackbone=LLaMA2-7B-Chat, TP (M)=7000, AP (M)=02026.02 | 43.7 | — | — | — | — | — | — | 87.8 | 47.9 | 82.8 | 47.1 | 85.5 | 35.3 | 81.1 | 20.5 | 73.8 | 38.9 | 82.2 | |
| LLaMA ProBackbone=LLaMA2-7B-Chat, TP (M)=405, AP (M)=4052026.02 | 40.4 | — | — | — | — | — | — | 83.9 | 46 | 84.9 | 43.5 | 87.2 | 31.3 | 81.3 | 16.4 | 72.4 | 35.5 | 81.9 | |
| LAPEBackbone=LLaMA2-7B-Chat, TP (M)=91, AP (M)=02026.02 | 39.3 | — | — | — | — | — | — | 80.6 | 30.4 | 80.5 | 40.5 | 85.8 | 29 | 76.2 | 15.5 | 73.1 | 30.9 | 79.2 | |
| BaseBackbone=LLaMA2-7B-Chat, TP (M)=0, AP (M)=02026.02 | 38.7 | — | — | — | — | — | — | 81.8 | 30 | 82.8 | 44.1 | 87.4 | 35.5 | 85 | 16 | 73.1 | 32.8 | 82 | |
| RCNBackbone=LLaMA2-7B-Chat, TP (M)=91, AP (M)=02026.02 | 25.5 | — | — | — | — | — | — | 84.1 | 30.8 | 82 | 33.6 | 85.8 | 40.4 | 86.1 | 14.6 | 72.7 | 28.9 | 82.1 | |
| SFT-3BModel Type=Dense, Architecture=Decoder-only, Backbone=Qwen2.5-3B2025.05 | — | 51.4 | 54.8 | 40.65 | 20.4 | 28.33 | 39.12 | — | — | — | — | — | — | — | — | — | — | — | |
| THOR-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-12025.05 | — | 52.52 | 56.25 | 43.96 | 21.93 | 28.41 | 40.61 | — | — | — | — | — | — | — | — | — | — | — | |
| THOR-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-22025.05 | — | 55.79 | 61.06 | 46 | 23.35 | 28.23 | 42.89 | — | — | — | — | — | — | — | — | — | — | — | |
| THOR-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-p (p=0.5)2025.05 | — | 53.58 | 58.65 | 44.63 | 22.53 | 27.99 | 41.48 | — | — | — | — | — | — | — | — | — | — | — | |
| TM-averageResource scenario=low-resource, Technique=Technique 2 (§4.1)2023.06 | — | 0.4713 | 0.5082 | 0.355 | 0.1487 | 0.213 | 0.3392 | — | — | — | — | — | — | — | — | — | — | — | |
| TM-baseResource scenario=low-resource2023.06 | — | 0.4353 | 0.4936 | 0.3276 | 0.1443 | 0.2003 | 0.3202 | — | — | — | — | — | — | — | — | — | — | — | |
| TM-singleResource scenario=low-resource, Technique=Technique 1 (§4.1)2023.06 | — | 0.4704 | 0.5084 | 0.3533 | 0.148 | 0.2122 | 0.3385 | — | — | — | — | — | — | — | — | — | — | — | |
| TM-weightResource scenario=low-resource, Technique=§4.22023.06 | — | 0.4797 | 0.5228 | 0.3584 | 0.1659 | 0.2258 | 0.3505 | — | — | — | — | — | — | — | — | — | — | — | |
| Trim-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-1, Backbone=Qwen1.5-MoE-A2.7B (trimmed)2025.05 | — | 51.8 | 55.83 | 40.03 | 19.55 | 25.5 | 38.54 | — | — | — | — | — | — | — | — | — | — | — | |
| Trim-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-2, Backbone=Qwen1.5-MoE-A2.7B (trimmed)2025.05 | — | 51.84 | 57.12 | 45.1 | 22.68 | 29.02 | 41.15 | — | — | — | — | — | — | — | — | — | — | — | |
| Trim-MoEModel Type=MoE, Architecture=Decoder-only, Routing Strategy=Top-p (p=0.5), Backbone=Qwen1.5-MoE-A2.7B (trimmed)2025.05 | — | 55.67 | 60.18 | 39.39 | 19.21 | 29.21 | 40.73 | — | — | — | — | — | — | — | — | — | — | — | |
| w/o TMResource scenario=low-resource2023.06 | — | 0.4762 | 0.5085 | 0.344 | 0.1445 | 0.2022 | 0.3351 | — | — | — | — | — | — | — | — | — | — | — |