Machine Translation (EN->DE) on IWSLT 2017 (test)
37.1BLEUFAAST
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FAASTBackbone=Qwen2.5-7B-Instruct, Protocol=FAAST, Data Setting=full2026.05 | 37.1 | — | — | — | |
| FAASTBackbone=Qwen2.5-3B-Instruct, Protocol=FAAST, Data Setting=full2026.05 | 36.4 | — | — | — | |
| oursStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 36 | — | — | — | |
| In-Context LearningBackbone=Qwen2.5-7B-Instruct, Protocol=In-Context Learning, Data Setting=1-shot2026.05 | 35.7 | — | — | — | |
| DistiLLMStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 35.5 | — | — | — | |
| FAASTBackbone=Qwen2.5-7B-Instruct, Protocol=FAAST, Data Setting=1-shot2026.05 | 35.34 | — | — | — | |
| T5-XL (teacher)Model Architecture=T5-XL2024.06 | 35.2 | — | — | — | |
| GKDStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 34.7 | — | — | — | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Protocol=zero-shot, Data Setting=full2026.05 | 34.69 | — | — | — | |
| ImitKDStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 34.1 | — | — | — | |
| SFT (student)Student Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 33.7 | — | — | — | |
| FAASTBackbone=Qwen2.5-3B-Instruct, Protocol=FAAST, Data Setting=1-shot2026.05 | 33.23 | — | — | — | |
| Qwen2.5-3B-InstructBackbone=Qwen2.5-3B-Instruct, Protocol=zero-shot, Data Setting=full2026.05 | 32.92 | — | — | — | |
| SeqKDStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 32.9 | — | — | — | |
| oursStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 32.4 | — | — | — | |
| Google translatorTrained Model=-2025.05 | 32.39 | 80.2 | 92 | 68.2 | |
| In-Context LearningBackbone=Qwen2.5-3B-Instruct, Protocol=In-Context Learning, Data Setting=1-shot2026.05 | 32.33 | — | — | — | |
| GPT4.1-nanoTrained Model=-2025.05 | 31.89 | 83.9 | 96 | 70.6 | |
| KDStudent Model Architecture=T5-Large, Teacher Model=T5-XL2024.06 | 31.7 | — | — | — | |
| GKDStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 31.2 | — | — | — | |
| DistiLLMStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 30.8 | — | — | — | |
| SFT (student)Student Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 30.1 | — | — | — | |
| KDStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 29.8 | — | — | — | |
| SeqKDStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 29.2 | — | — | — | |
| ImitKDStudent Model Architecture=T5-Base, Teacher Model=T5-XL2024.06 | 28.7 | — | — | — | |
| FAASTBackbone=Qwen2.5-7B-Instruct, Protocol=FAAST, Data Setting=full2026.05 | 27.75 | — | — | — | |
| UniRBackbone=Llama3.2-3B, Trained Model=1B2025.05 | 27.03 | 81.43 | 93.7 | 67.39 | |
| FAASTBackbone=Qwen2.5-7B-Instruct, Protocol=FAAST, Data Setting=1-shot2026.05 | 26.77 | — | — | — | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Protocol=zero-shot, Data Setting=full2026.05 | 25.53 | — | — | — | |
| In-Context LearningBackbone=Qwen2.5-7B-Instruct, Protocol=In-Context Learning, Data Setting=1-shot2026.05 | 25.39 | — | — | — | |
| GRPO FullBackbone=Llama3.2-3B, Trained Model=3B2025.05 | 25.26 | 80.13 | 92.88 | 66.09 | |
| FAASTBackbone=Qwen2.5-3B-Instruct, Protocol=FAAST, Data Setting=full2026.05 | 25.22 | — | — | — | |
| GRPO LoRABackbone=Llama3.2-3B, Trained Model=3B2025.05 | 24.18 | 78.93 | 91.97 | 65.03 | |
| Backbone + 1BBackbone=Llama3.2-3B, Trained Model=-2025.05 | 24.1 | 78.7 | 91.47 | 64.76 | |
| oursStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 23.7 | — | — | — | |
| FAASTBackbone=Qwen2.5-3B-Instruct, Protocol=FAAST, Data Setting=1-shot2026.05 | 23.35 | — | — | — | |
| Qwen2.5-3B-InstructTrained Model=-2025.05 | 23.28 | 78.25 | 91.26 | 64.26 | |
| Qwen2.5-3B-InstructBackbone=Qwen2.5-3B-Instruct, Protocol=zero-shot, Data Setting=full2026.05 | 23.22 | — | — | — | |
| Backbone onlyBackbone=Llama3.2-3B, Trained Model=-2025.05 | 23.19 | 77.72 | 90.97 | 63.96 | |
| In-Context LearningBackbone=Qwen2.5-3B-Instruct, Protocol=In-Context Learning, Data Setting=1-shot2026.05 | 23.03 | — | — | — | |
| GKDStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 22.7 | — | — | — | |
| DistiLLMStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 22.5 | — | — | — | |
| ImitKDStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 22.2 | — | — | — | |
| KDStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 21.7 | — | — | — | |
| SFT (student)Student Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 21.5 | — | — | — | |
| SeqKDStudent Model Architecture=T5-Small, Teacher Model=T5-XL2024.06 | 21.2 | — | — | — |