Machine Translation on WMT En-De 14 (val)
27.61BLEUG-XKD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| G-XKDInitialization Setting=large→base2026.02 | 27.61 | — | |
| MiniLLMInitialization Setting=large→base2026.02 | 27.54 | — | |
| S-XKDInitialization Setting=large→base2026.02 | 27.44 | — | |
| GKDInitialization Setting=large→base2026.02 | 27.43 | — | |
| SeqXKDInitialization Setting=large→base2026.02 | 27.28 | — | |
| SeqKDInitialization Setting=large→base2026.02 | 27.26 | — | |
| SKDInitialization Setting=large→base2026.02 | 27.24 | — | |
| SFTInitialization Setting=large→base2026.02 | 27.22 | — | |
| G-XKDInitialization Setting=large→small2026.02 | 26.73 | — | |
| MiniLLMInitialization Setting=large→small2026.02 | 26.65 | — | |
| S-XKDInitialization Setting=large→small2026.02 | 26.63 | — | |
| GKDInitialization Setting=large→small2026.02 | 26.61 | — | |
| S-STEArchitecture=Transformer-base, Avg epoch loss=4.6172024.09 | 26.53 | 4.011 | |
| DenseArchitecture=Transformer-base, Avg epoch loss=4.5552024.09 | 26.49 | 3.977 | |
| SeqXKDInitialization Setting=large→small2026.02 | 26.23 | — | |
| SeqKDInitialization Setting=large→small2026.02 | 26.15 | — | |
| SKDInitialization Setting=large→small2026.02 | 26.1 | — | |
| SR-STEArchitecture=Transformer-base, Avg epoch loss=4.612024.09 | 26.08 | 4.023 | |
| SFTInitialization Setting=large→small2026.02 | 26.04 | — | |
| STEPArchitecture=Transformer-base, Avg epoch loss=4.6822024.09 | 26.01 | 4.085 |