Machine Translation on IWSLT
35.56BLEUPenumbral
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Penumbral2023.06 | 35.56 | — | |
| Umbral2023.06 | 35.07 | — | |
| Laplacian Kernel2023.06 | 34.68 | — | |
| Model Default2023.06 | 34.56 | — | |
| Dot Product2023.06 | 34.56 | — | |
| dH Hyperboloid2023.06 | 33.8 | — | |
| NeoDiffModel type=H, Beam size=102025.05 | 33.14 | — | |
| dH Hn xi2023.06 | 32.54 | — | |
| NeoDiffModel type=H, Beam size=12025.05 | 32.39 | — | |
| DifformerModel type=C, Beam size=102025.05 | 32.09 | — | |
| DifformerModel type=C, Beam size=12025.05 | 30.94 | — | |
| AR-DiffusionModel type=AR, Beam size=102025.05 | 30.64 | — | |
| SeqDiffuSeqModel type=C, Beam size=102025.05 | 30.03 | — | |
| SeqDiffuSeqModel type=C, Beam size=12025.05 | 28.65 | — | |
| AbsorbingModel type=D, Beam size=52025.05 | 28.32 | — | |
| AR-DiffusionModel type=AR, Beam size=12025.05 | 26.78 | — | |
| MultinomialModel type=D, Beam size=52025.05 | 21.28 | — | |
| GPT-4oThreshold=0.152024.06 | 0.32 | 2.66 | |
| GPT-4oThreshold=0.202024.06 | 0.32 | 3.08 | |
| Mistral-Nemo 12BThreshold=0.152024.06 | 0.27 | 2.87 | |
| Mistral-Nemo 12BThreshold=0.202024.06 | 0.27 | 3.4 | |
| Llama-3 8BThreshold=0.152024.06 | 0.24 | 3.4 | |
| Llama-3 8BThreshold=0.202024.06 | 0.23 | 3.92 | |
| Mistral 7BThreshold=0.152024.06 | 0.22 | 3.52 | |
| Mistral 7BThreshold=0.202024.06 | 0.22 | 4.14 | |
| Gemma-2 9BThreshold=0.152024.06 | 0.21 | 4.4 | |
| Claude 3.5 SonnetThreshold=0.152024.06 | 0.2 | 4.63 | |
| Claude 3.5 SonnetThreshold=0.202024.06 | 0.2 | 4.87 | |
| Gemma-2 9BThreshold=0.202024.06 | 0.2 | 4.75 | |
| Gemma 7BThreshold=0.202024.06 | 0.09 | 5.84 | |
| Gemma 7BThreshold=0.152024.06 | 0.08 | 5.39 | |
| Gemma 7BFramework=Adaptive HPF, Threshold=0.152024.06 | — | 6.5947 | |
| Gemma 7BFramework=Adaptive HPF, Threshold=0.202024.06 | — | 6.6605 | |
| Llama-3 8BFramework=Adaptive HPF, Threshold=0.152024.06 | — | 6.2006 | |
| Llama-3 8BFramework=Adaptive HPF, Threshold=0.202024.06 | — | 6.2841 | |
| Mistral 7BFramework=Adaptive HPF, Threshold=0.152024.06 | — | 6.2478 | |
| Mistral 7BFramework=Adaptive HPF, Threshold=0.202024.06 | — | 6.4604 | |
| Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.152024.06 | — | 6.3955 | |
| Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.202024.06 | — | 6.4936 |