List your company
ResearchFormerly WizWand

Machine Translation on IWSLT

35.56BLEUPenumbral
-1.33928.240417.8227.3996Jun 1, 2023Sep 30, 2023Jan 29, 2024May 29, 2024Sep 27, 2024Jan 26, 2025May 28, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Penumbral2023.0635.56—
Umbral2023.0635.07—
Laplacian Kernel2023.0634.68—
Model Default2023.0634.56—
Dot Product2023.0634.56—
dH Hyperboloid2023.0633.8—
NeoDiffModel type=H, Beam size=102025.0533.14—
dH Hn xi2023.0632.54—
NeoDiffModel type=H, Beam size=12025.0532.39—
DifformerModel type=C, Beam size=102025.0532.09—
DifformerModel type=C, Beam size=12025.0530.94—
AR-DiffusionModel type=AR, Beam size=102025.0530.64—
SeqDiffuSeqModel type=C, Beam size=102025.0530.03—
SeqDiffuSeqModel type=C, Beam size=12025.0528.65—
AbsorbingModel type=D, Beam size=52025.0528.32—
AR-DiffusionModel type=AR, Beam size=12025.0526.78—
MultinomialModel type=D, Beam size=52025.0521.28—
GPT-4oThreshold=0.152024.060.322.66
GPT-4oThreshold=0.202024.060.323.08
Mistral-Nemo 12BThreshold=0.152024.060.272.87
Mistral-Nemo 12BThreshold=0.202024.060.273.4
Llama-3 8BThreshold=0.152024.060.243.4
Llama-3 8BThreshold=0.202024.060.233.92
Mistral 7BThreshold=0.152024.060.223.52
Mistral 7BThreshold=0.202024.060.224.14
Gemma-2 9BThreshold=0.152024.060.214.4
Claude 3.5 SonnetThreshold=0.152024.060.24.63
Claude 3.5 SonnetThreshold=0.202024.060.24.87
Gemma-2 9BThreshold=0.202024.060.24.75
Gemma 7BThreshold=0.202024.060.095.84
Gemma 7BThreshold=0.152024.060.085.39
Gemma 7BFramework=Adaptive HPF, Threshold=0.152024.06—6.5947
Gemma 7BFramework=Adaptive HPF, Threshold=0.202024.06—6.6605
Llama-3 8BFramework=Adaptive HPF, Threshold=0.152024.06—6.2006
Llama-3 8BFramework=Adaptive HPF, Threshold=0.202024.06—6.2841
Mistral 7BFramework=Adaptive HPF, Threshold=0.152024.06—6.2478
Mistral 7BFramework=Adaptive HPF, Threshold=0.202024.06—6.4604
Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.152024.06—6.3955
Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.202024.06—6.4936