List your company
ResearchFormerly WizWand

Machine Translation on IWSLT (test)

5.3PerplexityImitKD
-0.60839.27179.15119.029Sep 15, 2020
Updated 8mo ago

Evaluation results

MethodLinks
ImitKDArchitecture=SRU (3-layer), Training Data=Full2020.095.3—0.3480.354
VanillaArchitecture=SRU (3-layer), Training Data=Full2020.095.4—0.3420.348
TeacherArchitecture=Transformer (8-layer)2020.095.6—0.3440.352
ImitKDArchitecture=SRU (2-layer), Training Data=Full2020.095.7—0.3370.345
VanillaArchitecture=Transformer (2-layer), Training Data=Full2020.095.9—0.3380.342
ImitKDArchitecture=Transformer (2-layer), Training Data=Full2020.095.9—0.3440.348
VanillaArchitecture=SRU (2-layer), Training Data=Full2020.096—0.330.338
SeqKDArchitecture=SRU (3-layer), Training Data=Full2020.096.1—0.3430.348
VanillaArchitecture=Transformer (2-layer)2020.096.4—0.3280.334
SeqKDArchitecture=SRU (2-layer), Training Data=Full2020.096.8—0.3310.337
SeqKDArchitecture=Transformer (2-layer), Training Data=Full2020.097.1—0.340.344
VanillaArchitecture=SRU (3-layer)2020.097.4—0.30.312
VanillaArchitecture=SRU (2-layer)2020.097.4—0.2950.306
ImitKDArchitecture=Transformer (2-layer)2020.097.5—0.3430.346
ImitKDArchitecture=SRU (2-layer)2020.0912.7—0.3330.337
ImitKDArchitecture=SRU (3-layer)2020.0914.7—0.3410.344
SeqKDArchitecture=Transformer (2-layer)2020.0923.4—0.340.342
SeqKDArchitecture=SRU (2-layer)2020.09102.1—0.320.324
SeqKDArchitecture=SRU (3-layer)2020.09153—0.330.331
Baseline# Softmaxes=12018.09—0.2741——
BaselineLR Schedule=Baseline, Training Epochs=502020.03—0.3497——
BPE-MoS# Softmaxes=92018.09—0.2891——
Gemma 7BFramework=Adaptive HPF, Threshold=0.152024.06—0.0845——
Gemma 7BFramework=Adaptive HPF, Threshold=0.22024.06—0.0773——
Hybrid-LightRNN-MoS# Softmaxes=92018.09—0.2879——
KneeLR Schedule=Knee, Training Epochs=502020.03—0.3553——
Knee (short budget)LR Schedule=Knee (short budget), Training Epochs=352020.03—0.3508——
Llama-3 8BFramework=Adaptive HPF, Threshold=0.152024.06—0.2114——
Llama-3 8BFramework=Adaptive HPF, Threshold=0.22024.06—0.2115——
Mistral 7BFramework=Adaptive HPF, Threshold=0.152024.06—0.2——
Mistral 7BFramework=Adaptive HPF, Threshold=0.22024.06—0.2057——
Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.152024.06—0.1556——
Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.22024.06—0.1535——