Language Modeling on ECBD Target 2022 (A)
7.2Perplexity (PPL)Prepend Def.
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Prepend Def.Backbone=LLAMA-2-7B2023.06 | 7.2 | -1.4 | |
| Distillation (Mg = GPT3.5)Backbone=LLAMA-2-7B, Generator=GPT-3.52023.06 | 7.8 | -0.8 | |
| Distillation (Mg = Ms)Backbone=LLAMA-2-7B, Generator=Ms2023.06 | 8 | -0.6 | |
| Finetuning on de (last only)Backbone=LLAMA-2-7B2023.06 | 8.5 | -0.1 | |
| Pre-Edit PPLBackbone=LLAMA-2-7B2023.06 | 8.6 | — | |
| Prepend Random Def.Backbone=LLAMA-2-7B2023.06 | 8.6 | 0 | |
| Finetuning on de + Te (full)Backbone=LLAMA-2-7B2023.06 | 8.9 | 0.3 | |
| Finetuning on de (full)Backbone=LLAMA-2-7B2023.06 | 9 | 0.4 | |
| Prepend Def.Backbone=GPT-Neo-1.3B2023.06 | 21.9 | -9.1 | |
| Prepend Def.Backbone=GPT2-XL2023.06 | 24 | -8.9 | |
| Distillation (Mg = GPT3.5)Backbone=GPT-Neo-1.3B, Generator=GPT-3.52023.06 | 25.3 | -5.7 | |
| Distillation (Mg = Ms)Backbone=GPT-Neo-1.3B, Generator=Ms2023.06 | 26 | -5 | |
| Distillation (Mg = GPT3.5)Backbone=GPT2-XL, Generator=GPT-3.52023.06 | 26.8 | -6.1 | |
| Distillation (Mg = Ms)Backbone=GPT2-XL, Generator=Ms2023.06 | 27.6 | -5.3 | |
| Finetuning on de (full)Backbone=GPT-Neo-1.3B2023.06 | 28.5 | -2.5 | |
| Finetuning on de + Te (full)Backbone=GPT-Neo-1.3B2023.06 | 28.9 | -2.1 | |
| Finetuning on de (full)Backbone=GPT2-XL2023.06 | 30 | -2.9 | |
| Finetuning on de + Te (full)Backbone=GPT2-XL2023.06 | 30.6 | -2.3 | |
| Finetuning on de (last only)Backbone=GPT-Neo-1.3B2023.06 | 30.7 | -0.3 | |
| Pre-Edit PPLBackbone=GPT-Neo-1.3B2023.06 | 31 | — | |
| MEMITBackbone=GPT2-XL2023.06 | 32.6 | -0.2 | |
| Finetuning on de (last only)Backbone=GPT2-XL2023.06 | 32.8 | -0.1 | |
| Pre-Edit PPLBackbone=GPT2-XL2023.06 | 32.9 | — | |
| MENDBackbone=GPT-Neo-1.3B2023.06 | 35.2 | 4.2 | |
| Prepend Random Def.Backbone=GPT2-XL2023.06 | 40.3 | 7.4 | |
| Prepend Random Def.Backbone=GPT-Neo-1.3B2023.06 | 42.9 | 11.9 |