Language Modeling on ECBD Specificity 2022 (Δ)
8.6Perplexity (PPL)Distillation (Mg = Ms)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Distillation (Mg = Ms)Backbone=LLAMA-2-7B, Generator=Ms2023.06 | 8.6 | -0.2 | |
| Distillation (Mg = GPT3.5)Backbone=LLAMA-2-7B, Generator=GPT-3.52023.06 | 8.6 | -0.2 | |
| Finetuning on de (full)Backbone=LLAMA-2-7B2023.06 | 8.7 | -0.1 | |
| Pre-Edit PPLBackbone=LLAMA-2-7B2023.06 | 8.8 | — | |
| Finetuning on de (last only)Backbone=LLAMA-2-7B2023.06 | 8.8 | 0 | |
| Finetuning on de + Te (full)Backbone=LLAMA-2-7B2023.06 | 8.8 | 0 | |
| Prepend Def.Backbone=LLAMA-2-7B2023.06 | 8.8 | — | |
| Prepend Random Def.Backbone=LLAMA-2-7B2023.06 | 8.8 | — | |
| Distillation (Mg = GPT3.5)Backbone=GPT2-XL, Generator=GPT-3.52023.06 | 25.1 | -0.3 | |
| Distillation (Mg = Ms)Backbone=GPT2-XL, Generator=Ms2023.06 | 25.2 | -0.2 | |
| Pre-Edit PPLBackbone=GPT2-XL2023.06 | 25.4 | — | |
| Finetuning on de (full)Backbone=GPT2-XL2023.06 | 25.4 | 0 | |
| Finetuning on de (last only)Backbone=GPT2-XL2023.06 | 25.4 | 0 | |
| MEMITBackbone=GPT2-XL2023.06 | 25.4 | 0 | |
| Prepend Def.Backbone=GPT2-XL2023.06 | 25.4 | — | |
| Prepend Random Def.Backbone=GPT2-XL2023.06 | 25.4 | — | |
| Finetuning on de + Te (full)Backbone=GPT2-XL2023.06 | 25.5 | 0.1 | |
| Distillation (Mg = GPT3.5)Backbone=GPT-Neo-1.3B, Generator=GPT-3.52023.06 | 25.6 | -0.5 | |
| Distillation (Mg = Ms)Backbone=GPT-Neo-1.3B, Generator=Ms2023.06 | 25.9 | -0.2 | |
| Finetuning on de (full)Backbone=GPT-Neo-1.3B2023.06 | 26 | -0.1 | |
| Pre-Edit PPLBackbone=GPT-Neo-1.3B2023.06 | 26.1 | — | |
| Finetuning on de (last only)Backbone=GPT-Neo-1.3B2023.06 | 26.1 | 0 | |
| Finetuning on de + Te (full)Backbone=GPT-Neo-1.3B2023.06 | 26.1 | 0 | |
| Prepend Def.Backbone=GPT-Neo-1.3B2023.06 | 26.1 | — | |
| Prepend Random Def.Backbone=GPT-Neo-1.3B2023.06 | 26.1 | — | |
| MENDBackbone=GPT-Neo-1.3B2023.06 | 26.4 | 0.3 |