Language Modeling on XSum
14.71Perplexity(w+a)kNN-LMa (rescore)
Evaluation Results
| Method | Links | |
|---|---|---|
| (w+a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=combined, rescore=true, context_representation=LN22022.11 | 14.71 | |
| (a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=adaptation, rescore=true, context_representation=LN22022.11 | 14.85 | |
| (w+a)kNN-LMaLMa=domain-adapted GPT-2, datastore=combined, context_representation=LN22022.11 | 15.2 | |
| (a)kNN-LMaLMa=domain-adapted GPT-2, datastore=adaptation, context_representation=LN22022.11 | 15.3 | |
| (a)kNN-LMLM=GPT-2, datastore=adaptation, context_representation=original2022.11 | 17.01 | |
| (w+a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=combined, rescore=true, context_representation=LN22022.11 | 17.53 | |
| (a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=adaptation, rescore=true, context_representation=LN22022.11 | 17.72 | |
| (w+a)kNN-LMaLMa=domain-adapted GPT-2, datastore=combined, context_representation=LN22022.11 | 17.99 | |
| (a)kNN-LMaLMa=domain-adapted GPT-2, datastore=adaptation, context_representation=LN22022.11 | 18.12 | |
| (w)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=pretraining, rescore=true, context_representation=LN22022.11 | 18.23 | |
| (w)kNN-LMaLMa=domain-adapted GPT-2, datastore=pretraining, context_representation=LN22022.11 | 18.42 | |
| LMa (only)LMa=domain-adapted GPT-22022.11 | 18.95 | |
| (a)kNN-LMLM=GPT-2, datastore=adaptation, context_representation=original2022.11 | 19.39 | |
| (w)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=pretraining, rescore=true, context_representation=LN22022.11 | 20.98 | |
| (w)kNN-LMaLMa=domain-adapted GPT-2, datastore=pretraining, context_representation=LN22022.11 | 21.22 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=LN22022.11 | 21.32 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=original2022.11 | 21.64 | |
| LMa (only)LMa=domain-adapted GPT-22022.11 | 21.84 | |
| LM (only)LM=GPT-2, configuration=standard2022.11 | 22.45 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=LN22022.11 | 23.68 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=original2022.11 | 24.03 | |
| LM (only)LM=GPT-22022.11 | 24.87 | |
| (a)kNN (only)datastore=adaptation2022.11 | 38.38 | |
| (a)kNN (only)datastore=adaptation2022.11 | 47.75 | |
| (w)kNN (only)datastore=pretraining2022.11 | 83.96 | |
| (w)kNN (only)datastore=pretraining2022.11 | 100.92 |