Language Modeling on SciQ
11.95Perplexity(w+a)kNN-LMa (rescore)
Evaluation Results
| Method | Links | |
|---|---|---|
| (w+a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=combined, rescore=true, context_representation=LN22022.11 | 11.95 | |
| (w+a)kNN-LMaLMa=domain-adapted GPT-2, datastore=combined, context_representation=LN22022.11 | 12.15 | |
| (a)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=adaptation, rescore=true, context_representation=LN22022.11 | 12.72 | |
| (a)kNN-LMaLMa=domain-adapted GPT-2, datastore=adaptation, context_representation=LN22022.11 | 12.88 | |
| (w)kNN-LMa (rescore)LMa=domain-adapted GPT-2, datastore=pretraining, rescore=true, context_representation=LN22022.11 | 14.22 | |
| (w)kNN-LMaLMa=domain-adapted GPT-2, datastore=pretraining, context_representation=LN22022.11 | 14.62 | |
| (a)kNN-LMLM=GPT-2, datastore=adaptation, context_representation=original2022.11 | 14.71 | |
| LMa (only)LMa=domain-adapted GPT-22022.11 | 16.09 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=LN22022.11 | 18.5 | |
| (w)kNN-LMLM=GPT-2, datastore=pretraining, context_representation=original2022.11 | 19.19 | |
| LM (only)LM=GPT-22022.11 | 22.15 | |
| (w)kNN (only)datastore=pretraining2022.11 | 46.86 | |
| (a)kNN (only)datastore=adaptation2022.11 | 57.82 |