Factoid Knowledge Evaluation on KVR
62.4LAMA ScoreREMIX (Random / K-Pile)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| REMIX (Random / K-Pile)Stage 1 mixing=Random Word Sequence, Stage 2 mixing=Knowledge Pile2024.11 | 62.4 | 69.5 | 70.2 | 67.4 | |
| Parameter ExpansionImplementation=LoRA adaptors2024.11 | 52.3 | 52.2 | 68.8 | 57.8 | |
| No Mixing2024.11 | 2.1 | 17.4 | 33.8 | 17.8 | |
| Behavior RegularizationObjective=KL between training model vs original reference model2024.11 | 0.2 | 15.6 | 36.6 | 17.5 | |
| Weight RegularizationImplementation=Elastic Weight Consolidation (EWC)2024.11 | 0.1 | 4.3 | 76.7 | 27.3 |