Factoid Knowledge Evaluation on PopQA
85.8LAMA ScoreREMIX (Random / K-Pile)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| REMIX (Random / K-Pile)Stage 1 mixing=Random Word Sequence, Stage 2 mixing=Knowledge Pile2024.11 | 85.8 | 90.7 | 80.5 | 85.7 | |
| Parameter ExpansionImplementation=LoRA adaptors2024.11 | 83 | 84.3 | 80 | 82.4 | |
| Weight RegularizationImplementation=Elastic Weight Consolidation (EWC)2024.11 | 12.1 | 67.4 | 76.7 | 52.1 | |
| No Mixing2024.11 | 7.7 | 57.8 | 72.5 | 46 | |
| Behavior RegularizationObjective=KL between training model vs original reference model2024.11 | 7.5 | 59.3 | 55.5 | 40.7 |