Model Editing on COUNTERFACT 2,000-record GPT-J (test)
91.5Score (S)ROME
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ROMEEditor=ROME, Base Model=GPT-J (6B)2022.02 | 91.5 | 99.9 | 99.4 | 99.1 | 74.1 | 78.9 | 5.2 | 620.1 | 43 | |
| FT+LEditor=Fine-Tuning with L2 regularization, Base Model=GPT-J (6B)2022.02 | 68.7 | 99.6 | 95 | 47.9 | 30.4 | 78.6 | 6.8 | 622.8 | 35.5 | |
| MENDEditor=MEND, Base Model=GPT-J (6B)2022.02 | 63.2 | 97.4 | 71.5 | 53.6 | 11 | 53.9 | -6 | 620.5 | 32.6 | |
| FTEditor=Fine-Tuning, Base Model=GPT-J (6B)2022.02 | 25.5 | 100 | 99.9 | 96.6 | 71 | 10.3 | -50.7 | 387.8 | 24.6 | |
| GPT-JEditor=Pre-edited baseline, Base Model=GPT-J (6B)2022.02 | 23.6 | 16.3 | -7.2 | 18.6 | -7.4 | 83 | 7.3 | 621.8 | 29.8 |