Knowledge Modification on WikiData counterfact
100Edit Success RateFT-M
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FT-MBackbone=Llama2-7b-chat2024.06 | 100 | 72.39 | 40.76 | 586.8 | 54.71 | |
| ICEBackbone=Llama2-7b-chat2024.06 | 100 | 73.49 | 45.88 | 583.29 | 18.95 | |
| FT-MArchitecture=GPT2-xl2024.06 | 100 | 42.08 | 63.43 | 602.91 | 12.64 | |
| ICEArchitecture=GPT2-xl2024.06 | 100 | 39.78 | 67.06 | 560.33 | 8.05 | |
| FT-MBackbone=Llama2-7b-chat2024.06 | 100 | 72.55 | 2,499 | 514.86 | 10.74 | |
| ICEBackbone=Llama2-7b-chat2024.06 | 99.98 | 73.74 | 2,737 | 599.57 | 0.1 | |
| ROMEBackbone=Llama2-7b-chat2024.06 | 98.68 | 42.45 | 21.13 | 585.4 | 109.97 | |
| MEMITBackbone=Llama2-7b-chat2024.06 | 98.13 | 44.16 | 19.48 | 576.26 | 122.48 | |
| ROMEArchitecture=GPT2-xl2024.06 | 96.08 | 31.31 | 16.67 | 600.94 | 3.71 | |
| MEMITArchitecture=GPT2-xl2024.06 | 55.03 | 20 | 24.79 | 604.58 | 11.1 | |
| FT-LArchitecture=GPT2-xl2024.06 | 40.85 | 20.09 | 65.29 | 596.77 | 26.55 | |
| FT-LBackbone=Llama2-7b-chat2024.06 | 36.13 | 29.37 | 38.37 | 566.55 | 89.24 | |
| FT-LBackbone=Llama2-7b-chat2024.06 | 14.28 | 6.94 | 101 | 472.37 | 0.1 | |
| MEMITBackbone=Llama2-7b-chat2024.06 | 12.1 | 4.53 | 78 | 416.77 | 7.71 | |
| ROMEBackbone=Llama2-7b-chat2024.06 | 9.43 | 4.5 | 134 | 294.67 | 6.12 |