Model Editing on CounterFact (Reliability, Generality, Locality)
92ReliabilityDAFNet
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DAFNetBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 92 | 86.7 | 94.3 | 91 | |
| TPBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 91.4 | 68.6 | 39 | 66.3 | |
| MEMITBackbone=GPT-J (6B), # Editing=10002024.05 | 82.3 | 36.4 | 30.7 | 49.8 | |
| MENDBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 81.1 | 67.2 | 77.1 | 75.1 | |
| TPBackbone=LLAMA2 (7B), # Editing=10002024.05 | 64.7 | 32.5 | 11.6 | 36.3 | |
| DAFNetBackbone=GPT-J (6B), # Editing=10002024.05 | 53.1 | 38.1 | 82.3 | 57.8 | |
| MALMENBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 52.4 | 42.3 | 36.6 | 43.8 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=10002024.05 | 50.4 | 35.8 | 76.9 | 54.4 | |
| TPBackbone=GPT-J (6B), # Editing=10002024.05 | 47.3 | 17 | 1.4 | 21.9 | |
| MEMITBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 45.4 | 29.3 | 92.9 | 55.9 | |
| GRACEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 44.6 | 28.5 | 93.4 | 55.5 | |
| ROMEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 41.1 | 21.8 | 91.8 | 51.6 | |
| FTBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 34.1 | 25.4 | 49.8 | 36.4 | |
| MALMENBackbone=LLAMA2 (7B), # Editing=10002024.05 | 15.8 | 16.4 | 22.5 | 18.3 | |
| MALMENBackbone=GPT-J (6B), # Editing=10002024.05 | 15 | 12.4 | 25.1 | 17.5 | |
| FTBackbone=GPT-J (6B), # Editing=10002024.05 | 12.9 | 5.1 | 1.1 | 6.4 | |
| KNBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 12.3 | 9.2 | 67.9 | 29.8 | |
| KEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 8 | 2.9 | 90.3 | 33.7 | |
| FTBackbone=LLAMA2 (7B), # Editing=10002024.05 | 1.5 | 0.1 | 1.7 | 1.1 | |
| GRACEBackbone=GPT-J (6B), # Editing=10002024.05 | 0.3 | 0.4 | 0.1 | 0.3 | |
| ROMEBackbone=GPT-J (6B), # Editing=10002024.05 | 0.2 | 0.2 | 0 | 0.1 | |
| ROMEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0.2 | 0.1 | 0.1 | 0.1 | |
| KNBackbone=GPT-J (6B), # Editing=10002024.05 | 0.1 | 0.4 | 1 | 0.5 | |
| MEMITBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0.1 | 0.1 | 1 | 0.4 | |
| GRACEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0.1 | 0.2 | 0.1 | 0.1 | |
| KEBackbone=GPT-J (6B), # Editing=10002024.05 | 0 | 0 | 0.1 | 0 | |
| MENDBackbone=GPT-J (6B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| KNBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| KEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0.3 | 0.1 | |
| MENDBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 |