Model Editing on CounterFact (Efficacy, Generalization, Specificity)
96.12EfficacyHORSE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HORSEBase Model=GPT-J, Number of Edits=80002026.01 | 96.12 | 35.23 | 53 | |
| MEMITBase Model=GPT-J, Number of Edits=80002026.01 | 95.54 | 36.49 | 11.63 | |
| MALMENBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 94.85 | 50.54 | 63.55 | |
| HORSEBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 94.76 | 50.86 | 63.59 | |
| EMMETBase Model=GPT-J, Number of Edits=80002026.01 | 86.37 | 29.75 | 8.7 | |
| HORSEBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 83.46 | 57.83 | 72.71 | |
| EMMETBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 71.78 | 33.01 | 14.89 | |
| MEMITBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 63.64 | 34.01 | 10.63 | |
| MALMENBase Model=GPT-J, Number of Edits=80002026.01 | 62.42 | 22.34 | 39.13 | |
| EMMETBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 59.62 | 32.61 | 15.3 | |
| MEMITBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 53.17 | 34.16 | 13.77 | |
| MALMENBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 46.03 | 26.18 | 38.01 | |
| LoRABase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 32.27 | 21.95 | 41.57 | |
| LoRABase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 30.09 | 21.76 | 49.36 | |
| FTBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 17.52 | 13.98 | 20.35 | |
| PMETBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 16.01 | 12.43 | 36.69 | |
| PMETBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 15.43 | 10.98 | 26.52 | |
| FTBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 13.89 | 10.97 | 34.5 | |
| LoRABase Model=GPT-J, Number of Edits=80002026.01 | 11.93 | 5.47 | 30.11 | |
| FTBase Model=GPT-J, Number of Edits=80002026.01 | 11.82 | 6.84 | 26.79 | |
| PMETBase Model=GPT-J, Number of Edits=80002026.01 | 6.61 | 2.73 | 14.89 | |
| MENDBase Model=GPT-J, Number of Edits=80002026.01 | 0 | 0 | 0 | |
| MENDBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 0 | 0 | 0 | |
| MENDBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 0 | 0 | 0 |