Model Editing on zsRE (Efficacy, Generalization, Specificity)
99.79EfficacyAlphaEdit
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AlphaEditModel=GPT-J-6B2025.10 | 99.79 | 96 | 28.29 | |
| HORSEBase Model=GPT-J, Number of Edits=80002026.01 | 98.91 | 88.43 | 29.37 | |
| MALMENBase Model=GPT-J, Number of Edits=80002026.01 | 98.75 | 88.52 | 24.49 | |
| EvoEditModel=GPT-J-6B2025.10 | 98.69 | 96.46 | 28.62 | |
| MEMITModel=GPT-J-6B2025.10 | 94.91 | 90.22 | 30.39 | |
| AlphaEditModel=GPT2-XL2025.10 | 94.81 | 86.11 | 25.88 | |
| EvoEditModel=GPT2-XL2025.10 | 94.51 | 87.7 | 25.89 | |
| MALMENBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 93.02 | 85.26 | 45.85 | |
| HORSEBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 92.32 | 87.28 | 53.28 | |
| RLEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 91.34 | 89.68 | 41.94 | |
| HORSEBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 90.25 | 86.27 | 60.24 | |
| MALMENBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 90.14 | 82.86 | 45.82 | |
| UltraEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 90.07 | 87.36 | 49.51 | |
| UltraEditBackbone=LLaMA-3-8B-Instruct2025.05 | 90.07 | 87.36 | 49.51 | |
| UltraEdit*Backbone=LLaMA-3-8B-Instruct, Edit Scenario=ultra-large-scale2025.05 | 87.8 | 85.48 | 46.74 | |
| UltraEdit*Backbone=LLaMA-3-8B-Instruct2025.05 | 87.8 | 85.48 | 46.74 | |
| MEMITBase Model=GPT-J, Number of Edits=80002026.01 | 86.17 | 67.22 | 25.95 | |
| UltraEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 85.3 | 80.8 | 47.38 | |
| UltraEditBackbone=Mistral-7B-v0.32025.05 | 85.3 | 80.8 | 47.38 | |
| UltraEdit*Backbone=Mistral-7B-v0.3, Edit Scenario=ultra-large-scale2025.05 | 81.13 | 76.78 | 48.06 | |
| UltraEdit*Backbone=Mistral-7B-v0.32025.05 | 81.13 | 76.78 | 48.06 | |
| MEMITModel=GPT2-XL2025.10 | 79.17 | 71.44 | 26.42 | |
| UltraEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 78.03 | 72.42 | 27.05 | |
| UltraEditBackbone=GPT-J2025.05 | 78.03 | 72.42 | 27.05 | |
| EMMETBase Model=GPT-J, Number of Edits=80002026.01 | 76.35 | 63.48 | 26.25 | |
| AlphaEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 74.34 | 67.85 | 22.94 | |
| RLEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 73.34 | 68.93 | 22 | |
| UltraEdit*Backbone=GPT-J, Edit Scenario=ultra-large-scale2025.05 | 72.95 | 68.68 | 25.91 | |
| UltraEdit*Backbone=GPT-J2025.05 | 72.95 | 68.68 | 25.91 | |
| RLEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 72.57 | 68.87 | 23.17 | |
| MEMITBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 62.21 | 55.39 | 26.16 | |
| MEMITBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 59.56 | 54.06 | 26.99 | |
| ROMEModel=GPT-J-6B2025.10 | 56.42 | 54.65 | 9.86 | |
| LoRABase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 53.8 | 47.48 | 38.59 | |
| LoRABase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 50.58 | 48.05 | 44.77 | |
| AlphaEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 50.23 | 43.31 | 12.54 | |
| ROMEModel=GPT2-XL2025.10 | 47.5 | 43.56 | 14.27 | |
| EMMETBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 41.05 | 38.14 | 18.64 | |
| WISEBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 40.94 | 40.27 | 37.4 | |
| FTBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 35.36 | 34.89 | 46.61 | |
| EMMETBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 34.85 | 32.65 | 17.22 | |
| LoRABase Model=GPT-J, Number of Edits=80002026.01 | 34.28 | 32.91 | 27.01 | |
| WISEBackbone=GPT-J, Edit Scenario=standard2025.05 | 34.13 | 33.14 | 26.81 | |
| WISEBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 34.01 | 32.61 | 46.05 | |
| FTBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 32.74 | 32.4 | 44.45 | |
| FTBase Model=GPT-J, Number of Edits=80002026.01 | 23.4 | 21.93 | 16.42 | |
| PMETBase Model=GPT-J, Number of Edits=80002026.01 | 22.64 | 21.16 | 24.04 | |
| FTBackbone=GPT-J, Edit Scenario=standard2025.05 | 15.11 | 13.55 | 2.61 | |
| FTBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 13.69 | 12.43 | 19.87 | |
| FTBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 12.24 | 10.97 | 9.06 | |
| PMETBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 3.17 | 2.68 | 0.07 | |
| MENDBase Model=LLaMA-2-7B-hf, Number of Edits=80002026.01 | 2.56 | 2.56 | 3.46 | |
| MENDBackbone=GPT-J2025.05 | 2.52 | 2.55 | 0.19 | |
| MENDBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 1.92 | 1.92 | 1.89 | |
| MALMENBackbone=LLaMA-3-8B-Instruct2025.05 | 0.2 | 0.12 | 0.09 | |
| MENDBase Model=GPT-J, Number of Edits=80002026.01 | 0.16 | 0.18 | 0.02 | |
| MEMITBackbone=LLaMA-3-8B-Instruct2025.05 | 0.14 | 0.14 | 1.4 | |
| RECTBackbone=GPT-J2025.05 | 0.03 | 0.03 | 0.12 | |
| PMETBase Model=Mistral-7B-v0.1, Number of Edits=80002026.01 | 0.02 | 0.03 | 0.12 | |
| MALMENBackbone=GPT-J2025.05 | 0.02 | 0.01 | 0.02 | |
| AlphaEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 0 | 0 | 0 | |
| MEMITBackbone=GPT-J2025.05 | 0 | 0 | 0 | |
| PRUNEBackbone=GPT-J2025.05 | 0 | 0 | 0.01 | |
| MENDBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | |
| MEMITBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | |
| MALMENBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | |
| RECTBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | |
| PRUNEBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | |
| MENDBackbone=LLaMA-3-8B-Instruct2025.05 | 0 | 0 | 0 | |
| RECTBackbone=LLaMA-3-8B-Instruct2025.05 | 0 | 0 | 0 | |
| PRUNEBackbone=LLaMA-3-8B-Instruct2025.05 | 0 | 0 | 0.24 |