Model Editing on WikiBigEdit (Efficacy, Specificity, Personas, Reasoning)
79.6EfficacyUltraEdit
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| UltraEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 79.6 | 73.49 | 48.51 | 66.55 | 35.64 | |
| UltraEditBackbone=LLaMA-3-8B-Instruct2025.05 | 79.6 | 73.49 | 48.51 | 66.55 | 35.64 | |
| UltraEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 76 | 70.15 | 46.09 | 62.27 | 35.8 | |
| UltraEditBackbone=Mistral-7B-v0.32025.05 | 76 | 70.15 | 46.09 | 62.27 | 35.8 | |
| RLEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 75.35 | 70 | 37.21 | 65.55 | 28.13 | |
| UltraEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 73.84 | 66.57 | 37.17 | 56.9 | 29.27 | |
| UltraEditBackbone=GPT-J2025.05 | 73.84 | 66.57 | 37.17 | 56.9 | 29.27 | |
| UltraEditModel=Qwen2.5-7B-Instruct2025.05 | 73.37 | 65.86 | 45.12 | 54.65 | 32.74 | |
| UltraEdit*Backbone=Mistral-7B-v0.3, Edit Scenario=ultra-large-scale2025.05 | 71.78 | 65.63 | 55.4 | 56.11 | — | |
| UltraEdit*Backbone=Mistral-7B-v0.32025.05 | 71.78 | 65.63 | 55.4 | 56.11 | — | |
| AlphaEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 69.66 | 55.03 | 21.2 | 42.6 | 0.07 | |
| UltraEdit*Backbone=LLaMA-3-8B-Instruct, Edit Scenario=ultra-large-scale2025.05 | 68.99 | 63.59 | 52.28 | 55.04 | — | |
| UltraEdit*Backbone=LLaMA-3-8B-Instruct2025.05 | 68.99 | 63.59 | 52.28 | 55.04 | — | |
| UltraEdit*Backbone=GPT-J, Edit Scenario=ultra-large-scale2025.05 | 66.46 | 60.54 | 47.9 | 51.73 | — | |
| UltraEdit*Backbone=GPT-J2025.05 | 66.46 | 60.54 | 47.9 | 51.73 | — | |
| UltraEdit*Model=Qwen2.5-7B-Instruct2025.05 | 66.34 | 60.42 | 51.74 | 50.53 | — | |
| RLEditBackbone=GPT-J, Edit Scenario=standard2025.05 | 66.18 | 60.97 | 32.2 | 55.78 | 25.94 | |
| AlphaEditBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 63.24 | 54.68 | 20.17 | 42.58 | 0.01 | |
| RLEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 57.55 | 52.47 | 28.78 | 49.21 | 22.41 | |
| WISEBackbone=GPT-J, Edit Scenario=standard2025.05 | 49.88 | 45.39 | 30 | 40.52 | 21.01 | |
| WISEBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 37.31 | 33.44 | 11.61 | 27.44 | 8.95 | |
| WISEBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 34.07 | 32.26 | 28.91 | 29.55 | 21.19 | |
| FTBackbone=GPT-J, Edit Scenario=standard2025.05 | 21.9 | 17.56 | 8.47 | 13.91 | 9.24 | |
| AlphaEditModel=Qwen2.5-7B-Instruct2025.05 | 20.31 | 15.49 | 2.17 | 9.01 | 0.23 | |
| FTBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 13.77 | 14.86 | 11.84 | 11.55 | 7.51 | |
| FTBackbone=LLaMA-3-8B-Instruct, Edit Scenario=standard2025.05 | 13 | 11.7 | 6.75 | 11.02 | 4.38 | |
| FTModel=Qwen2.5-7B-Instruct2025.05 | 10.35 | 7.59 | 3.68 | 5.55 | 5.84 | |
| RLEditModel=Qwen2.5-7B-Instruct2025.05 | 2.83 | 1.81 | 0.43 | 1.45 | 0.39 | |
| PRUNEBackbone=GPT-J2025.05 | 2.36 | 2.32 | 1.01 | 1.86 | 0 | |
| RECTModel=Qwen2.5-7B-Instruct2025.05 | 2.34 | 2.29 | 0.83 | 0.78 | 0 | |
| PRUNEModel=Qwen2.5-7B-Instruct2025.05 | 2.34 | 2.34 | 0.97 | 1.37 | 0 | |
| RECTBackbone=GPT-J2025.05 | 2.13 | 1.99 | 0.59 | 2.06 | 0 | |
| MEMITBackbone=GPT-J2025.05 | 1.59 | 1.59 | 0.5 | 0.8 | 0 | |
| MEMITModel=Qwen2.5-7B-Instruct2025.05 | 0.54 | 0.33 | 0.38 | 0.32 | 0 | |
| RECTBackbone=LLaMA-3-8B-Instruct2025.05 | 0.21 | 0.24 | 0.06 | 0.29 | 0 | |
| MALMENModel=Qwen2.5-7B-Instruct2025.05 | 0.06 | 0.02 | 0.02 | 0 | 0 | |
| MENDBackbone=GPT-J2025.05 | 0.02 | 0.01 | 0.02 | 0.02 | 0.02 | |
| MEMITBackbone=LLaMA-3-8B-Instruct2025.05 | 0.02 | 0.02 | 0.09 | 0.02 | 0 | |
| PRUNEBackbone=LLaMA-3-8B-Instruct2025.05 | 0.02 | 0.02 | 0.09 | 0.02 | 0 | |
| MENDBackbone=Mistral-7B-v0.32025.05 | 0.01 | 0 | 0 | 0.01 | 0 | |
| MENDBackbone=LLaMA-3-8B-Instruct2025.05 | 0.01 | 0.01 | 0.1 | 0.01 | 0 | |
| AlphaEditBackbone=Mistral-7B-v0.3, Edit Scenario=standard2025.05 | 0 | 0 | 0 | 0 | 0 | |
| MENDModel=Qwen2.5-7B-Instruct2025.05 | 0 | 0 | 0 | 0 | 0 | |
| MALMENBackbone=GPT-J2025.05 | 0 | 0.01 | 0.01 | 0 | 0 | |
| MEMITBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | 0 | 0 | |
| MALMENBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0.01 | 0 | 0 | |
| RECTBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | 0 | 0 | |
| PRUNEBackbone=Mistral-7B-v0.32025.05 | 0 | 0 | 0 | 0 | 0 | |
| MALMENBackbone=LLaMA-3-8B-Instruct2025.05 | 0 | 0 | 0 | 0 | 0 |