Lifelong Model Editing on WikiBigEdit
81.22EfficacySTABLEEDIT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| STABLEEDITBackbone=Llama-3-8B-Instruct2026.05 | 81.22 | 75.98 | 45.22 | — | — | |
| STABLEEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 81.22 | 75.98 | 45.22 | 70.7 | 35.03 | |
| ULTRAEDITBackbone=Llama-3-8B-Instruct2026.05 | 79.6 | 73.49 | 48.51 | — | — | |
| ULTRAEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 79.6 | 73.49 | 48.51 | 66.55 | 35.64 | |
| STABLEEDITBackbone=Mistral-7B-v0.32026.05 | 78.9 | 72.57 | 47.36 | — | — | |
| STABLEEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 78.9 | 72.57 | 47.36 | 63.96 | 37.22 | |
| ULTRAEDITBackbone=Mistral-7B-v0.32026.05 | 76 | 70.15 | 46.09 | — | — | |
| ULTRAEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 76 | 70.15 | 46.09 | 62.27 | 35.8 | |
| RLEDITBackbone=Llama-3-8B-Instruct2026.05 | 75.35 | 70 | 37.21 | — | — | |
| RLEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 75.35 | 70 | 37.21 | 65.55 | 28.13 | |
| STABLEEDITBackbone=GPT-J-6B2026.05 | 75.1 | 68.13 | 44.04 | — | — | |
| STABLEEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 75.1 | 68.13 | 44.04 | 60.53 | 37.79 | |
| ULTRAEDITBackbone=GPT-J-6B2026.05 | 73.84 | 66.57 | 37.17 | — | — | |
| ULTRAEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 73.84 | 66.57 | 37.17 | 56.9 | 29.27 | |
| ULTRAEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 73.37 | 65.86 | 45.12 | 54.65 | 32.74 | |
| ALPHAEDITBackbone=GPT-J-6B2026.05 | 69.66 | 55.03 | 21.2 | — | — | |
| ALPHAEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 69.66 | 55.03 | 21.2 | 42.6 | 0.07 | |
| RLEDITBackbone=GPT-J-6B2026.05 | 69.18 | 63.01 | 32.75 | — | — | |
| RLEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 69.18 | 63.01 | 32.75 | 56.88 | 26.14 | |
| STABLEEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 68.95 | 63.52 | 46.94 | 58.66 | 36.48 | |
| ALPHAEDITBackbone=Llama-3-8B-Instruct2026.05 | 63.24 | 54.68 | 20.17 | — | — | |
| ALPHAEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 63.24 | 54.68 | 20.17 | 42.58 | 0.01 | |
| RLEDITBackbone=Mistral-7B-v0.32026.05 | 57.55 | 52.47 | 28.78 | — | — | |
| RLEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 57.55 | 52.47 | 28.78 | 49.21 | 22.41 | |
| RLEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 55.88 | 47.66 | 29.48 | 41.14 | 22.21 | |
| Pre-editedBackbone=Mistral-7B-v0.32026.05 | 39.14 | 38.21 | 41.62 | — | — | |
| Pre-editedBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 39.14 | 38.21 | 41.62 | 35.54 | 29.93 | |
| Pre-editedBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 30.97 | 30.4 | 34.5 | 28.43 | 22.09 | |
| Pre-editedBackbone=GPT-J-6B2026.05 | 29.97 | 29.08 | 32.58 | — | — | |
| Pre-editedBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 29.97 | 29.08 | 32.58 | 26.46 | 21.81 | |
| Pre-editedBackbone=Llama-3-8B-Instruct2026.05 | 24.92 | 35.46 | 38.87 | — | — | |
| Pre-editedBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 24.92 | 35.46 | 38.87 | 32.7 | 26.42 | |
| FTBackbone=GPT-J-6B2026.05 | 21.9 | 17.56 | 8.47 | — | — | |
| FTBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 21.9 | 17.56 | 8.47 | 13.91 | 9.24 | |
| ALPHAEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 20.31 | 15.49 | 2.17 | 9.01 | 0.23 | |
| FTBackbone=Mistral-7B-v0.32026.05 | 13.77 | 14.86 | 11.84 | — | — | |
| FTBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 13.77 | 14.86 | 11.84 | 11.55 | 7.51 | |
| FTBackbone=Llama-3-8B-Instruct2026.05 | 13 | 11.7 | 6.75 | — | — | |
| FTBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 13 | 11.7 | 6.75 | 11.02 | 4.38 | |
| FTBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 10.35 | 7.59 | 3.68 | 5.55 | 5.84 | |
| MEMITBackbone=GPT-J-6B2026.05 | 1.59 | 1.59 | 0.5 | — | — | |
| MEMITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 1.59 | 1.59 | 0.5 | 0.8 | 0 | |
| MEMITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.54 | 0.33 | 0.38 | 0.32 | 0 | |
| MENDBackbone=Llama-3-8B-Instruct2026.05 | 0.26 | 0.27 | 0.18 | — | — | |
| MENDBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.26 | 0.27 | 0.18 | 0.26 | 0.34 | |
| MALMENBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.06 | 0.02 | 0.02 | 0 | 0 | |
| MEMITBackbone=Llama-3-8B-Instruct2026.05 | 0.02 | 0.02 | 0.09 | — | — | |
| MENDBackbone=GPT-J-6B2026.05 | 0.02 | 0.01 | 0.02 | — | — | |
| MEMITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.02 | 0.02 | 0.09 | 0.02 | 0 | |
| MENDBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.02 | 0.01 | 0.02 | 0.02 | 0.02 | |
| MENDBackbone=Mistral-7B-v0.32026.05 | 0.01 | 0 | 0 | — | — | |
| MENDBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.01 | 0 | 0 | 0.01 | 0 | |
| MEMITBackbone=Mistral-7B-v0.32026.05 | 0 | 0 | 0 | — | — | |
| ALPHAEDITBackbone=Mistral-7B-v0.32026.05 | 0 | 0 | 0 | — | — | |
| MALMENBackbone=Mistral-7B-v0.32026.05 | 0 | 0.01 | 0 | — | — | |
| MALMENBackbone=Llama-3-8B-Instruct2026.05 | 0 | 0 | 0 | — | — | |
| MALMENBackbone=GPT-J-6B2026.05 | 0 | 0.01 | 0.01 | — | — | |
| MEMITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | 0 | 0 | |
| ALPHAEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | 0 | 0 | |
| MALMENBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0.01 | 0 | 0 | 0 | |
| MALMENBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | 0 | 0 | |
| MALMENBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0.01 | 0.01 | 0 | 0 | |
| MENDBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | 0 | 0 |