Lifelong Model Editing on ZsRE
91.09EfficacyRLEDIT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RLEDITBackbone=Llama-3-8B-Instruct2026.05 | 91.09 | 89.62 | 41.43 | — | |
| RLEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 91.09 | 89.62 | 41.43 | — | |
| STABLEEDITBackbone=Llama-3-8B-Instruct2026.05 | 90.61 | 88.38 | 48.23 | — | |
| STABLEEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 90.61 | 88.38 | 48.23 | — | |
| ULTRAEDITBackbone=Llama-3-8B-Instruct2026.05 | 90.07 | 87.36 | 49.51 | — | |
| ULTRAEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 90.07 | 87.36 | 49.51 | — | |
| STABLEEDITBackbone=Mistral-7B-v0.32026.05 | 87.39 | 82.93 | 50.26 | — | |
| STABLEEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 87.39 | 82.93 | 50.26 | — | |
| ALPHAEDITBackbone=Llama-3-8B-Instruct2026.05 | 86.1 | 81.15 | 26.26 | — | |
| ALPHAEDITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 86.1 | 81.15 | 26.26 | — | |
| ULTRAEDITBackbone=Mistral-7B-v0.32026.05 | 85.3 | 80.8 | 47.38 | — | |
| ULTRAEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 85.3 | 80.8 | 47.38 | — | |
| RLEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 84.7 | 82.79 | 38.26 | — | |
| HiEditrandBase Model=GEMMA-2-9B2026.04 | 82.65 | 78.98 | 31.95 | 67.89 | |
| HiEditfullBase Model=GEMMA-2-9B2026.04 | 82.12 | 78.4 | 32.4 | 68.73 | |
| STABLEEDITBackbone=GPT-J-6B2026.05 | 82.12 | 78.39 | 31.52 | — | |
| STABLEEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 82.12 | 78.39 | 31.52 | — | |
| HiEditfullBase Model=LLAMA-3-8B2026.04 | 82.1 | 79.99 | 48.42 | 75.16 | |
| ULTRAEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 82.03 | 77.08 | 45.51 | — | |
| HiEditrandBase Model=LLAMA-3-8B2026.04 | 81.95 | 79.63 | 47.97 | 74.66 | |
| RLEditBase Model=LLAMA-3-8B2026.04 | 81.43 | 79.49 | 42.73 | 70.72 | |
| STABLEEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 80.4 | 76.65 | 47.59 | — | |
| ULTRAEDITBackbone=GPT-J-6B2026.05 | 78.03 | 72.42 | 27.05 | — | |
| ULTRAEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 78.03 | 72.42 | 27.05 | — | |
| RLEDITBackbone=GPT-J-6B2026.05 | 72.65 | 68.45 | 21.79 | — | |
| RLEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 72.65 | 68.45 | 21.79 | — | |
| RLEDITBackbone=Mistral-7B-v0.32026.05 | 71.62 | 67.6 | 30.7 | — | |
| RLEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 71.62 | 67.6 | 30.7 | — | |
| RLEditBase Model=GEMMA-2-9B2026.04 | 69.11 | 65.87 | 25.17 | 57.66 | |
| ALPHAEDITBackbone=GPT-J-6B2026.05 | 50.23 | 43.31 | 12.54 | — | |
| ALPHAEDITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 50.23 | 43.31 | 12.54 | — | |
| Pre-editedBackbone=Mistral-7B-v0.32026.05 | 44.46 | 43.55 | 48.08 | — | |
| Pre-editedBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 44.46 | 43.55 | 48.08 | — | |
| Pre-editedBackbone=Llama-3-8B-Instruct2026.05 | 36.76 | 35.83 | 38.93 | — | |
| Pre-editedBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 36.76 | 35.83 | 38.93 | — | |
| Pre-editedBase Model=LLAMA-3-8B2026.04 | 36.58 | 35.89 | 38.64 | 38.26 | |
| FTBase Model=GEMMA-2-9B2026.04 | 35.38 | 32.31 | 30.46 | 33.4 | |
| Pre-editedBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 34.32 | 33.39 | 38.06 | — | |
| Pre-editedBase Model=GEMMA-2-9B2026.04 | 32.75 | 31.87 | 39.46 | 32.36 | |
| Pre-editedBackbone=GPT-J-6B2026.05 | 27.22 | 26.42 | 27.33 | — | |
| Pre-editedBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 27.22 | 26.42 | 27.33 | — | |
| DAFNetBase Model=GEMMA-2-9B2026.04 | 23.02 | 22.17 | 28.32 | 23.15 | |
| DAFNetBase Model=LLAMA-3-8B2026.04 | 21.15 | 20.41 | 21.15 | 21.13 | |
| MENDBase Model=GEMMA-2-9B2026.04 | 20.78 | 20.29 | 12.34 | 21.11 | |
| FTBackbone=Llama-3-8B-Instruct2026.05 | 18.51 | 18.11 | 5.01 | — | |
| FTBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 18.51 | 18.11 | 5.01 | — | |
| ALPHAEDITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 16.32 | 13.96 | 1.66 | — | |
| AlphaEditBase Model=LLAMA-3-8B2026.04 | 16.22 | 14.58 | 3.87 | 9.1 | |
| AlphaEditBase Model=GEMMA-2-9B2026.04 | 15.79 | 15.32 | 20.21 | 13.19 | |
| FTBackbone=GPT-J-6B2026.05 | 15.11 | 13.55 | 2.61 | — | |
| FTBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 15.11 | 13.55 | 2.61 | — | |
| MENDBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 15 | 14.41 | 0.47 | — | |
| FTBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 14.02 | 10.91 | 3.39 | — | |
| FTBackbone=Mistral-7B-v0.32026.05 | 13.69 | 12.43 | 19.87 | — | |
| FTBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 13.69 | 12.43 | 19.87 | — | |
| RECTBase Model=GEMMA-2-9B2026.04 | 11.26 | 11.25 | 16.19 | 9.62 | |
| MEMITBase Model=GEMMA-2-9B2026.04 | 10.54 | 10.4 | 14.66 | 9.22 | |
| PRUNEBase Model=GEMMA-2-9B2026.04 | 10.46 | 10.5 | 14.66 | 9.46 | |
| MALMENBase Model=GEMMA-2-9B2026.04 | 9.3 | 8.87 | 12.9 | 11.22 | |
| FTBase Model=LLAMA-3-8B2026.04 | 9.16 | 8.11 | 1.95 | 8.82 | |
| MALMENBase Model=LLAMA-3-8B2026.04 | 5.09 | 4.74 | 0.72 | 4.26 | |
| MALMENBackbone=Llama-3-8B-Instruct2026.05 | 3.86 | 3.07 | 0.26 | — | |
| MALMENBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 3.86 | 3.07 | 0.26 | — | |
| MENDBackbone=Mistral-7B-v0.32026.05 | 3.33 | 3.08 | 0.2 | — | |
| MENDBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 3.33 | 3.08 | 0.2 | — | |
| ROMEBase Model=LLAMA-3-8B2026.04 | 3.29 | 3.24 | 0.66 | 2.67 | |
| MENDBackbone=GPT-J-6B2026.05 | 2.52 | 2.55 | 0.19 | — | |
| MENDBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 2.52 | 2.55 | 0.19 | — | |
| MALMENBackbone=Mistral-7B-v0.32026.05 | 1.15 | 1.06 | 0.02 | — | |
| MALMENBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 1.15 | 1.06 | 0.02 | — | |
| MEMITBackbone=Llama-3-8B-Instruct2026.05 | 0.14 | 0.14 | 1.4 | — | |
| MEMITBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.14 | 0.14 | 1.4 | — | |
| MALMENBackbone=GPT-J-6B2026.05 | 0.02 | 0.01 | 0.02 | — | |
| MALMENBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.02 | 0.01 | 0.02 | — | |
| MEMITBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0.02 | 0.02 | 0.17 | — | |
| MEMITBase Model=LLAMA-3-8B2026.04 | 0 | 0 | 0.12 | 0 | |
| PRUNEBase Model=LLAMA-3-8B2026.04 | 0 | 0 | 0 | 0 | |
| RECTBase Model=LLAMA-3-8B2026.04 | 0 | 0 | 0 | 0 | |
| MENDBase Model=LLAMA-3-8B2026.04 | 0 | 0 | 0 | 0.02 | |
| ROMEBase Model=GEMMA-2-9B2026.04 | 0 | 0 | 0 | 0 | |
| MEMITBackbone=Mistral-7B-v0.32026.05 | 0 | 0 | 0 | — | |
| ALPHAEDITBackbone=Mistral-7B-v0.32026.05 | 0 | 0 | 0 | — | |
| MENDBackbone=Llama-3-8B-Instruct2026.05 | 0 | 0 | 0 | — | |
| MEMITBackbone=GPT-J-6B2026.05 | 0 | 0 | 0 | — | |
| MEMITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | — | |
| ALPHAEDITBackbone=Mistral-7B-v0.3, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | — | |
| MENDBackbone=Llama-3-8B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | — | |
| MEMITBackbone=GPT-J-6B, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | — | |
| MALMENBackbone=Qwen2.5-7B-Instruct, Total Edits=20K&17K, Batch Size (nt)=1002026.05 | 0 | 0 | 0 | — |