Lifelong Free-text Knowledge Editing on MRLF-Bench
71.19BLEUEVOEDIT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EVOEDITBase Model=LLaMA-2, Number of edits (T)=1002025.12 | 71.19 | 3.82 | |
| EVOEDITBase Model=LLaMA-2, Number of edits (T)=5002025.12 | 70.96 | 3.65 | |
| EVOEDITBase Model=LLaMA-2, Number of edits (T)=10002025.12 | 70.24 | 3.7 | |
| EVOEDITBase Model=LLaMA-2, Number of edits (T)=15002025.12 | 69.91 | 3.75 | |
| EVOEDITBase Model=LLaMA-2, Number of edits (T)=20002025.12 | 69.9 | 3.75 | |
| Fine-TuningBase Model=LLaMA-2, Number of edits (T)=5002025.12 | 66.82 | 5.26 | |
| Fine-TuningBase Model=LLaMA-2, Number of edits (T)=1002025.12 | 66.64 | 5.9 | |
| EVOEDITBase Model=LLaMA-3, Number of edits (T)=5002025.12 | 65.61 | 6.23 | |
| Fine-TuningBase Model=LLaMA-2, Number of edits (T)=10002025.12 | 65.28 | 5.42 | |
| EVOEDITBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 64.92 | 6.56 | |
| EVOEDITBase Model=LLaMA-3, Number of edits (T)=1002025.12 | 64.86 | 7.22 | |
| EVOEDITBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 64.85 | 6.52 | |
| EVOEDITBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 64.84 | 6.61 | |
| Fine-TuningBase Model=LLaMA-2, Number of edits (T)=15002025.12 | 64.48 | 5.55 | |
| Fine-TuningBase Model=LLaMA-2, Number of edits (T)=20002025.12 | 64.11 | 5.59 | |
| EVOEDITBase Model=GPT-J, Number of edits (T)=1002025.12 | 62.79 | 5.78 | |
| EVOEDITBase Model=GPT-J, Number of edits (T)=5002025.12 | 62.03 | 6.14 | |
| EVOEDITBase Model=GPT-J, Number of edits (T)=10002025.12 | 61.44 | 6.15 | |
| EVOEDITBase Model=GPT-J, Number of edits (T)=15002025.12 | 61.1 | 6.23 | |
| EVOEDITBase Model=GPT-J, Number of edits (T)=20002025.12 | 60.87 | 6.22 | |
| Fine-TuningBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 60.42 | 30.33 | |
| Fine-TuningBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 60.27 | 31.54 | |
| Fine-TuningBase Model=LLaMA-3, Number of edits (T)=5002025.12 | 60.24 | 34.09 | |
| Fine-TuningBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 60.14 | 32.95 | |
| Fine-TuningBase Model=LLaMA-3, Number of edits (T)=1002025.12 | 59.85 | 42.7 | |
| Fine-TuningBase Model=GPT-2, Number of edits (T)=1002025.12 | 56.71 | 17.57 | |
| Fine-TuningBase Model=GPT-2, Number of edits (T)=5002025.12 | 56.62 | 17.7 | |
| Fine-TuningBase Model=GPT-2, Number of edits (T)=10002025.12 | 56.5 | 16.13 | |
| Fine-TuningBase Model=GPT-2, Number of edits (T)=15002025.12 | 56.45 | 15.6 | |
| Fine-TuningBase Model=GPT-2, Number of edits (T)=20002025.12 | 56.33 | 15.23 | |
| Fine-TuningBase Model=GPT-J, Number of edits (T)=1002025.12 | 54.76 | 13.47 | |
| Fine-TuningBase Model=GPT-J, Number of edits (T)=5002025.12 | 53.65 | 12.23 | |
| Fine-TuningBase Model=GPT-J, Number of edits (T)=10002025.12 | 53.14 | 12.34 | |
| Fine-TuningBase Model=GPT-J, Number of edits (T)=20002025.12 | 53.11 | 12.47 | |
| Fine-TuningBase Model=GPT-J, Number of edits (T)=15002025.12 | 53.08 | 12.49 | |
| EVOEDITBase Model=GPT-2, Number of edits (T)=1002025.12 | 49.59 | 12.28 | |
| EVOEDITBase Model=GPT-2, Number of edits (T)=5002025.12 | 48.13 | 12.09 | |
| EVOEDITBase Model=GPT-2, Number of edits (T)=15002025.12 | 47.53 | 11.88 | |
| EVOEDITBase Model=GPT-2, Number of edits (T)=20002025.12 | 47.53 | 12.03 | |
| EVOEDITBase Model=GPT-2, Number of edits (T)=10002025.12 | 47.51 | 11.81 | |
| AlphaEditBase Model=LLaMA-3, Number of edits (T)=5002025.12 | 39.53 | 31.94 | |
| AlphaEditBase Model=LLaMA-3, Number of edits (T)=1002025.12 | 39.49 | 30.92 | |
| Pre-EditingBase Model=LLaMA-3, Number of edits (T)=5002025.12 | 37.89 | 25.5 | |
| Pre-EditingBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 37.55 | 34.48 | |
| Pre-EditingBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 37.5 | 24.05 | |
| Pre-EditingBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 37.43 | 24.32 | |
| MEMITBase Model=LLaMA-3, Number of edits (T)=1002025.12 | 36.36 | 40.77 | |
| Pre-EditingBase Model=LLaMA-3, Number of edits (T)=1002025.12 | 36.2 | 40.77 | |
| Pre-EditingBase Model=LLaMA-2, Number of edits (T)=1002025.12 | 35.94 | 16.65 | |
| Pre-EditingBase Model=LLaMA-2, Number of edits (T)=5002025.12 | 35.94 | 14.82 | |
| Pre-EditingBase Model=LLaMA-2, Number of edits (T)=10002025.12 | 35.67 | 14.31 | |
| Pre-EditingBase Model=LLaMA-2, Number of edits (T)=20002025.12 | 35.56 | 14.4 | |
| Pre-EditingBase Model=LLaMA-2, Number of edits (T)=15002025.12 | 35.43 | 14.37 | |
| MEMITBase Model=LLaMA-3, Number of edits (T)=5002025.12 | 34.37 | 71.17 | |
| MEMITBase Model=GPT-J, Number of edits (T)=1002025.12 | 33.73 | 26.62 | |
| MEMITBase Model=GPT-J, Number of edits (T)=5002025.12 | 33.22 | 28.12 | |
| AlphaEditBase Model=GPT-J, Number of edits (T)=1002025.12 | 31.69 | 28.08 | |
| MEMITBase Model=GPT-J, Number of edits (T)=10002025.12 | 31.01 | 34.63 | |
| AlphaEditBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 30.59 | 123.36 | |
| Pre-EditingBase Model=GPT-J, Number of edits (T)=1002025.12 | 29.92 | 32.88 | |
| Pre-EditingBase Model=GPT-J, Number of edits (T)=5002025.12 | 29.91 | 30.64 | |
| Pre-EditingBase Model=GPT-J, Number of edits (T)=10002025.12 | 29.63 | 29.22 | |
| Pre-EditingBase Model=GPT-J, Number of edits (T)=15002025.12 | 29.54 | 28.84 | |
| Pre-EditingBase Model=GPT-J, Number of edits (T)=20002025.12 | 29.5 | 28.63 | |
| MENDBase Model=LLaMA-2, Number of edits (T)=15002025.12 | 29.05 | 54.06 | |
| MENDBase Model=LLaMA-2, Number of edits (T)=20002025.12 | 28.99 | 41.46 | |
| MENDBase Model=LLaMA-2, Number of edits (T)=10002025.12 | 28.82 | 45.91 | |
| MEMITBase Model=GPT-J, Number of edits (T)=15002025.12 | 27.93 | 55.02 | |
| MENDBase Model=LLaMA-2, Number of edits (T)=5002025.12 | 27.17 | 64.56 | |
| AlphaEditBase Model=GPT-J, Number of edits (T)=5002025.12 | 27.13 | 60.1 | |
| MEMITBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 25.04 | 641.18 | |
| MEMITBase Model=GPT-J, Number of edits (T)=20002025.12 | 24.65 | 107.25 | |
| AlphaEditBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 23.59 | 545.8 | |
| AlphaEditBase Model=GPT-2, Number of edits (T)=1002025.12 | 22.52 | 71.33 | |
| MEMITBase Model=GPT-2, Number of edits (T)=1002025.12 | 22.44 | 63.54 | |
| AlphaEditBase Model=LLaMA-2, Number of edits (T)=1002025.12 | 22.4 | 8.49 | |
| MEMITBase Model=GPT-2, Number of edits (T)=5002025.12 | 21.38 | 75.86 | |
| Pre-EditingBase Model=GPT-2, Number of edits (T)=1002025.12 | 20.4 | 64 | |
| MEMITBase Model=GPT-2, Number of edits (T)=10002025.12 | 20.08 | 92.96 | |
| MENDBase Model=GPT-2, Number of edits (T)=20002025.12 | 19.78 | 62.97 | |
| MENDBase Model=GPT-2, Number of edits (T)=15002025.12 | 19.75 | 63.63 | |
| Pre-EditingBase Model=GPT-2, Number of edits (T)=10002025.12 | 19.71 | 60.53 | |
| Pre-EditingBase Model=GPT-2, Number of edits (T)=5002025.12 | 19.69 | 63.72 | |
| Pre-EditingBase Model=GPT-2, Number of edits (T)=15002025.12 | 19.66 | 59.89 | |
| MENDBase Model=GPT-2, Number of edits (T)=10002025.12 | 19.66 | 65.46 | |
| Pre-EditingBase Model=GPT-2, Number of edits (T)=20002025.12 | 19.64 | 60.07 | |
| AlphaEditBase Model=GPT-J, Number of edits (T)=10002025.12 | 19.59 | 361.11 | |
| MENDBase Model=GPT-2, Number of edits (T)=5002025.12 | 19.41 | 93.38 | |
| AlphaEditBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 19.09 | 1,738.88 | |
| MENDBase Model=GPT-2, Number of edits (T)=1002025.12 | 19.09 | 107.82 | |
| MEMITBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 18.83 | 3,299.03 | |
| MEMITBase Model=GPT-2, Number of edits (T)=15002025.12 | 18.5 | 141.95 | |
| AlphaEditBase Model=GPT-2, Number of edits (T)=5002025.12 | 17.94 | 177.79 | |
| MENDBase Model=LLaMA-2, Number of edits (T)=1002025.12 | 17.74 | 247.49 | |
| MENDBase Model=LLaMA-3, Number of edits (T)=10002025.12 | 17.14 | 2,577.66 | |
| MEMITBase Model=GPT-2, Number of edits (T)=20002025.12 | 16.99 | 215.53 | |
| MENDBase Model=LLaMA-3, Number of edits (T)=15002025.12 | 16.05 | 3,372.93 | |
| AlphaEditBase Model=GPT-J, Number of edits (T)=15002025.12 | 15.47 | 1,192.35 | |
| MENDBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 15.14 | 4,106.28 | |
| MEMITBase Model=LLaMA-3, Number of edits (T)=20002025.12 | 15.08 | 7,909.69 |