Model Editing on RIPE
99.76ReliabilityLTE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LTEBackbone=GPT-J2026.04 | 99.76 | 87.48 | 82.96 | 90.07 | |
| LTEBackbone=LLaMA-3 (8B)2026.04 | 99.62 | 86.29 | 83.5 | 89.8 | |
| LightEditBackbone=GPT-J2026.04 | 99.42 | 96.61 | 99.88 | 98.64 | |
| AlphaEditBackbone=GPT-J2026.04 | 99.3 | 40.9 | 79 | 73.07 | |
| LightEditBackbone=LLaMA-3 (8B)2026.04 | 98.48 | 96.25 | 99.81 | 98.18 | |
| AlphaEditBackbone=LLaMA-3 (8B)2026.04 | 98.25 | 48.2 | 28.6 | 58.35 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 97.8 | 66.4 | 72.2 | 78.8 | |
| TPBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 77 | 55.1 | 51.3 | 61.1 | |
| MENDBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 66.4 | 29.4 | 29.7 | 41.8 | |
| FTBackbone=GPT-J2026.04 | 64.43 | 58.55 | 20.77 | 47.92 | |
| MEMITBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 58.4 | 29.6 | 38.7 | 42.2 | |
| RECIPEBackbone=GPT-J2026.04 | 58.13 | 48.34 | 39.83 | 48.77 | |
| RECIPEBackbone=LLaMA-3 (8B)2026.04 | 56.83 | 50.53 | 56.84 | 54.73 | |
| GRACEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 56.7 | 30.6 | 41.1 | 42.8 | |
| MEMITBackbone=GPT-J2026.04 | 52.29 | 34.18 | 41.07 | 42.51 | |
| MALMENBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 51.5 | 33.8 | 20.5 | 35.3 | |
| FTBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 49.2 | 33.7 | 71 | 51.3 | |
| ROMEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 48.3 | 27.1 | 42.5 | 39.3 | |
| DAFNetBackbone=GPT-J (6B), # Editing=10002024.05 | 48.3 | 3,130 | 5,730 | 45.6 | |
| TPBackbone=GPT-J (6B), # Editing=10002024.05 | 48.1 | 2,910 | 1,520 | 30.8 | |
| ROMEBackbone=GPT-J (6B), # Editing=10002024.05 | 47.5 | 1,690 | 1,340 | 26 | |
| GRACEBackbone=GPT-J (6B), # Editing=10002024.05 | 46.7 | 1,630 | 1,380 | 25.6 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=10002024.05 | 44.1 | 3,430 | 8,580 | 54.7 | |
| TPBackbone=LLAMA2 (7B), # Editing=10002024.05 | 42.3 | 2,680 | 990 | 26.3 | |
| MALMENBackbone=LLAMA2 (7B), # Editing=10002024.05 | 42.3 | 3,840 | 3,850 | 39.8 | |
| GRACEBackbone=LLaMA-3 (8B)2026.04 | 37 | 27.21 | 100 | 54.74 | |
| MALMENBackbone=GPT-J (6B), # Editing=10002024.05 | 31.1 | 1,910 | 3,530 | 28.5 | |
| GRACEBackbone=GPT-J2026.04 | 31.08 | 23.72 | 100 | 51.6 | |
| BASEBackbone=LLaMA-3 (8B)2026.04 | 29.6 | 27.06 | 100 | 52.22 | |
| BASEBackbone=GPT-J2026.04 | 24.11 | 23.48 | 100 | 49.2 | |
| KNBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 21.8 | 15.4 | 55.4 | 30.9 | |
| KEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 9.9 | 4.4 | 42.8 | 19 | |
| FTBackbone=LLaMA-3 (8B)2026.04 | 3.52 | 2.35 | 1.52 | 2.46 | |
| FTBackbone=GPT-J (6B), # Editing=10002024.05 | 3.1 | 90 | 80 | 1.6 | |
| FTBackbone=LLAMA2 (7B), # Editing=10002024.05 | 2.7 | 100 | 220 | 2 | |
| R-ROMEBackbone=LLaMA-3 (8B)2026.04 | 1.62 | 0.85 | 0.05 | 0.84 | |
| R-ROMEBackbone=GPT-J2026.04 | 0.63 | 0.58 | 0.25 | 0.49 | |
| ROMEBackbone=LLaMA-3 (8B)2026.04 | 0.58 | 0.31 | 0 | 0.3 | |
| ROMEBackbone=GPT-J2026.04 | 0.54 | 0.31 | 0.54 | 0.46 | |
| MENDBackbone=GPT-J (6B), # Editing=10002024.05 | 0.2 | 10 | 10 | 0.1 | |
| KNBackbone=GPT-J (6B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| MEMITBackbone=GPT-J (6B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| KEBackbone=GPT-J (6B), # Editing=10002024.05 | 0 | 0 | 20 | 0.1 | |
| KNBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| ROMEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| MEMITBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| GRACEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| KEBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| MENDBackbone=LLAMA2 (7B), # Editing=10002024.05 | 0 | 0 | 0 | 0 | |
| MEMITBackbone=LLaMA-3 (8B)2026.04 | 0 | 0 | 0.79 | 0.26 |