Model Editing on ZSRE (Reliability, Generality, Locality)
0.975ReliabilityDAFNet
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DAFNetBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 0.975 | 0.974 | 0.949 | 0.966 | |
| TPBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.864 | 0.84 | 0.864 | 0.856 | |
| CrispEditData=ZsRE 10K2026.02 | 0.774 | 0.687 | — | — | |
| MENDBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 0.738 | 0.703 | 0.661 | 0.701 | |
| MALMENBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 0.664 | 0.678 | 0.437 | 0.593 | |
| ROMEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.535 | 0.516 | 0.94 | 0.664 | |
| LocBF-FTData=ZsRE 10K2026.02 | 0.535 | 0.477 | — | — | |
| FTBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.528 | 0.532 | 0.921 | 0.66 | |
| GRACEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.523 | 0.508 | 0.957 | 0.663 | |
| MEMITBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.497 | 0.494 | 0.919 | 0.637 | |
| KNBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=false2024.05 | 0.202 | 0.208 | 0.524 | 0.311 | |
| UltraEditData=ZsRE 10K2026.02 | 0.201 | 0.167 | — | — | |
| KEBackbone=LLAMA2 (7B), # Editing=1, Augmented with DAFSet=true2024.05 | 0.129 | 0.086 | 0.906 | 0.374 | |
| LLaMA-3-8B-InstructData=ZsRE 10K2026.02 | 0.02 | 0.015 | — | — | |
| Adam-NSCLData=ZsRE 10K2026.02 | 0.012 | 0.011 | — | — | |
| AlphaEditData=ZsRE 10K2026.02 | 0.003 | 0.002 | — | — |