Sequential Model Editing on ZSRE (test)
99.6ReliabilityDAFNet
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DAFNetBackbone=GPT-J (6B), # Editing=102024.05 | 99.6 | 97.6 | 94.8 | 97.3 | |
| MALMENBackbone=GPT-J (6B), # Editing=102024.05 | 99.1 | 95.3 | 92.8 | 95.8 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=12024.05 | 97.5 | 97.4 | 94.9 | 96.6 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=102024.05 | 97.2 | 92 | 93.3 | 94.1 | |
| DAFNetBackbone=GPT-J (6B), # Editing=1002024.05 | 89.5 | 76.5 | 90.2 | 85.4 | |
| TPBackbone=GPT-J (6B), # Editing=102024.05 | 85.2 | 78.3 | 77.2 | 80.2 | |
| DAFNetBackbone=LLAMA2 (7B), # Editing=1002024.05 | 84.7 | 72 | 93.6 | 83.5 | |
| MEMITBackbone=GPT-J (6B), # Editing=102024.05 | 82.1 | 76 | 94.7 | 84.2 | |
| GRACEBackbone=GPT-J (6B), # Editing=102024.05 | 81.8 | 78.4 | 94.5 | 84.9 | |
| ROMEBackbone=GPT-J (6B), # Editing=102024.05 | 81.1 | 78.8 | 94.6 | 84.8 | |
| FTBackbone=GPT-J (6B), # Editing=102024.05 | 10.3 | 10.8 | 0.3 | 7.1 | |
| KNBackbone=GPT-J (6B), # Editing=102024.05 | 1 | 1.1 | 1.9 | 1.3 | |
| MENDBackbone=GPT-J (6B), # Editing=102024.05 | 0.4 | 0.4 | 0.5 | 0.4 | |
| KEBackbone=GPT-J (6B), # Editing=102024.05 | 0 | 0 | 0.7 | 0.3 |