Sequential Model Editing on WikiBigEdit (500K edits)
74.54EfficacySTABLEEDIT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| STABLEEDITBackbone=Mistral-7B-v0.3, Number of samples per step (nt)=1002026.05 | 74.54 | 67.8 | 56.49 | 58.88 | |
| STABLEEDITBackbone=Llama-3-8B-Instruct, Number of samples per step (nt)=1002026.05 | 72.44 | 66.87 | 53.83 | 60.39 | |
| ULTRAEDITBackbone=Mistral-7B-v0.3, Number of samples per step (nt)=1002026.05 | 71.78 | 65.63 | 55.4 | 56.11 | |
| STABLEEDITBackbone=GPT-J-6B, Number of samples per step (nt)=1002026.05 | 71.47 | 65.04 | 52.84 | 57.2 | |
| STABLEEDITBackbone=Qwen2.5-7B-Instruct, Number of samples per step (nt)=1002026.05 | 69.2 | 62.95 | 54.09 | 54.05 | |
| ULTRAEDITBackbone=Llama-3-8B-Instruct, Number of samples per step (nt)=1002026.05 | 68.99 | 63.59 | 52.28 | 55.04 | |
| ULTRAEDITBackbone=GPT-J-6B, Number of samples per step (nt)=1002026.05 | 66.46 | 60.54 | 47.9 | 51.73 | |
| ULTRAEDITBackbone=Qwen2.5-7B-Instruct, Number of samples per step (nt)=1002026.05 | 62.73 | 57.19 | 48.52 | 49.04 | |
| Pre-editedBackbone=Mistral-7B-v0.3, Number of samples per step (nt)=1002026.05 | 39.14 | 38.21 | 41.62 | 35.54 | |
| Pre-editedBackbone=Qwen2.5-7B-Instruct, Number of samples per step (nt)=1002026.05 | 30.97 | 30.4 | 34.5 | 28.43 | |
| Pre-editedBackbone=GPT-J-6B, Number of samples per step (nt)=1002026.05 | 29.97 | 29.08 | 32.58 | 26.46 | |
| Pre-editedBackbone=Llama-3-8B-Instruct, Number of samples per step (nt)=1002026.05 | 24.92 | 35.46 | 38.87 | 32.7 |