Model Editing on WikiBigEdit
69.5MMLULLaMA-3-8B-Instruct
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaMA-3-8B-InstructEvaluation Context=QA Context, Evaluation Protocol=WILD2026.02 | 69.5 | 93 | 91 | 69.3 | 50.7 | 58 | 73.5 | — | — | — | — | |
| LLaMA-3-8B-InstructEvaluation Context=No Context, Evaluation Protocol=WILD2026.02 | 69.5 | 164 | 161 | 69.3 | 50.7 | 58 | 73.5 | — | — | — | — | |
| FTEvaluation Context=QA Context, Time=5m 12s, Evaluation Protocol=WILD2026.02 | 69.5 | 233 | 232 | 49.4 | 49.2 | 42.5 | 59 | — | — | — | — | |
| FTEvaluation Context=No Context, Time=5m 12s, Evaluation Protocol=WILD2026.02 | 69.5 | 42 | 43 | 49.4 | 49.2 | 42.5 | 59 | — | — | — | — | |
| UltraEditEvaluation Context=QA Context, Time=3m 15s, Evaluation Protocol=WILD2026.02 | 69.3 | 592 | 548 | 67.7 | 52.4 | 53.5 | 74.5 | — | — | — | — | |
| UltraEditEvaluation Context=No Context, Time=3m 15s, Evaluation Protocol=WILD2026.02 | 69.3 | 554 | 520 | 67.7 | 52.4 | 53.5 | 74.5 | — | — | — | — | |
| CrispEditEvaluation Context=QA Context, Time=6m 29s, Evaluation Protocol=WILD2026.02 | 69.3 | 770 | 702 | 70.5 | 51.8 | 55 | 74 | — | — | — | — | |
| CrispEditEvaluation Context=No Context, Time=6m 29s, Evaluation Protocol=WILD2026.02 | 69.3 | 284 | 305 | 70.5 | 51.8 | 55 | 74 | — | — | — | — | |
| Adam-NSCLEvaluation Context=QA Context, Time=30m 45s, Evaluation Protocol=WILD2026.02 | 69.2 | 136 | 136 | 45.3 | 50.2 | 42.5 | 39 | — | — | — | — | |
| Adam-NSCLEvaluation Context=No Context, Time=30m 45s, Evaluation Protocol=WILD2026.02 | 69.2 | 34 | 34 | 45.3 | 50.2 | 42.5 | 39 | — | — | — | — | |
| LocBF-FTEvaluation Context=QA Context, Time=15m 47s, Evaluation Protocol=WILD2026.02 | 69.2 | 504 | 467 | 73.2 | 52 | 55.5 | 73.5 | — | — | — | — | |
| LocBF-FTEvaluation Context=No Context, Time=15m 47s, Evaluation Protocol=WILD2026.02 | 69.2 | 167 | 157 | 73.2 | 52 | 55.5 | 73.5 | — | — | — | — | |
| CrispEdit-SeqEvaluation Context=QA Context, Time=38m 47s, Evaluation Protocol=WILD2026.02 | 69.2 | 667 | 598 | 68.8 | 50.4 | 53 | 73 | — | — | — | — | |
| CrispEdit-SeqEvaluation Context=No Context, Time=38m 47s, Evaluation Protocol=WILD2026.02 | 69.2 | 408 | 386 | 68.8 | 50.4 | 53 | 73 | — | — | — | — | |
| FT SequentialEvaluation Context=QA Context, Time=10m 13s, Evaluation Protocol=WILD2026.02 | 68.1 | 134 | 126 | 34.5 | 51.8 | 43 | 29.5 | — | — | — | — | |
| FT SequentialEvaluation Context=No Context, Time=10m 13s, Evaluation Protocol=WILD2026.02 | 68.1 | 18 | 15 | 34.5 | 51.8 | 43 | 29.5 | — | — | — | — | |
| LoRAEvaluation Context=QA Context, Time=58m 42s, Evaluation Protocol=WILD2026.02 | 67.8 | 300 | 258 | 70.7 | 55.4 | 48 | 75 | — | — | — | — | |
| LoRAEvaluation Context=No Context, Time=58m 42s, Evaluation Protocol=WILD2026.02 | 67.8 | 270 | 157 | 70.7 | 55.4 | 48 | 75 | — | — | — | — | |
| LoRA SequentialEvaluation Context=QA Context, Time=4h 54m, Evaluation Protocol=WILD2026.02 | 67.8 | 209 | 187 | 73.8 | 54.4 | 48 | 71 | — | — | — | — | |
| LoRA SequentialEvaluation Context=No Context, Time=4h 54m, Evaluation Protocol=WILD2026.02 | 67.8 | 79 | 73 | 73.8 | 54.4 | 48 | 71 | — | — | — | — | |
| UltraEditBackbone=Qwen-2.5-1.5B-Instruct2026.02 | 62.5 | — | — | 41.4 | 44.5 | 44.5 | 60 | 27.5 | 25.8 | 53.2 | 45.5 | |
| Adam-NSCLBackbone=Qwen-2.5-1.5B-Instruct2026.02 | 62.3 | — | — | 36.2 | 46.4 | 41.5 | 33 | 31.9 | 28.4 | 11.6 | 10.4 | |
| Qwen-2.5-1.5B-Instructstatus=Base Model (Unedited)2026.02 | 61.9 | — | — | 48.3 | 50.9 | 52 | 58 | 8.4 | 8.6 | 7 | 6.4 | |
| CrispEditExecution=Batch, Backbone=Qwen-2.5-1.5B-Instruct2026.02 | 61.2 | — | — | 38.7 | 47 | 45.5 | 58.5 | 62.9 | 52 | 57.3 | 46.3 | |
| LocBF-FTBackbone=Qwen-2.5-1.5B-Instruct2026.02 | 60.4 | — | — | 34.8 | 46.1 | 43.5 | 58 | 76.8 | 61.9 | 66 | 55.2 | |
| CrispEditExecution=Sequential, Backbone=Qwen-2.5-1.5B-Instruct2026.02 | 59.9 | — | — | 34 | 47.9 | 44.5 | 55 | 53.4 | 43.3 | 83.4 | 60 | |
| AlphaEditEvaluation Context=QA Context, Time=7h 37m, Evaluation Protocol=WILD2026.02 | 58.5 | 729 | 668 | 61.6 | 50.2 | 50.5 | 58 | — | — | — | — | |
| AlphaEditEvaluation Context=No Context, Time=7h 37m, Evaluation Protocol=WILD2026.02 | 58.5 | 739 | 683 | 61.6 | 50.2 | 50.5 | 58 | — | — | — | — | |
| FTBackbone=Qwen-2.5-1.5B-Instruct2026.02 | 54.3 | — | — | 30.7 | 46.2 | 39.5 | 52 | 59.5 | 50.4 | 42.2 | 37 | |
| MEMITEvaluation Context=QA Context, Time=10h 42m, Evaluation Protocol=WILD2026.02 | 24.6 | 0 | 0 | 13.6 | 52.3 | 23.5 | 0 | — | — | — | — | |
| MEMITEvaluation Context=No Context, Time=10h 42m, Evaluation Protocol=WILD2026.02 | 24.6 | 0 | 0 | 13.6 | 52.3 | 23.5 | 0 | — | — | — | — | |
| AlphaEditBackbone=Qwen-2.5-1.5B-Instruct2026.02 | 24.4 | — | — | 13.2 | 48.9 | 23.5 | 1 | 0.7 | 0.7 | 1.5 | 1.3 | |
| MENDEvaluation Context=QA Context, Time=38m 36s, Evaluation Protocol=WILD2026.02 | 22.9 | 0 | 0 | 18.2 | 0 | 26 | 0 | — | — | — | — | |
| MENDEvaluation Context=No Context, Time=38m 36s, Evaluation Protocol=WILD2026.02 | 22.9 | 0 | 0 | 18.2 | 0 | 26 | 0 | — | — | — | — | |
| GRACEN=1000 edits, Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | — | 63 | 7 | — | — | — | — | — | — | — | — | |
| HoReNN=1000 edits, Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | — | 96 | 79 | — | — | — | — | — | — | — | — | |
| REPAIRN=1000 edits, Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | — | 58 | 54 | — | — | — | — | — | — | — | — | |
| WISEN=1000 edits, Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | — | 47 | 38 | — | — | — | — | — | — | — | — |