Knowledge Editing on MzsRE Edit: EN, Test: EN
10,000ReliabilityIKE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| IKEBackbone=LLaMA, Edit Language=EN, Examples=16 monolingual2023.12 | 10,000 | 9,865 | 38.48 | 17.26 | — | |
| ReMaKE-few-monoBackbone=LLaMA, Edit Language=EN, Examples=16 monolingual2023.12 | 10,000 | 9,825 | 99.46 | 31.49 | — | |
| ReMaKE-few-biBackbone=LLaMA, Edit Language=EN, Examples=16 bilingual2023.12 | 10,000 | 9,825 | 99.46 | 31.49 | — | |
| ReMaKE-zeroBackbone=LLaMA, Edit Language=EN, shot_type=zero-shot2023.12 | 9,637 | 8,681 | 99.46 | 34.59 | — | |
| SERACBackbone=LLaMA, Edit Language=EN2023.12 | 9,125 | 2,678 | 100 | 0 | — | |
| ReMaKE-few-biSetting=16-shot bilingual, Backbone=BLOOMZ, Edit Language=English2023.12 | 7,120 | 6,581 | 9,919 | 781 | — | |
| ReMaKE-zeroSetting=zero-shot, Backbone=BLOOMZ, Edit Language=English2023.12 | 6,904 | 6,326 | 9,919 | 1,265 | — | |
| ROMEBackbone=LLaMA, Edit Language=EN2023.12 | 6,891 | 5,653 | 92.87 | 0 | — | |
| BLOOMZStatus=Pre-editing, Backbone=BLOOMZ2023.12 | 188 | 135 | — | 659 | — | |
| LLaMABackbone=LLaMA, Edit Language=EN, Status=Pre-editing2023.12 | 108 | 94 | — | 8.48 | — | |
| FT-MBackbone=Qwen2.5-7B-Instruct2025.02 | 100 | 99.35 | 92.44 | 52.59 | — | |
| LTEBackbone=Qwen2.5-7B-Instruct2025.02 | 99.78 | 99.28 | 87.64 | 74.23 | — | |
| X-KDEBackbone=Qwen2.5-7B-Instruct2025.02 | 99.72 | 99.56 | 88.79 | 73.96 | — | |
| ROMEBackbone=Qwen2.5-7B-Instruct2025.02 | 99.48 | 92.83 | 98.63 | 56.69 | — | |
| IKEBackbone=Qwen2.5-7B-Instruct2025.02 | 97.32 | 98.56 | 50.26 | 67.3 | — | |
| MEMITBackbone=Qwen2.5-7B-Instruct2025.02 | 96.77 | 89.3 | 98.78 | 55.6 | — | |
| ReMaKEBackbone=LLaMA, Protocol=few-shot (16 examples)2023.12 | 93.76 | 93.44 | 99.09 | 52.39 | — | |
| ReMaKEBackbone=LLaMA, Protocol=zero-shot2023.12 | 83.71 | 79.3 | 99.3 | 38.42 | — | |
| IKE2023.12 | 76.58 | 75.19 | 36.39 | 41.42 | — | |
| ReMaKEBackbone=BLOOMZ, Protocol=few-shot (16 examples)2023.12 | 73.49 | 72.21 | 98.8 | 34.3 | — | |
| FT-LBackbone=Qwen2.5-7B-Instruct2025.02 | 62.89 | 65.93 | 75.06 | 38.83 | — | |
| ReMaKEBackbone=BLOOMZ, Protocol=zero-shot2023.12 | 60.77 | 59.11 | 98.78 | 1.46 | — | |
| ROME2023.12 | 50.18 | 48.8 | 95.47 | 3.23 | — | |
| FT-LBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 54.98 | |
| FT-MBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 76.72 | |
| IKEBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 70.97 | |
| LTEBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 81.93 | |
| MEMITBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 76.57 | |
| ROMEBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 77.3 | |
| X-KDEBackbone=Qwen2.5-7B-Instruct2025.02 | — | — | — | — | 87.1 |