Knowledge Editing on MzsRE Edit: German, Test: EN
5,276ReliabilityReMaKE-few-bi
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReMaKE-few-biSetting=16-shot bilingual, Backbone=BLOOMZ, Edit Language=English2023.12 | 5,276 | 5,114 | 99.6 | 162 | |
| ReMaKE-zeroSetting=zero-shot, Backbone=BLOOMZ, Edit Language=English2023.12 | 3,459 | 3,324 | 99.6 | 229 | |
| ReMaKEBackbone=LLaMA, Protocol=few-shot (16 examples)2023.12 | 92.83 | 92.22 | 99.34 | 54.32 | |
| ReMaKEBackbone=LLaMA, Protocol=zero-shot2023.12 | 78.45 | 74.19 | 99.69 | 39.52 | |
| ReMaKEBackbone=BLOOMZ, Protocol=few-shot (16 examples)2023.12 | 74.5 | 73.86 | 98.96 | 34.83 | |
| IKE2023.12 | 72.94 | 71.77 | 36.18 | 41.74 | |
| ROME2023.12 | 66.78 | 65.62 | 96.06 | 4.72 | |
| ReMaKEBackbone=BLOOMZ, Protocol=zero-shot2023.12 | 58.36 | 56.71 | 98.91 | 0.85 | |
| ReMaKE-few-biBackbone=BLOOMZ, Mode=few-shot bilingual (16 examples)2023.12 | 55.99 | 54.91 | 98.38 | 7.27 | |
| BLOOMZStatus=Pre-editing, Backbone=BLOOMZ2023.12 | 40 | 27 | — | 135 | |
| ReMaKE-zeroBackbone=BLOOMZ, Mode=zero-shot2023.12 | 39.43 | 36.34 | 98.38 | 9.69 | |
| BLOOMZBackbone=BLOOMZ, Mode=pre-editing2023.12 | 1.88 | 1.35 | — | 6.59 |