Knowledge Editing on MzsRE Edit: Russian, Test: EN
67.22ReliabilityReMaKE
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReMaKEBackbone=LLaMA, Protocol=few-shot (16 examples)2023.12 | 67.22 | 66.81 | 98.44 | 44.7 | |
| ReMaKE-few-biBackbone=BLOOMZ, Mode=few-shot bilingual (16 examples)2023.12 | 36.74 | 34.86 | 97.71 | 5.38 | |
| ReMaKEBackbone=BLOOMZ, Protocol=few-shot (16 examples)2023.12 | 35.62 | 33.86 | 97.78 | 41.94 | |
| ReMaKEBackbone=LLaMA, Protocol=zero-shot2023.12 | 33.64 | 32.44 | 99.42 | 31.81 | |
| ReMaKE-zeroBackbone=BLOOMZ, Mode=zero-shot2023.12 | 23.82 | 21.27 | 97.71 | 7.67 | |
| ReMaKEBackbone=BLOOMZ, Protocol=zero-shot2023.12 | 23.23 | 22.51 | 98.16 | 0.57 | |
| ROME2023.12 | 4.36 | 5.03 | 97.65 | 4.26 | |
| IKE2023.12 | 4.27 | 4.38 | 3.46 | 2.26 | |
| BLOOMZBackbone=BLOOMZ, Mode=pre-editing2023.12 | 1.88 | 1.35 | — | 6.59 |