Knowledge Editing on MzsRE Edit: Portuguese, Test: EN
5,343ReliabilityReMaKE-few-bi
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReMaKE-few-biBackbone=BLOOMZ, Mode=few-shot bilingual (16 examples)2023.12 | 5,343 | 5,195 | 98.52 | 673 | |
| ReMaKE-zeroBackbone=BLOOMZ, Mode=zero-shot2023.12 | 3,271 | 3,136 | 98.52 | 1,157 | |
| BLOOMZBackbone=BLOOMZ, Mode=pre-editing2023.12 | 188 | 135 | — | 659 | |
| ReMaKEBackbone=LLaMA, Protocol=few-shot (16 examples)2023.12 | 90.52 | 89.62 | 98.26 | 48.41 | |
| ReMaKEBackbone=LLaMA, Protocol=zero-shot2023.12 | 76.91 | 72.87 | 98.54 | 34.56 | |
| ReMaKEBackbone=BLOOMZ, Protocol=few-shot (16 examples)2023.12 | 74.38 | 60.04 | 98.05 | 31.39 | |
| IKE2023.12 | 72.03 | 70 | 37.69 | 42.3 | |
| ReMaKEBackbone=BLOOMZ, Protocol=zero-shot2023.12 | 46.07 | 44.22 | 99.01 | 0.57 | |
| ROME2023.12 | 26.88 | 26.93 | 95.19 | 4.26 |