Multi-hop Knowledge Editing on MQUAKE-CF-3K (100 edited)
56AccuracyPokeMQA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PokeMQABackbone=GPT-3.5-turbo-instruct, Model Size=Undisclosed, Prompting Style=Question decomposition2023.12 | 56 | 49.63 | |
| MeLLoBackbone=GPT-3.5-turbo-instruct, Model Size=Undisclosed, Prompting Style=Question decomposition2023.12 | 40.87 | 28.13 | |
| PokeMQABackbone=Vicuna, Model Size=7B, Prompting Style=Question decomposition2023.12 | 38.77 | 31.23 | |
| PokeMQABackbone=LLaMa-2, Model Size=7B, Prompting Style=Question decomposition2023.12 | 37.33 | 27.83 | |
| MeLLoBackbone=LLaMa-2, Model Size=7B, Prompting Style=Question decomposition2023.12 | 20 | 10.07 | |
| MeLLoBackbone=Vicuna, Model Size=7B, Prompting Style=Question decomposition2023.12 | 12.83 | 6.77 | |
| MEMITBackbone=LLaMa-2, Model Size=7B, Prompting Style=Question decomposition2023.12 | 9.4 | 2.47 | |
| MEMITBackbone=LLaMa-2, Model Size=7B, Prompting Style=Chain-of-thought2023.12 | 9.23 | — | |
| ROMEBackbone=LLaMa-2, Model Size=7B, Prompting Style=Question decomposition2023.12 | 3.5 | 0.03 | |
| ROMEBackbone=LLaMa-2, Model Size=7B, Prompting Style=Chain-of-thought2023.12 | 2.87 | — | |
| FTBackbone=LLaMa-2, Model Size=7B, Prompting Style=Question decomposition2023.12 | 2.37 | 0.03 | |
| FTBackbone=LLaMa-2, Model Size=7B, Prompting Style=Chain-of-thought2023.12 | 2.13 | — |